An AI incident response process sets out what happens when an AI tool produces something wrong, biased, or harmful – before it happens, not after. Every governance framework needs one, because even a well-assessed, properly approved tool will eventually get something wrong.
What counts as an AI incident?
Broader than most people initially assume. Obvious cases include an AI tool producing a factually wrong output that gets acted on, or generating something offensive or discriminatory. Less obvious cases include a tool being used outside its approved purpose, staff relying on AI-generated content without the required human check, or a vendor changing how a tool processes data without notice. Treating only the dramatic cases as incidents means the quieter, more common ones go unrecorded.
Who should staff report an incident to?
One clear, well-communicated route – not “whoever seems relevant at the time.” Staff need to know exactly who to tell, and that reporting won’t land them in trouble for having used the tool in the first place. A reporting process people are afraid to use gets used far less than one that treats early reporting as good practice, not an admission of fault.
What should happen once an incident is reported?
- Contain – stop the immediate harm, which may mean pausing use of the tool
- Assess – work out what happened, what data or decisions were affected, and how many people it touches
- Notify – tell anyone who needs to know, including data subjects or the ICO if the incident is also a personal data breach
- Document – record what happened and the decisions made, regardless of severity
- Review – decide whether the tool can continue to be used, needs new conditions, or should be withdrawn
How does this connect to data breach response?
Where an AI incident involves personal data, it may also be a personal data breach requiring the usual UK GDPR notification steps. The two processes shouldn’t run separately – an AI incident response process should point directly to the organisation’s data breach procedure the moment personal data is involved, rather than treating them as unconnected.
If you’d like help building an AI incident response process, see our AI Governance advisory services.