An AI risk and impact assessment identifies what could go wrong with an AI tool before it goes into regular use, and what controls are needed to manage that risk. It sits alongside (and often overlaps with) a Data Protection Impact Assessment, but covers a wider set of concerns: accuracy, bias, explainability, and business impact, not just personal data. Here’s a practical walkthrough.
When should an AI risk assessment be carried out?
Before any new AI tool is approved for regular use, and again whenever an existing tool’s use changes materially – a new data source, a new output type, or a new decision it feeds into. A tool used purely for internal drafting carries different risk to one that scores applicants or filters customer complaints, so the depth of assessment should scale with what the tool actually does.
What does the assessment need to cover?
A working assessment typically works through:
- Purpose – what the tool is being used for, and what decision or output it produces
- Data flow – what data goes in, where it comes from, and whether it includes personal or special category data
- Accuracy and reliability – how the output is checked, and what happens if it’s wrong
- Bias – whether the tool could produce systematically different outcomes for different groups, and how that would be detected; fairness and bias mitigation are a specific focus of the ICO’s guidance on AI and data protection
- Human oversight – who reviews the output, and whether a human can meaningfully override it
- Vendor and contract terms – what the provider does with input data, and whether liability is addressed
Who should be involved?
The person requesting the tool rarely has the full picture alone. A useful assessment usually draws in whoever owns data protection compliance, the team that will actually use the tool day to day, and someone senior enough to sign off the residual risk. For higher-risk tools, that sign-off should sit with someone accountable at board or senior management level, not left to the requesting team.
What happens with the output of the assessment?
The assessment should end in a clear decision: approve, approve with conditions, or reject. If approved, it feeds two other pieces of the governance framework – the tool gets added to the system register, and any conditions (data restrictions, review frequency, escalation route) are documented and assigned an owner. If rejected, the reasoning should be recorded too, since the same tool is likely to be proposed again.
If you’d like support building an AI risk assessment process for your organisation, see our AI Governance advisory services.