AI risk management: the risks that matter, and how to control them

What makes AI risk different

Most organizations already run riskriskIn the EU AI Act's terms, the combination of the probability that a harm occurs and the severity of it if it does. The link between a principle (via the harm that would breach it) and a control (the measure that reduces it). Naming the harm and assessing its risk is required by Art. 9 before any mitigation measure is chosen. See harm, control, residual risk.Open full entry → management. The reason AI needs its own treatment is not that the discipline changes, it is that AI breaks four assumptions the discipline was built on.

AI is probabilistic. The same system, given the same kind of input, does not always produce the same answer. A classic application is either correct or defective; a model is right some percentage of the time, and that percentage is the honest way to describe it. "Tested and approved" in the traditional sense, one test pass before release, does not exist for a system whose behavior is a distribution.

AI learns from data. Whatever is in the data becomes behavior. A hiring model trained on years of skewed hiring decisions does not contain a line of biased code; it contains biased history, faithfully learned. The risk does not enter through the build, it enters through the past.

AI changes after go-live. The world shifts, the data shifts with it, and a model that was accurate at launch quietly degrades. This is driftdriftThe gradual divergence of an AI system's behavior or performance from its validated state after deployment, without any code change or error. Drift is silent by nature: nothing breaks, accuracy simply decays until someone measures it. See model drift, agent drift.Open full entry →, and its defining feature is that nothing breaks: no error, no crash, just decisions that get gradually worse until someone measures them. A system can fail an audit it passed a year earlier without a single change to the system.

AI operates at scale and speed. One flawed design decision does not produce one flawed outcome, it produces thousands, before any human has reviewed the first one. Agentic AIagentic AISystems where a model takes actions (calling tools, executing multi-step plans), amplifying both capability and every failure mode; governed with action allowlists, approvals and full logging.Open full entry → raises this further, because the system no longer waits for a human between steps: what changes then is worked out in the agentic AI cornerstone article.

Together these four explain why AI risk management cannot be a gate at go-live. It is continuous work: the risk profile of an AI systemAI systemA machine-based system that, for explicit or implicit objectives, infers from input how to generate outputs (predictions, content, recommendations or decisions) that can influence physical or virtual environments. The OECD-style definition followed by the EU AI Act.Open full entry → is not a property you establish once, it is a state you have to keep measuring.

The pattern: recognize, assess, control

Every risk in this article is handled with the same three moves.

Recognize means knowing what the risk looks like in the wild, not in a definition. Most AI risks announce themselves in patterns: a complaint cluster, an approval rate that never dips, an output distribution that leans one way.

Assess means answering two questions for your specific system, not for AI in general: how likely is this here, and how bad is it if it happens here. A biased ranking in an internal document search is a nuisance; the same biasbiasA systematic skew in data, model behavior, or outcomes that treats one group differently from another without justification. Bias usually enters through training data that reflects historical patterns. For high-risk AI systems, Article 10 of the EU AI Act requires examination of datasets for possible biases and measures to detect, prevent, and mitigate them. See fairness, proxy discrimination.Open full entry → in cv screening changes people's lives. Same risk type, different risk.

ControlcontrolThe concrete, testable measure that reduces a specific risk, and through that risk protects the principle behind it. Also called a risk management measure, risk response, or risk treatment. Always traceable to the risk it addresses: under EU AI Act Art. 9 every control must map back to a specific risk, and controls recorded separately from their risks is a recognized compliance failure. It works in one of three types: preventive, detective, or corrective. See risk, control types, evidence.Open full entry → means choosing measures that reduce the risk and can be tested. Controls come in the three types any auditor recognizes: preventive controls that stop the harmharmThe concrete damage an AI system can do that a responsible-AI principle exists to prevent: in the EU AI Act's terms, harm to a person's health, safety, or fundamental rights. Harm is the bridge between an abstract principle and a governable risk; governance becomes operational the moment an organization names the specific harms it wants to prevent. For fairness, a harm is a group receiving systematically worse outcomes because of a characteristic that should not have counted. See principle, risk.Open full entry → before it occurs, detective controls that reveal it while the system runs, and corrective controls that limit the damage and feed the lesson back.

This is the same chain that runs through the whole governancegovernanceThe system through which an organization steers itself: corporate governance, risk management, compliance, lines of accountability, risk appetite, and the operating model. It exists across everything the organization does, before and beyond AI. AI governance is this same system extended for AI. See AI governance, governance design, execution level.Open full entry → model, from principleprincipleOne of the seven responsible-AI values a governed system should live up to (fairness, safety and reliability, privacy, security and robustness, transparency and explainability, accountability, human oversight). A principle is abstract: it states an outcome, not a lever you can pull. It becomes governable by naming the harm that would breach it, assessing the risk that harm carries, and placing controls against that risk. Held this way, a principle becomes a pillar. See pillar, harm, risk.Open full entry → to evidenceevidenceThe concrete proof that a control is designed, implemented, and working: a test report, an audit trail, an impact assessment, a monitoring log. Each link in the governance chain produces an artifact, and together they are what an organization hands to its own board, a regulator, a customer, or an affected person to show, not say, that a system is governed. Its absence is itself the failure: a risk register without test results, or a mitigation claimed without validation, is a governance gap, not a paperwork one. The closing link of the governance chain. See control, governance.Open full entry →, applied risk-first. The EU AI ActEU AI ActRegulation (EU) 2024/1689, the European Union's law on artificial intelligence. It takes a risk-based approach: prohibited practices, requirements for high-risk AI systems, transparency obligations for specific uses, and a separate regime for general-purpose AI models. Obligations are divided between providers and deployers. See general-purpose AI, conformity assessment.Open full entry → requires exactly this discipline for high-risk systems: Article 9 of Regulation (EU) 2024/1689 requires a risk management system that runs as a continuous iterative process throughout the entire lifecycle of the system, with regular systematic review and updating. The obligation sits with the providerproviderThe actor who develops an AI system (or has it developed) and places it on the market or into service under its own name. It carries manufacturer-style duties: design controls, documentation, conformity.Open full entry →, and the risks in scope are those to health, safety, and fundamental rights. The full cycle, with the principle as its anchor, is laid out in the model.

The risks, pillar by pillar

Each of the seven pillarspillarA responsible-AI principle as something an organization actively holds rather than merely endorses: one of the seven pillars of responsible AI, one per principle. A pillar is held, not implemented, by naming the harms that would breach the principle, assessing their risk, and placing controls that reduce it. Distinct from agentic AI, which is not one of the seven but a condition that changes how all of them are governed. See principle, harm, risk, agentic AI.Open full entry → is a promise an AI system can break. This section walks through how each one breaks in practice. For the full control tables with law references, each section links to the pillar page.

Fairness

The risk: the system systematically disadvantages a group. It rarely does so openly. The classic route is historical data, a model that learns from years of skewed decisions reproduces the skew. The subtler route is proxy discriminationproxy discriminationDiscrimination that persists after protected attributes are removed, because other variables (postcode, shopping patterns) stand in for them.Open full entry →: the model never sees a protected characteristic, but learns a stand-in for it, a postal code, a career gap, a school name.

Recognize it in outcome distributions that lean one way across groups, in complaint patterns, in a rejection rate for one population that no business reason explains. Assess it by asking which groups this system touches, whether the training history contains skew, and what one wrong decision costs the person on the receiving end. Control it with a representativenessrepresentativenessHow well training data reflects the population and conditions the system will face in deployment, the fitness-for-purpose core of AI data quality.Open full entry → check on the training datatraining dataThe data used to fit an AI model's parameters; its quality, lawful rights and representativeness are central governance concerns.Open full entry → before release, bias testing across groups once the system is live, and a threshold with a suspend-and-retrain route when it is breached. Full control table: see the fairness pillar page.

Safety and reliability

The risk: the system fails, or confidently makes things up, at a moment that matters. And separately: the system that worked at launch degrades. Drift is the defining reliability risk of AI because it is silent, accuracy decays without an error being thrown, and the organization keeps trusting numbers that stopped being true.

Recognize it in rising error rates, in users quietly working around the system, in outcomes that surprise the people who know the domain. Assess it by asking what a wrong output causes here, who catches it and how fast, and how far accuracy can fall before harm starts. Control it with acceptance thresholds set before go-live, performance monitoring with alerting against those thresholds in operation, and a defined fallback to human handling when the system falls below them. Full control table: see the safety and reliability pillar page.

Privacy

The risk: personal data ends up where it should not. In training data that was never meant for this purpose, in model outputs that reproduce what the model memorized, in logs nobody thought of as a data store. And the quieter risk: purpose creep, a system built for one use sliding into another that its data basis never covered.

Recognize it in personal data appearing in outputs or logs, and in use cases that have drifted from what was originally assessed. Assess it by asking what personal data enters the system, whether the model can reproduce it, and whether this processing would trigger a data protection impact assessmentimpact assessmentA structured evaluation, carried out in the plan-and-design stage, of the harms an AI system could cause and the risk those harms carry, before the system is built. The first place the governance chain is run, and the cheapest point in the life cycle to reduce risk. The anchor artifact of the planning stage; under the EU AI Act, a fundamental-rights impact assessment is required for certain high-risk deployers. See harm, risk, life cycle.Open full entry → under Article 35 GDPRGDPRRegulation (EU) 2016/679, the General Data Protection Regulation, the EU's law on the processing of personal data. It applies to AI wherever personal data enters training, inputs, outputs, or logs, and it operates alongside the EU AI Act rather than being replaced by it. See controller, processor, lawful basis, DPIA.Open full entry →. Control it with data minimizationdata minimizationProcessing only data that is adequate, relevant and necessary. In ML it is implemented through pseudonymization, feature selection, synthetic data and privacy-enhancing techniques.Open full entry → before training, output scanning for personal data in operation, and a working deletion route when data is found where it should not be. Full control table: see the privacy pillar page.

Security and robustness

The risk: someone manipulates the system into doing what it should not. AI adds attack surfaces classic security does not cover: prompt injectionprompt injectionSmuggling adversarial instructions into a generative system's input (directly or via retrieved content) to override its intended behavior.Open full entry → that turns instructions inside the input into behavior, data poisoningdata poisoningAn attack that corrupts training data so the model learns attacker-chosen behavior; a core adversarial-ML threat to the data pipeline.Open full entry → that plants tomorrow's failure in today's training set, adversarial inputsadversarial inputAn input deliberately crafted to make an AI system produce a wrong or harmful output, often through changes a human would not notice. Adversarial inputs are a core attack type under the security and robustness pillar and a standard target of pre-release testing. See red teaming, prompt injection.Open full entry → crafted to flip an output. The governance question is not the technical depth of each attack, that is security engineering territory, but whether the risk is owned, assessed, and evidenced like every other risk.

Recognize it in anomalous input patterns and in output changes nobody can explain. Assess it by asking who can reach the system, what manipulating it would buy an attacker, and how you would notice. Control it with input validation, adversarial testing before and after release, and access control on the model and its data. Full control table: see the security and robustness pillar page.

Transparency and explainability

The risk: nobody can explain why the system decided what it decided. That becomes concrete the day a customer, a regulator, or a court asks. A decision that cannot be reconstructed cannot be defended, and "the model decided" is not an explanation, it is the absence of one.

Recognize it in answers that stop at "that is what the system said," and in decisions with no logging behind them. Assess it by asking who has to be able to explain this system's decisions, to whom, and at what level of detail, an internal reviewer needs something different than an affected person. Control it with explainabilityexplainabilityThe ability to give a meaningful reason for a specific output of an AI system to the people it affects. It is distinct from transparency, which is disclosure that and how AI is used.Open full entry → requirements set per use case before build, decision logging in operation, and a route that produces a meaningful explanation on request. Full control table: see the transparency and explainability pillar page.

Accountability

The risk: the system has no owner. Decisions are made and nobody carries them. The confusion is often structural: the provider built the model, the deployerdeployerAn organization using an AI system under its own authority in its activities. It carries the operator duties: use per instructions, oversight, input relevance, monitoring, notices.Open full entry → runs it, and each assumes the other holds the obligation. When something goes wrong, "the system did it" turns out to be the organization's actual position, and that position fails the moment anyone tests it.

Recognize it in an inventory entry with no name attached, in provider and deployer obligations nobody has written down, in questions about a system that get forwarded until they disappear. Assess it by asking who carries which obligation for this system, whether that is recorded, and whether the named person knows. Control it with a named owner for every system, the provider and deployer split fixed contractually, and an audit trail that ties decisions to the mandate they ran under. Full control table: see the accountability pillar page.

Human oversight

The risk: oversight that exists on paper and has stopped existing in practice. A human is formally in the loop, but reviews so many system decisions that reviewing collapses into confirming. Automation biasautomation biasThe human tendency to over-trust automated outputs: accepting a system's recommendation without genuinely weighing the case, which hollows out human oversight.Open full entry → does this quietly: the approval rate climbs toward 100 percent and everyone reads that as the system performing well, when it equally describes oversight that has gone to sleep. The second risk: oversight that could not intervene even if it wanted to, no mandate, no time, no route.

Recognize it in approval rates that never dip, in reviewers who cannot recall the last time they overruled the system, in escalation routes that exist in a document and nowhere else. Assess it by asking whether the human can meaningfully evaluate the decision, has the authority and time to reject it, and whether rejections happen in practice. Control it with an override rate that is tracked as a KPI, low is a warning, not a compliment, escalation routes that are tested, and a periodic check that oversight still functions as designed. For what changes when the system acts on its own, see the agentic AI cornerstone article. Full control table: see the human oversight pillar page.

Making it operational

The seven sections above describe the work per risk. What makes it a practice rather than a one-time exercise is the machinery around it.

The register comes first. Every AI system the organization builds, buys, or embeds, including the AI quietly switched on inside SaaS tools, is recorded with a named owner and a risk tier. You cannot manage the risk of a system you do not know you have, and in most organizations the register's first draft is a discovery exercise, not an administrative one.

Assessment has a cadence. High-risk systems get the full treatment: the recognize, assess, control pattern applied per relevant pillar. And the assessment is repeated, on every material change, a new model version, new data, a new use case, and periodically even without one, because drift does not wait for a change request.

The roles are the three lines. The first line runs the system and owns its risk. The second line challenges the assessment, is the bias test good enough. The third line audits independently: not whether the paperwork exists, but whether the controls operate. The judgment on residual riskresidual riskThe risk that remains after controls have reduced it. No control reduces a risk to zero, and not every control is worth its cost, so a deliberate judgment is made: whether the cost of further control is justified by the reduction it would buy, and whether the remaining risk is acceptable against the organization's risk appetite. This is a design-level judgment, where execution reports back up and governance accepts the residual risk, calls for more control, or declines the use case. EU AI Act Art. 9(5) requires it to be judged acceptable per hazard and overall. See risk, control, risk appetite.Open full entry →, is what remains after the controls acceptable, belongs to the design level, measured against the AI risk appetiterisk appetiteThe level of risk an organization's leadership is willing to accept in pursuit of its objectives, set at the governance design level. It is the benchmark against which residual risk is judged acceptable or not, inherited from the organization's broader governance and applied to AI. A concept from enterprise risk management (COSO ERM) before it is an AI one. See residual risk, governance design.Open full entry → the organization set.

Every step leaves evidence. The register without test results is a list, not risk management. Each assessment, each control test, each residual-risk decision produces an artifactartifactThe concrete record that proves a control was carried out: a test report, an impact assessment, a monitoring log, a release sign-off. An artifact is the tangible form evidence takes, the thing an auditor reaches for to confirm that a control was not just designed but actually operated. Each stage of the AI life cycle produces its own anchor artifact. Distinct from evidence as a whole: evidence is the proof, an artifact is one piece of it. See evidence, life cycle.Open full entry →, and those artifacts are what turn "we manage AI risk" from a claim into something the organization can show.