What is AI governance
Understand and design
AI governance is the system through which an organization makes responsible use of its AI and can prove it. It is not an ethics statement or a one-time project but an operating discipline that runs for the full life of every AI system the organization builds, buys, or embeds, carrying each responsible-AI principle down a chain from the harm that would breach it, through the risk and the control that reduces it, to the evidence that the control works.
Understand AI governance in five reads
This article sets out the operating logic of AI governanceAI governanceGovernance extended for AI: the same organizational steering at the highest level, widened to cover what makes AI different (it works in probabilities rather than fixed rules, learns from data, and can act at a speed and scale no human reviewer can match). It inherits the existing governance structure and brings AI inside the disciplines the organization already runs, rather than creating a parallel system in a silo. It operates on two levels, design and execution. See governance, governance design, execution level, responsible AI.Open full entry → in full: what governancegovernanceThe system through which an organization steers itself: corporate governance, risk management, compliance, lines of accountability, risk appetite, and the operating model. It exists across everything the organization does, before and beyond AI. AI governance is this same system extended for AI. See AI governance, governance design, execution level.Open full entry → is, how it extends to AI, and the chain that runs from a responsible-AI principleprincipleOne of the seven responsible-AI values a governed system should live up to (fairness, safety and reliability, privacy, security and robustness, transparency and explainability, accountability, human oversight). A principle is abstract: it states an outcome, not a lever you can pull. It becomes governable by naming the harm that would breach it, assessing the risk that harm carries, and placing controls against that risk. Held this way, a principle becomes a pillar. See pillar, harm, risk.Open full entry → down to the evidenceevidenceThe concrete proof that a control is designed, implemented, and working: a test report, an audit trail, an impact assessment, a monitoring log. Each link in the governance chain produces an artifact, and together they are what an organization hands to its own board, a regulator, a customer, or an affected person to show, not say, that a system is governed. Its absence is itself the failure: a risk register without test results, or a mitigation claimed without validation, is a governance gap, not a paperwork one. The closing link of the governance chain. See control, governance.Open full entry → that a specific controlcontrolThe concrete, testable measure that reduces a specific risk, and through that risk protects the principle behind it. Also called a risk management measure, risk response, or risk treatment. Always traceable to the risk it addresses: under EU AI Act Art. 9 every control must map back to a specific risk, and controls recorded separately from their risks is a recognized compliance failure. It works in one of three types: preventive, detective, or corrective. See risk, control types, evidence.Open full entry → works.
Three terms run through all of this and are easy to confuse: ethics, responsible AIresponsible AIThe set of principles an AI system should live up to: fairness, safety and reliability, privacy, security and robustness, transparency and explainability, accountability, and human oversight. Widely shared and sitting under the EU AI Act and the major frameworks. On their own the principles are statements of intent; the law turns them into duties that cannot be met unless they are carried inside the organization's governance, which is how responsible AI lands in governance rather than beside it. The seven principles are organized into seven pillars, one pillar per principle. See principle, pillar, governance. The seventh principle carries two names in practice: human oversight in the seven-pillar model, and human-centricity in the IAPP AIGP body of knowledge; the substance overlaps.Open full entry →, and governance. They sit in a natural order, from the broadest to the most operational. Ethics asks what it means to treat people well. Responsible AI makes that question specific to AI. It turns broad values into principles a system can be held to: fairnessfairnessThe responsible-AI principle that systems should not create or reinforce unjust discrimination; operationalized through bias testing, representative data and per-group thresholds. It has multiple, mutually incompatible mathematical definitions. Under the EU AI Act, providers of high-risk AI systems must examine their data sets for possible biases (Article 10), and several discriminatory uses are prohibited outright (Article 5). See bias, proxy discrimination, high-risk AI system, responsible AI.Open full entry →, safety, privacyprivacyThe principle that personal data used by or produced through an AI system stays within the purpose and the legal basis it was collected for. Three routes cause most of the trouble: personal data in training material that was never intended for it, model output that reproduces what the model retained, and purpose creep, where a system built for one use drifts into another the original basis never covered. The GDPR governs this in full, and the EU AI Act adds data governance duties for high-risk systems (Article 10). See DPIA, purpose limitation, responsible AI.Open full entry →, and the rest. AI governance is the most operational of the three, the system through which an organization actually delivers those principles and proves it.
Ethics gives the values, responsible AI gives the principles, governance makes them real. This article is about governance, and how it carries the principles.
The values
Ethics
What it means to treat people well.
The principles
Responsible AI
Seven principles an AI system can be held to.
The system
AI governance
Delivers the principles, and proves it. This article.
Governance, and how AI governance extends it
Governance, in its broad sense, is the system an organization already has for steering itself: corporate governance, riskriskIn the EU AI Act's terms, the combination of the likelihood that a harm occurs and the severity of it if it does. The link between a principle (via the harm that would breach it) and a control (the measure that reduces it). Naming the harm and assessing its risk is required by Art. 9 before any mitigation measure is chosen. See harm, control, residual risk.Open full entry → management, compliance, the lines of accountabilityaccountabilityThe principle that a named human or organization answers for an AI system's outcomes, through ownership, documentation, audit trails and redress; never the system itself. The EU AI Act attaches obligations to the role rather than the technology, with provider duties in Article 16 and deployer duties in Article 26, supported by technical documentation (Article 11) and record-keeping (Article 12). See provider, deployer, record-keeping, responsible AI.Open full entry →, the risk appetiterisk appetiteThe level of risk an organization's leadership is willing to accept in pursuit of its objectives, set at the governance design level. It is the benchmark against which residual risk is judged acceptable or not, inherited from the organization's broader governance and applied to AI. A concept from enterprise risk management (COSO ERM) before it is an AI one. See residual risk, governance design.Open full entry → the board sets, the operating model, and more. It exists before any AI systemAI systemA machine-based system that, for explicit or implicit objectives, infers from input how to generate outputs (predictions, content, recommendations or decisions) that can influence physical or virtual environments. The OECD-style definition followed by the EU AI Act.Open full entry → is switched on, and it governs everything the organization does.
AI governance is not a separate system that runs alongside the existing one. It is the same governance system at the highest level, extended for AI. What makes AI need that extension is that it behaves differently from the systems governance was built for: AI works in probabilities rather than fixed rules, it learns from data, and it can act at a speed and scale no human reviewer can match. General-purpose and generative AIgenerative AIAI systems that produce new content (text, images, audio, code) rather than only classifying or predicting. Large language models are the prominent example.Open full entry → sharpen this further: one bought model can surface in dozens of use cases, each with its own risk.
Extending governance means bringing AI inside the disciplines the organization already runs: AI risk into the risk management framework, the EU AI ActEU AI ActRegulation (EU) 2024/1689, the European Union's law on artificial intelligence. It takes a risk-based approach: prohibited practices, requirements for high-risk AI systems, transparency obligations for specific uses, and a separate regime for general-purpose AI models. Obligations are divided between providers and deployers. See general-purpose AI, conformity assessment.Open full entry → and related obligations into the compliance scope, AI data use into data protection and security. The alternative is a parallel rulebook in a silo the governance function never sees. The NIST AI Risk Management Framework and ISO/IEC 42001ISO/IEC 42001The international requirements standard for AI management systems, published in 2023 and certifiable. It defines how an organization establishes, implements, maintains, and continually improves a management system for AI. Certification against ISO/IEC 42001 does not create a legal presumption of conformity with the EU AI Act. See AI management system, harmonized standard.Open full entry → are both explicit on this point: governance is a broad, cross-cutting organizational layer, not a technical add-on.
That extension is not a single act. Like the governance it builds on, it runs at two levels: design and execution.
Two levels: design and execution
A governance system operates at two levels, and almost every governance question belongs to one of them. AI governance is not a third level added on top; it is these same two levels, each extended to reach AI. Keeping them apart is the first thing to get right, because they fail in very different ways.
Design: setting the rules
Design is the strategic level, set deliberately and reviewed periodically, and it answers one question: how do we govern AI as an organization? It sets the rules, in policy, the roles, the operating model, and the risk appetite the board fixes for AI. In practice that means requiring every AI system to be classified by risk, defining how that classification is done, and writing down who carries which obligations, the providerproviderThe actor who develops an AI system (or has it developed) and places it on the market or into service under its own name. It carries manufacturer-style duties: design controls, documentation, conformity.Open full entry → and the deployerdeployerAn organization using an AI system under its own authority in its activities. It carries the operator duties: use per instructions, oversight, input relevance, monitoring, notices.Open full entry → among them, with the minimum requirements each must meet. Design says what must happen and who is accountable; on its own it touches no single system.
The building blocks the organization sets and reviews
Policy, procedures & processes
The rules every AI system follows, including how each system is classified by risk.
Roles & accountability
Who carries which obligation, the provider and the deployer among them.
Culture
A risk-aware culture and a visible leadership commitment.
Operating model
The structure and the three lines of defense.
AI risk appetite
How much AI risk the organization is willing to accept.
Supplier & third-party controls
The contractual frame for external AI: audit rights, incident notification, exit terms, due-diligence requirements.
Execution: carrying it out
Execution is the continuous work that makes the design real, and it answers a different question: are we doing what we designed, and does it work? It is where the rules meet actual systems: building the inventory, classifying each one, assessing the high-risk systems, testing the controls, and monitoring them across the full life of every system. Findings feed back up and the design adjusts, so the two form a loop rather than a one-way handover.
The continuous building blocks that make the design real
Inventory & classify
Map every AI system and give each a risk tier: the triage across the whole portfolio.
Assess each system
Run the full risk cycle on the high-risk systems the triage surfaces: harm, risk, control, evidence.
Monitor & surveil
Watch live systems for drift and emerging harm.
Internal audit & review
Independent challenge and management review.
Evidence & improvement
Produce the proof, and feed findings back into the design.
Supplier assessment & monitoring
Assess vendors on onboarding, reassess on model change, and keep evidence they meet the responsible-AI objectives.
Why the principles land in governance
Responsible AI has converged on seven principles: an AI system should be fair, safe and reliable, private, secure, transparent and explainable, accountable, and under human oversighthuman oversightDesigned-in human ability to monitor, intervene in, override or shut down an AI system. It is meaningful only when the human has authority, information and time to act. One of the seven pillars of responsible AI, and under the EU AI Act a requirement for high-risk AI systems: Article 14 requires that those systems are designed so natural persons can effectively oversee them. Oversight that exists on paper but amounts to confirming in practice does not meet that bar. See override rate, automation bias, high-risk AI system, fairness, safety and reliability, privacy, security and robustness, transparency and explainability, accountability, responsible AI. In the IAPP AIGP body of knowledge, this principle appears as human-centricity, with human oversight as one of its elements.Open full entry →. These principles are widely shared, and versions of them appear in the EU AI Act, the OECD AI PrinciplesOECD AI PrinciplesThe intergovernmental principles for trustworthy AI adopted by the OECD in 2019 and updated in 2024. They set value-based expectations such as transparency, accountability, and human-centered values, and have shaped later frameworks and legislation. See responsible AI.Open full entry →, and the NIST AI RMFNIST AI RMFThe AI Risk Management Framework of the US National Institute of Standards and Technology, published as version 1.0 in 2023. It is a voluntary framework built around four functions: govern, map, measure, and manage. In a layered setup, it serves as the risk method inside a management system such as ISO/IEC 42001. See ISO/IEC 42001, ISO/IEC 23894.Open full entry →. On their own, though, they are statements of intent, and intent is not where governance starts. Something has to make them bind.
The law is what does it. The EU AI Act does not order an organization to adopt a set of principles, and it mandates no particular governance structure. Instead it imposes concrete obligations that can only be met if the principles already live inside the organization's governance. Risk management for high-risk systems falls under Article 9; human oversight under Articles 14 and 26; further duties cover transparencytransparencyOpenness about the fact that AI is used and how it operates in general: disclosures, documentation, notices. Pairs with explainability, which addresses individual outcomes.Open full entry →, data quality, and record-keepingrecord-keepingThe EU AI Act obligation for high-risk AI systems to allow automatic recording of events over the system's lifetime, laid down in Article 12. Deployers must keep the logs under their control for a period appropriate to the system's purpose, at least six months, under Article 26(6). Logs are what make decisions reconstructable afterward. See evidence, human oversight.Open full entry →. Each obligation is a principle turned into a duty: you cannot deliver meaningful human oversight unless the oversight principle lives in your roles and policies, and you cannot satisfy the risk management duty unless fairness, safety, and the rest are something your risk framework actually assesses.
So the principles do not sit next to governance as an ethics statement. The law pushes them into it: into the policies, the risk management framework, the roles, the risk appetite. That is the design work described above, now with a reason behind it. And once a principle lives in governance, it still has to be made real for each individual system. That is the chain.
The chain: from principle to evidence
Those seven principles are organized into seven pillars, one pillarpillarA responsible-AI principle as something an organization actively holds rather than merely endorses: one of the seven pillars of responsible AI, one per principle. A pillar is held, not implemented, by naming the harms that would breach the principle, assessing their risk, and placing controls that reduce it. Distinct from agentic AI, which is not one of the seven but a condition that changes how all of them are governed. See principle, harm, risk, agentic AI.Open full entry → per principle. The way a single pillar is held for a single system is a governance chaingovernance chainThe traceable line by which a single pillar is held for a single system: principle, then the harm that would breach it, then the risk that harm carries, then the control that reduces the risk (preventive, detective, or corrective), then the residual risk judged against appetite, proven with evidence. The chain is what makes responsible AI accountable rather than aspirational, and what lets an organization move a principle from a policy statement to a working control it can point to. See principle, harm, risk, control, residual risk, evidence.Open full entry →, and it is what moves a principle from a policy statement to a working control: not "we are committed to fairness" but "here is the control that reduces this specific fairness risk, here is the test that shows it works, here is who owns it." The chain follows the recognized risk management cycle, and it runs in the steps below.
Risk & control · anchored in the principle
Identify the harm, assess the risk, treat it with a control, monitor the evidence, then review. The principle anchors the cycle.
From principle to harm
A principle on its own cannot be controlled directly. Take fairness: it is the outcome you want, not a lever you pull. Principles are abstract; harmharmHarm is the concrete damage an AI system causes or can cause: to a person, a group, an organization, or society. A risk is that same damage seen in advance, weighed by likelihood and severity; a harm that has occurred is remedied rather than managed.Open full entry → is concrete, and the EU AI Act is built on exactly this move: it concerns the ways an AI system can harm people's health, safety, or fundamental rights. So the first thing governance does with a principle is ask what harm would breach it. For fairness that harm is specific: a group of people receives systematically worse outcomes because of a characteristic that should not have counted, a loan model that rejects one demographic at a higher rate for reasons unrelated to creditworthiness.
From harm to risk
A harm becomes workable once you know how likely it is and how serious it would be. That is the risk: in the Act's terms, the combination of the probability that the harm occurs and the severity of it if it does. Assessing it is what turns "be fair" into something an organization can work on, and the Act requires this step before any measure is chosen.
From risk to control
Only now does a control enter: a concrete, testable measure that reduces a specific risk, and through that risk protects the principle behind it. Controls work in one of three ways that any auditor recognizes: preventive controls stop the harm before it occurs, detective controls reveal it through testing, logging, and monitoring, and corrective controls limit the damage and feed the lesson back into prevention. For the fairness risk above: a representativenessrepresentativenessHow well training data reflects the population and conditions the system will face in deployment, the fitness-for-purpose core of AI data quality.Open full entry → check on the training datatraining dataThe data used to fit an AI model's parameters; its quality, lawful rights and representativeness are central governance concerns.Open full entry → (preventive), biasbiasA systematic skew in data, model behavior, or outcomes that treats one group differently from another without justification. Bias usually enters through training data that reflects historical patterns. For high-risk AI systems, Article 10 of the EU AI Act requires examination of datasets for possible biases and measures to detect, prevent, and mitigate them. See fairness, proxy discrimination.Open full entry → testing across demographic groups once the model is live (detective), and a route to suspend and retrain when a threshold is breached (corrective). A preventive control alone is rarely enough. Controls should be traceable to the risks they address. Article 9 requires targeted risk management measures in response to the risks identified and evaluated for the system.
Residual risk closes the loop
No control reduces a risk to zero, and not every control is worth its cost. What is left after the controls are in place is the residual riskresidual riskThe risk that remains after controls have reduced it. No control reduces a risk to zero, and not every control is worth its cost, so a deliberate judgment is made: whether the cost of further control is justified by the reduction it would buy, and whether the remaining risk is acceptable against the organization's risk appetite. This is a design-level judgment, where execution reports back up and governance accepts the residual risk, calls for more control, or declines the use case. EU AI Act Art. 9(5) requires it to be judged acceptable per hazard and overall. See risk, control, risk appetite.Open full entry →, and a deliberate judgment has to be made: is it acceptable against the organization's risk appetite? That judgment belongs to the design level, where governance accepts the residual risk, calls for more control, or declines the use case. The chain closes here and loops: the residual-risk judgment feeds the design, which shapes the next round of execution. And like every other step, the judgment counts only if it can be shown to have been made.
Evidence is the point
What separates governance from good intentions is evidence, and evidence is something concrete. It is the test report showing the bias check ran and what it found, the audit trail recording who approved a high-risk system going live, the impact assessmentimpact assessmentA structured evaluation, carried out in the plan-and-design stage, of the harms an AI system could cause and the risk those harms carry, before the system is built. The first place the governance chain is run, and the cheapest point in the life cycle to reduce risk. The anchor artifact of the planning stage; under the EU AI Act, a fundamental-rights impact assessment is required for certain high-risk deployers. See harm, risk, life cycle.Open full entry → documenting which harms were considered and how they were addressed, the monitoring log proving the system was watched after launch, not just before. Each link in the chain produces an artifactartifactThe concrete record that proves a control was carried out: a test report, an impact assessment, a monitoring log, a release sign-off. An artifact is the tangible form evidence takes, the thing an auditor reaches for to confirm that a control was not just designed but actually operated. Each stage of the AI life cycle produces its own anchor artifact. Distinct from evidence as a whole: evidence is the proof, an artifact is one piece of it. See evidence, life cycle.Open full entry →, and together those artifacts are what an organization hands to its own board, a regulator, a customer, or an affected person when they ask it to show, not say, that a system is governed.
The absence of that evidence is itself the failure. A risk registerrisk registerThe living record of an AI system's identified risks, ratings, responses, owners and review dates, kept current from design through retirement.Open full entry → that lists risks without test results, a mitigation claimed but never validated, a system deployed and never monitored: each is a governance gap, not a paperwork one. Taken as a whole, the chain is what makes responsible AI accountable rather than aspirational. A principle is a statement of intent; the chain, proven with evidence, is the apparatus that turns that intent into something an organization can demonstrate, one harm, one risk, one control at a time. It is the difference between an organization that says its AI is responsible and one that can prove it.
Operations · keeping controls effective
A control is not done once written: design, implement, operate, verify, report, improve. Verifying and reporting are where the evidence comes from.
Where agentic AI fits
The responsible-AI principles were first worked out for a system that produces an output, a score, a recommendation, a draft, that a human reviews before anything happens. That human review is a natural checkpoint: a place where the principles can be verified before the output has any effect in the world.
Agentic AIagentic AISystems where a model takes actions (calling tools, executing multi-step plans), amplifying both capability and every failure mode; governed with action allowlists, approvals and full logging.Open full entry → does not remove that checkpoint, but it changes its nature. An agentic system carries out a chain of steps on its own, each step feeding the next, and the human is no longer positioned between every step and its consequence. The check moves from sitting inside each decision to sitting around the whole system: setting the bounds it operates within, monitoring the chain as it runs, and holding the ability to intervene. This is why agentic AI reaches across the entire model rather than adding to one part of it. Every principle now has to hold continuously and across a connected sequence of actions, not once per reviewed output, and the controls and the oversight have to be designed for a system that acts without pausing for confirmation. Agentic AI therefore changes how every existing principle has to be implemented, rather than introducing a principle of its own. That is the sense in which agentic AI is not a separate element but a condition that affects all seven principles and both governance levels: they have to be governed at once, over a chain of actions rather than a single decision. This is worked through in detail in the agentic AI cornerstone article.
Where to start
Setting up AI governance happens in two moves, and they match the two levels.
The first move is design: extend the governance the organization already has so that it covers AI. Bring AI risk into the risk management framework and the EU AI Act obligations into the compliance scope; set the policy that requires an AI inventoryAI inventoryA register of all AI systems an organization builds, buys or embeds, with owners and risk tiers. It is the prerequisite for governing any of them.Open full entry → and defines how systems are classified by risk; assign the roles and write down who is accountable; and set the risk appetite for AI so there is a benchmark to judge residual risk against. This is the work that turns broad governance into AI governance, and it is done deliberately, then reviewed.
The second move is execution: carry the design out. Build the inventory, because you cannot govern what you have not mapped, recording every AI system the organization builds, buys, or embeds, including the generative-AI features quietly switched on inside the SaaS tools already in use. Classify every system. Then apply the chain to each high-risk one: what is the harm, what is the risk it carries, which controls reduce that risk, of which type, is the residual risk acceptable, and where is the evidence. The gaps in that picture are the governance backlog, ranked by the risk of the system and the severity of the missing control.
From there, AI governance becomes a practice rather than a project: a living system that keeps each AI system within its boundaries and generates the evidence a regulator will ask for. Done well, governance does not slow AI down. It is what lets an organization use AI at scale, because the organization can understand, control, and demonstrate how its systems behave.
So what is AI governance?
AI governance is the system through which an organization makes responsible use of its AI and can prove it. It is the governance the organization already runs, extended for AI at two levels: design sets the rules, execution carries them out. It takes the seven responsible-AI principles and carries each one down a chain, from the harm that would breach it, through the risk, to the control that reduces it and the evidence that the control works. The law turns parts of these responsible AI concerns into binding duties for the actors and systems within its scope. The chain makes them real, and the evidence makes them provable, for every AI system the organization builds, buys, or embeds.
Governance designgovernance designThe design tier of AI governance: policy, roles, organizational structure, and risk appetite. Governance design sets the boundaries within which AI systems may operate; the execution level tests whether reality stays inside them. See execution level, risk appetite.Open full entry → determines what should be in place: the policies, the roles, the risk appetite, the structure. Whether it demonstrably works is a different question, and it has its own section. AI in control covers the execution layer: risk management, control testing, certification, and the evidence that turns a designed control into a working one.
Continue reading
- Responsible AI: the seven pillars that make it real
- The seven pillars of responsible AI
- EU AI Act, ISO/IEC 42001 and NIST AI RMF: how they fit together
- Governance across the AI life cycle
- Responsible AI vs AI governance: what is the difference
Professionals formalizing this capability often pursue the AIGP certification.
Frequently asked questions
- What is AI governance in simple terms?
- It is the system an organization uses to make responsible use of its AI and prove it. It carries each responsible-AI principle down a chain, from the harm that would breach it, to the risk, to the control that reduces it, to the evidence that the control works.
- What is the difference between AI governance and responsible AI?
- Responsible AI is the set of principles an AI system should meet; AI governance is the system that delivers those principles and proves it. Responsible AI is the goal; governance is how you reach it and evidence that you did.
- Does the EU AI Act require AI governance?
- In effect, yes. The Act mandates no particular structure, but its obligations, risk management, human oversight, and documentation, can only be met if AI governance already lives inside the organization's policies, risk framework, and roles.
- Who is responsible for AI governance in an organization?
- Accountability sits with the operating organization, risk, compliance, the AI function, and internal audit, under the lines of accountability the board sets. Many organizations appoint an AI Officer to coordinate it across the AI life cycle.