What is AI governance

AI governanceAI governanceGovernance extended for AI: the same organizational steering at the highest level, widened to cover what makes AI different (it works in probabilities rather than fixed rules, learns from data, and can act at a speed and scale no human reviewer can match). It inherits the existing governance structure and brings AI inside the disciplines the organization already runs, rather than creating a parallel system in a silo. It operates on two levels, design and execution. See governance, governance design, execution level, responsible AI.Open full entry → is the system through which an organization makes responsible use of its AI and can prove it. It is not an ethics statement and not a one-time project: it is an operating discipline that runs for the full life of every AI systemAI systemA machine-based system that, for explicit or implicit objectives, infers from input how to generate outputs — predictions, content, recommendations or decisions — that can influence physical or virtual environments. The OECD-style definition followed by the EU AI Act.Open full entry → the organization builds, buys, or embeds.

This article sets out the model the rest of GovCompass builds on: what governancegovernanceThe system through which an organization steers itself: corporate governance, risk management, compliance, lines of accountability, risk appetite, and the operating model. It exists across everything the organization does, before and beyond AI. AI governance is this same system extended for AI. See AI governance, governance design, execution level.Open full entry → is, how it extends to AI, and the chain that runs from a responsible-AI principleprincipleOne of the seven responsible-AI values a governed system should live up to (fairness, safety and reliability, privacy, security and robustness, transparency and explainability, accountability, human oversight). A principle is abstract: it states an outcome, not a lever you can pull. It becomes governable by naming the harm that would breach it, assessing the risk that harm carries, and placing controls against that risk. When GovCompass holds a principle this way it calls it a pillar. See pillar, harm, risk.Open full entry → down to the evidenceevidenceThe concrete proof that a control is designed, implemented, and working: a test report, an audit trail, an impact assessment, a monitoring log. Each link in the governance chain produces an artifact, and together they are what an organization hands to its own board, a regulator, a customer, or an affected person to show, not say, that a system is governed. Its absence is itself the failure: a risk register without test results, or a mitigation claimed without validation, is a governance gap, not a paperwork one. The closing link of the governance chain. See control, governance.Open full entry → that a specific controlcontrolThe concrete, testable measure that reduces a specific risk, and through that risk protects the principle behind it. Also called a risk management measure, risk response, or risk treatment. Always traceable to the risk it addresses: under EU AI Act Art. 9 every control must map back to a specific risk, and controls recorded separately from their risks is a recognized compliance failure. It works in one of three types: preventive, detective, or corrective. See risk, control types, evidence.Open full entry → works.

Three terms run through all of this and are easy to confuse: ethics, responsible AIresponsible AIThe set of principles an AI system should live up to: fairness, safety and reliability, privacy, security and robustness, transparency and explainability, accountability, and human oversight. Widely shared and sitting under the EU AI Act and the major frameworks. On their own the principles are statements of intent; the law turns them into duties that cannot be met unless they are carried inside the organization's governance, which is how responsible AI lands in governance rather than beside it. GovCompass organizes the seven principles into a control framework, the GovCompass-7, one pillar per principle. See principle, pillar, governance.Open full entry →, and governance. They sit in a natural order, from the broadest to the most operational. Ethics asks what it means to treat people well. Responsible AI makes that question specific to AI. It turns broad values into principles a system can be held to: fairnessfairnessThe responsible-AI principle that systems should not create or reinforce unjust discrimination; operationalized through bias testing, representative data and per-group thresholds — with multiple, mutually incompatible mathematical definitions.Open full entry →, safety, privacy, and the rest. AI governance is the most operational of the three, the system through which an organization actually delivers those principles and proves it.

Ethics gives the values, responsible AI gives the principles, governance makes them real. This article is about the third, and how it carries the second.

Governance, and how AI governance extends it

Governance, in its broad sense, is the system an organization already has for steering itself: corporate governance, riskriskIn the EU AI Act's terms, the combination of the probability that a harm occurs and the severity of it if it does. The link between a principle (via the harm that would breach it) and a control (the measure that reduces it). Naming the harm and assessing its risk is required by Art. 9 before any mitigation measure is chosen. See harm, control, residual risk.Open full entry → management, compliance, the lines of accountabilityaccountabilityThe principle that a named human or organization answers for an AI system's outcomes, through ownership, documentation, audit trails and redress — never the system itself.Open full entry →, the risk appetiterisk appetiteThe level of risk an organization's leadership is willing to accept in pursuit of its objectives, set at the governance design level. It is the benchmark against which residual risk is judged acceptable or not, inherited from the organization's broader governance and applied to AI. A concept from enterprise risk management (COSO ERM) before it is an AI one. See residual risk, governance design.Open full entry → the board sets, the operating model, and more. It exists before any AI system is switched on, and it governs everything the organization does.

AI governance is not a separate system that runs alongside the existing one. It is the same governance system at the highest level, extended for AI. What makes AI need that extension is that it behaves differently from the systems governance was built for: AI works in probabilities rather than fixed rules, it learns from data, and it can act at a speed and scale no human reviewer can match.

Extending governance, then, means taking the disciplines the organization already runs, risk management, compliance, data protection, security, and bringing AI inside them: adding AI risk to the risk management framework, bringing the EU AI Act and related obligations into the compliance scope, rather than writing a parallel rulebook in a silo the governance function never sees. The NIST AI Risk Management Framework and ISO/IEC 42001 are both explicit on this point: governance is a broad, cross-cutting organizational layer, not a technical add-on.

This extension is design work, and it is distinct from the day-to-day work of running AI systems. At the design level, it means setting the rules: updating policy so that every AI system has to be classified by risk, defining how that classification is done, and writing down who carries which obligations, the providerproviderThe actor who develops an AI system (or has it developed) and places it on the market or into service under its own name — carrying manufacturer-style duties: design controls, documentation, conformity.Open full entry → and the deployerdeployerAn organization using an AI system under its own authority in its activities — carrying operator duties: use per instructions, oversight, input relevance, monitoring, notices.Open full entry → among them, with the minimum requirements each must meet. The design says what must happen and who is accountable. Carrying it out, building the actual inventory, running the actual classification, testing the actual controls, is execution, and that is the next level down.

Two levels: design and execution

A governance system operates on two levels, and almost every governance question belongs to one of them. Keeping them apart matters, because an organization can be strong on one level and weak on the other, and the two failures look very different.

Governance designgovernance designThe tier at which governance structures, policies, frameworks, and standards are established; sets the rules the execution level must follow.Open full entry → is the strategic level: how the organization sets up its AI governance. The organizational structure, the operating model, the lines of accountability, the risk appetite, the policies, the roles. Design is set deliberately and reviewed periodically. It answers one question: how do we govern AI as an organization?

Execution is the level beneath it: the continuous work that makes the design real. Model development and testing, risk assessments, control testing, impact assessments, monitoring, and the production of evidence, carried out across the full life of each system. Execution answers a different question: are we doing what we designed, and does it work?

Auditors have a name for this split: they ask first whether a control is well designed, and separately whether it actually operates. The same holds for a whole governance system. The design can be sound on paper while the day-to-day execution quietly fails, and the reverse is equally possible. The most common failure is the first kind: an organization writes the policy, builds the inventory, and then lets the testing and monitoring lapse, ending with paperwork instead of practice. What you want is neither level standing alone but both working together, embedded in how the organization already runs, so that the design is genuinely lived and not just documented.

Why the principles land in governance

Responsible AI has settled on seven principles: an AI system should be fair, safe and reliable, private, secure, transparent and explainable, accountable, and under human oversighthuman oversightDesigned-in human ability to monitor, intervene in, override or shut down an AI system — meaningful only when the human has authority, information and time to act.Open full entry →. These principles are widely shared, and versions of them appear in the EU AI Act, the OECD AI Principles, and the NIST AI RMF. On their own, though, they are statements of intent, and intent is not where governance starts. Something has to make them bind.

The law is what does it. The EU AI Act does not order an organization to adopt a set of principles, and it mandates no particular governance structure. Instead it imposes concrete obligations that can only be met if the principles already live inside the organization's governance. Risk management for high-risk systems falls under Article 9; human oversight under Articles 14 and 26; further duties cover transparencytransparencyOpenness about the fact that AI is used and how it operates in general: disclosures, documentation, notices. Pairs with explainability, which addresses individual outcomes.Open full entry →, data quality, and record-keeping. Each obligation is a principle turned into a duty: you cannot deliver meaningful human oversight unless the oversight principle lives in your roles and policies, and you cannot satisfy the risk management duty unless fairness, safety, and the rest are something your risk framework actually assesses.

For regulated sectors, the Act makes this integration explicit. A financial institution may satisfy its AI monitoring obligation through the internal governance arrangements it already runs under financial-services law, the principles delivered through governance that already exists, rather than through a separate structure built alongside it.

So the principles do not sit next to governance as an ethics statement. The law pushes them into it: into the policies, the risk management framework, the roles, the risk appetite. That is the design work described above, now with a reason behind it. And once a principle lives in governance, it still has to be made real for each individual system. That is the chain.

The chain: from principle to evidence

GovCompass organizes those seven principles into a control framework, the GovCompass-7, one pillarpillarA responsible-AI principle as something an organization actively holds rather than merely endorses: one of the seven pillars of the GovCompass-7 control framework, one per principle. A pillar is held, not implemented, by naming the harms that would breach the principle, assessing their risk, and placing controls that reduce it. Distinct from the integrating element (agentic AI), which binds the seven rather than being one of them. See principle, harm, risk, integrating element.Open full entry → per principle, so that each can be held in the same disciplined way. The way a single pillar is held for a single system is a governance chaingovernance chainThe traceable line by which a single pillar is held for a single system: principle, then the harm that would breach it, then the risk that harm carries, then the control that reduces the risk (preventive, detective, or corrective), then the residual risk judged against appetite, proven with evidence. The chain is what makes responsible AI accountable rather than aspirational, and what lets an organization move a principle from a policy statement to a working control it can point to. See principle, harm, risk, control, residual risk, evidence.Open full entry →, and that chain is what lets an organization move a principle from something it states in a policy to something it can point to a working control for: not "we are committed to fairness" but "here is the control that reduces this specific fairness risk, here is the test that shows it works, here is who owns it."

That move is necessary because a principle on its own cannot be controlled directly. Take fairness. You cannot reach into a system and switch on "fairness" the way you implement a concrete measure; fairness is the outcome you want, not a lever you pull. What you can do is work out how the system might fail to be fair, and act on that.

This is where the EU AI Act's logic gives the chain its shape. Principles are abstract; harmharmThe concrete damage an AI system can do that a responsible-AI principle exists to prevent: in the EU AI Act's terms, harm to a person's health, safety, or fundamental rights. Harm is the bridge between an abstract principle and a governable risk; governance becomes operational the moment an organization names the specific harms it wants to prevent. For fairness, a harm is a group receiving systematically worse outcomes because of a characteristic that should not have counted. See principle, risk.Open full entry → is concrete, and governance becomes operational the moment an organization names the specific harms it wants to prevent. The Act is built on exactly this move: it concerns the ways an AI system can harm people's health, safety, or fundamental rights. So the first thing governance does with a principle is ask what harm would breach it. For fairness, the harm is concrete: a group of people receives systematically worse outcomes because of a characteristic that should not have counted, a loan model that rejects one demographic at a higher rate for reasons unrelated to creditworthiness.

A harm on its own is still not something you can act on until you know how likely it is and how serious it would be. That is the risk: in the Act's terms, the combination of the probability that the harm occurs and the severity of it if it does. Naming the harm and assessing its risk is what turns "be fair" into something an organization can actually work on, and the Act requires this step before any measure is chosen.

Only now does a control enter: a concrete, testable measure that reduces a specific risk, and through that risk protects the principle behind it. Controls work in one of three ways that any auditor recognizes: preventive controls that stop the harm before it can occur, detective controls that reveal it through testing, logging, and monitoring, and corrective controls that limit the damage and feed the lesson back into prevention. For the fairness risk above, that might mean a representativenessrepresentativenessHow well training data reflects the population and conditions the system will face in deployment — the fitness-for-purpose core of AI data quality.Open full entry → check on the training datatraining dataThe data used to fit an AI model's parameters; its quality, lawful rights and representativeness are central governance concerns.Open full entry → (preventive), bias testing across demographic groups once the model is live (detective), and a route to suspend and retrain when a threshold is breached (corrective).

A preventive control alone is rarely enough: a risk also has to be caught once the system is running, and there has to be a way to put things right when a control fails. And each control must trace back to the specific risk it addresses: the EU AI Act makes this binding, and risks recorded in one place with controls in another, no link between them, is a recognized compliance failure.

Governance is never about eliminating all risk; it is about knowing which risk remains and deciding whether it is acceptable. No control reduces a risk to zero, and not every control is worth its cost. What is left after the controls are in place is the residual riskresidual riskThe risk that remains after controls have reduced it. No control reduces a risk to zero, and not every control is worth its cost, so a deliberate judgment is made: whether the cost of further control is justified by the reduction it would buy, and whether the remaining risk is acceptable against the organization's risk appetite. This is a design-level judgment, where execution reports back up and governance accepts the residual risk, calls for more control, or declines the use case. EU AI Act Art. 9(5) requires it to be judged acceptable per hazard and overall. See risk, control, risk appetite.Open full entry →, and a deliberate judgment has to be made: is the cost of further control justified by the reduction in risk it would buy, and is the risk that remains acceptable against the organization's risk appetite. That judgment belongs to the design level. It is where execution reports back up, and where governance accepts the residual risk, calls for more control, or declines the use case. The chain closes here and loops: the residual-risk judgment feeds the design, which shapes the next round of execution. And like every other step, that judgment counts only if it can be shown to have been made.

Evidence is the point

What separates governance from good intentions is evidence, and evidence is something concrete. It is the test report showing the bias check ran and what it found, the audit trail recording who approved a high-risk system going live, the impact assessmentimpact assessmentA structured evaluation, carried out in the plan-and-design stage, of the harms an AI system could cause and the risk those harms carry, before the system is built. The first place the governance chain is run, and the cheapest point in the life cycle to reduce risk. The anchor artifact of the planning stage; under the EU AI Act, a fundamental-rights impact assessment is required for certain high-risk deployers. See harm, risk, life cycle.Open full entry → documenting which harms were considered and how they were addressed, the monitoring log proving the system was watched after launch, not just before. Each link in the chain produces an artifactartifactThe concrete record that proves a control was carried out: a test report, an impact assessment, a monitoring log, a release sign-off. An artifact is the tangible form evidence takes, the thing an auditor reaches for to confirm that a control was not just designed but actually operated. Each stage of the AI life cycle produces its own anchor artifact. Distinct from evidence as a whole: evidence is the proof, an artifact is one piece of it. See evidence, life cycle.Open full entry →, and together those artifacts are what an organization hands to its own board, a regulator, a customer, or an affected person when they ask it to show, not say, that a system is governed.

The absence of that evidence is itself the failure. A risk registerrisk registerThe living record of an AI system's identified risks, ratings, responses, owners and review dates — kept current from design through retirement.Open full entry → that lists risks without test results, a mitigation claimed but never validated, a system deployed and never monitored: each is a governance gap, not a paperwork one. Taken as a whole, the chain is what makes responsible AI accountable rather than aspirational. A principle is a statement of intent; the chain, proven with evidence, is the apparatus that turns that intent into something an organization can demonstrate, one harm, one risk, one control at a time. It is the difference between an organization that says its AI is responsible and one that can prove it.

Where agentic AI fits

The responsible-AI principles were first worked out for a system that produces an output, a score, a recommendation, a draft, that a human reviews before anything happens. That human review is a natural checkpoint: a place where the principles can be verified before the output has any effect in the world.

Agentic AIagentic AISystems where a model takes actions — calling tools, executing multi-step plans — amplifying both capability and every failure mode; governed with action allowlists, approvals and full logging.Open full entry → does not remove that checkpoint, but it changes its nature. An agentic system carries out a chain of steps on its own, each step feeding the next, and the human is no longer positioned between every step and its consequence. The check moves from sitting inside each decision to sitting around the whole system: setting the bounds it operates within, monitoring the chain as it runs, and holding the ability to intervene. This is why agentic AI reaches across the entire model rather than adding to one part of it. Every principle now has to hold continuously and across a connected sequence of actions, not once per reviewed output, and the controls and the oversight have to be designed for a system that acts without pausing for confirmation. Agentic AI therefore changes how every existing principle has to be implemented, rather than introducing a principle of its own. That is why GovCompass treats it as the integrating elementintegrating elementAgentic AI, the integrating element of the GovCompass framework: not an eighth pillar but the element at the center that binds the seven. When a system stops producing outputs a human reviews and starts acting on its own, the human checkpoint changes nature and every principle has to hold continuously across a connected sequence of actions. Agentic AI therefore changes how every principle has to be implemented rather than adding a principle of its own. See pillar, agentic AI.Open full entry → of the framework: the condition under which all the principles have to be governed at once, end to end. This is worked through in detail in the agentic AI articles.

Where to start

Setting up AI governance happens in two moves, and they match the two levels.

The first move is design: extend the governance the organization already has so that it covers AI. Bring AI risk into the risk management framework and the EU AI Act obligations into the compliance scope; set the policy that requires an AI inventoryAI inventoryA register of all AI systems an organization builds, buys or embeds, with owners and risk tiers — the prerequisite for governing any of them.Open full entry → and defines how systems are classified by risk; assign the roles and write down who is accountable, provider and deployer among them, with the minimum requirements each must meet; and set the risk appetite for AI so there is a benchmark to judge residual risk against. This is the work that turns broad governance into AI governance, and it is done deliberately, then reviewed.

The second move is execution: carry the design out. Build the inventory, because you cannot govern what you have not mapped, recording every AI system the organization builds, buys, or embeds, including the AI quietly switched on inside the SaaS tools already in use. Classify every system. Then apply the chain to each high-risk one: what is the harm, what is the risk it carries, which controls reduce that risk, of which type, is the residual risk acceptable, and where is the evidence. The gaps in that picture are the governance backlog, ranked by the risk of the system and the severity of the missing control.

From there, AI governance becomes a practice rather than a project: a living system that keeps each AI system within its boundaries and generates the evidence a regulator will ask for. Done well, governance does not slow AI down. It is what lets an organization use AI at scale, because the organization can understand, control, and demonstrate how its systems behave.

Continue reading

• How the GovCompass model maps to the major frameworks
• Governance across the AI life cycle
• Responsible AI vs AI governance: what is the difference