Element 5 of the GovCompass-7
Transparency & explainability
The right parties can understand that AI is in use, how it reaches its outputs, and why it produced a particular result.
What it means
Transparency and explainability is the property that the right parties can understand that an AI system is in use, how it reaches its outputs, and why it produced a particular result. The two halves serve different audiences. Transparency is largely outward-facing: the EU AI Act Art. 50 obligations to disclose that a person is interacting with an AI system, to label AI-generated content, and the Art. 26.7 duty to inform individuals subject to a high-risk system. Explainability is largely inward-facing: the capacity for an overseer, an auditor, or an affected individual to obtain a meaningful account of why the system produced a specific decision.
The two are often conflated, but they fail differently. A system can be perfectly transparent about its existence while remaining a black box about its reasoning, and it can be technically explainable while the organisation never discloses its use to anyone. The control set has to deliver both: disclosure to the people who have a right to know the system exists, and explanation to the people who have a right to understand its decisions.
Why it matters
Explainability is the precondition for almost every other control. An overseer who cannot understand why a model produced an output cannot meaningfully review it, which means human oversight collapses into rubber-stamping. An auditor who cannot trace a decision cannot test fairness or safety. An affected individual who cannot obtain an explanation cannot exercise the rights that the GDPR and the EU AI Act grant them. Transparency failures, meanwhile, are the most visible of all governance failures, because an undisclosed AI system discovered by a journalist or a regulator becomes a story about deception rather than a story about a technical shortfall.
Governing transparency and explainability
The controls have to be designed in, because explainability that is bolted on after the fact is usually unconvincing. The choice of model architecture, the logging design, and the disclosure mechanisms all need to be settled before deployment.
| Control layer | Control |
|---|---|
| Preventive | Select model architectures whose explainability matches the stakes of the use case, rather than defaulting to the most accurate model regardless of interpretability. Design the system to log the inputs and the factors that drove each significant output, so a decision can be reconstructed later. Build the Art. 50 disclosures and the Art. 26.7 individual notices into the user experience before launch, not as an afterthought. |
| Detective | Review a sample of explanations for quality, confirming that they are meaningful to their intended audience rather than technically accurate but unintelligible. Audit that disclosures are being presented at the point of interaction. Track explanation requests from individuals and the organisation's ability to satisfy them within the required timeline. |
| Corrective | Where an explanation cannot be produced for a decision that affected someone, treat it as a control failure: review the decision through a human process and correct the logging gap that prevented the explanation. Where a disclosure was missing, remediate the affected interactions and fix the mechanism. |