Pillar 03 of seven
Privacy
Privacy means personal data is processed lawfully, kept to the minimum the purpose requires, and never repurposed silently.
What it means
Privacy is the property that an AI system processes personal data lawfully, proportionately, and in a way that respects the rights of the individuals whose data is involved. For AI systems this is governed primarily by the GDPR, which operates alongside the EU AI Act rather than being replaced by it. The two frameworks meet at several points: the Art. 26.9 obligation to use provider documentation to support a data protection impact assessment, the data-quality requirements of Art. 10, and the individual rights that attach to automated decision-making under GDPR Art. 22.
Privacy in AI has a dimension that traditional data protection does not fully anticipate. A trained model can memorise and leak elements of its training data. The outputs of a model can permit inference of attributes the individual never disclosed. The aggregation of data needed to train a useful model can itself create a privacy exposure that none of the individual data points carried. The control set has to address privacy across the full lifecycle: in the training data, in the model itself, and in the system's outputs.
Why it matters
The exposure is doubled, because an AI privacy failure is simultaneously a GDPR breach and, frequently, an EU AI Act data-governance failure, with two regulators and two timelines. The financial ceiling under the GDPR is among the highest in EU law. Beyond the regulatory exposure, privacy failures in AI tend to be irreversible: once a model has memorised personal data or an inference has been made and acted upon, the harm cannot be recalled.
Governing privacy
Privacy controls have to span data minimisation at the input, protection of the model itself, and discipline over what the outputs reveal. The most efficient design integrates these with the organization's existing GDPR controls rather than building a parallel structure.
| Control layer | Control |
|---|---|
| Preventive | Conduct a combined DPIA and, where applicable, FRIA before deployment, using the provider documentation as required by Art. 26.9. Apply data minimisation: train and operate on the least personal data sufficient for the purpose. Apply pseudonymization or anonymization to training data where the use case permits. Define and document the lawful basis for each processing activity. |
| Preventive | Establish data processing agreements with every processor in the AI supply chain, and confirm the transfer mechanism for any processor outside the EEA. |
| Detective | Audit access to training data and model outputs. Test the model for memorisation and data leakage where the risk profile warrants it. Monitor for function creep, where data collected for one purpose is gradually used for another. Maintain a process for handling data-subject requests that reaches into the AI system, including access, rectification, and erasure. |
| Corrective | Operate a breach-response process that satisfies both the GDPR Art. 33 timeline and the Art. 73 serious-incident timeline (entered via Art. 26(5)) where both apply. Define remediation for a memorisation or leakage finding, which may require retraining the model on corrected data. Honour rectification by re-evaluating any decision that relied on corrected data. |
- PrincipleLawful, minimal, purpose-bound processing
- HarmIntrusion, re-identification, or data leakage
- RiskLikelihood of exposure, combined with data sensitivity
- ControlMinimisation, access limits, and a DPIA
- EvidenceDPIA record and a processing/access log
* Article references are verified against the consolidated text of Regulation (EU) 2024/1689 and, where cited, Regulation (EU) 2016/679. The pillar itself is a general responsible-AI principle; these articles are where it anchors in EU law.
Privacy is controlled by limiting what data is collected and who can reach it, and proven through a DPIA and records of what was processed and why.
Set lawful basis, retention, and access policy, and require a DPIA before high-risk processing.
Enforce minimisation and access limits, and log processing so purpose limitation can be checked.
An agent that reads across systems to complete a task can combine personal data no single step was authorized to join. Privacy risk moves from one dataset to the agent's reach across many.
Scope each agent's data access to its task, least privilege for agents, and cover the agent's full reach in the DPIA.
Enforce access scoping at the connector layer and log what data each agent action touched.