Accountability
Every material AI use should have an identifiable owner, an escalation route and an appropriate level of board oversight. Responsibility should not disappear between employees, procurement teams and external suppliers.
AI governance & regulation IWIB4AI session Needs review
Governing the Seen and Unseen: Why Boards Must Act on AI Standards Now
Boards can use AI standards to identify hidden AI use, assign responsibility and oversee organisational risks.
Draft – legal and case law references not yet verified
Summary
This use case examines how boards and senior leaders can govern both formally approved AI and undisclosed or embedded AI across their organisations. ISO standards, such as ISO/IEC 42001 for AI management systems, can help structure accountability, risk assessment and ongoing monitoring. The EU AI Act and GPAI Code of Practice provide complementary legal and compliance reference points, but their relevance depends on the organisation’s role and the systems involved.
The Problem
Boards cannot effectively oversee AI they do not know is being used, including employee-adopted tools and AI embedded in purchased services. The challenge is to turn standards into operational controls without treating certification or voluntary commitments as proof of legal compliance.
Ethical Grounds
Transparency
An AI inventory should capture approved tools, embedded capabilities and unapproved uses. Leaders need understandable reporting on each system’s purpose, limitations and risks rather than relying solely on supplier assurances.
Privacy
Hidden AI use can expose personal or confidential information through prompts, uploads and integrations. Discovery and governance processes should control these disclosures without introducing disproportionate employee surveillance.
Human oversight
Oversight requires people with the competence and authority to challenge AI outputs, stop unsafe uses and escalate concerns. Board policies should translate these responsibilities into practical controls rather than nominal approval requirements.
Legal Compliance
| Framework | Provision | Relevance |
|---|---|---|
| EU AI Act | Art. 4 | Providers and deployers must take measures to ensure sufficient AI literacy among staff and others operating AI on their behalf, supporting role-specific training across the organisation. |
| EU AI Act | Arts. 9 and 17 | For providers of high-risk AI systems, risk-management and quality-management requirements make documented responsibilities, controls and monitoring essential; ISO certification alone does not establish compliance. |
| EU AI Act | Art. 26 | Deployers of high-risk AI systems have obligations concerning appropriate use, competent human oversight and monitoring that boards should translate into operational responsibilities. |
| EU AI Act | Arts. 53, 55 and 56 | The GPAI Code of Practice supports providers in demonstrating compliance with general-purpose AI model obligations, including additional duties for models with systemic risk, rather than imposing a universal governance standard on all AI users. |
| GDPR | Arts. 24 and 35 | Where AI processes personal data, controllers must implement appropriate accountability measures and conduct a data protection impact assessment when the processing is likely to result in high risks to individuals. |
Sources
Court & Regulator Decisions
Moffatt v Air Canada (opens in a new tab)
Responsibility for chatbot misinformation illustrates why oversight of deployed AI must include testing, monitoring and clear ownership of customer-facing outputs.
Deutsche Wohnen (opens in a new tab)
Organisational GDPR liability supports board-level attention to effective accountability structures, rather than assuming responsibility depends on identifying one culpable manager.
IMY – Swedish Police Authority use of Clearview AI (opens in a new tab)
Police use of Clearview without assessment or authorisation directly illustrates the governance risks of unapproved AI use and inadequate internal oversight.
Key Takeaways
- Maintain an AI inventory covering approved tools, employee-adopted services and AI embedded in supplier products.
- Map each use to the organisation’s legal role, risk level, accountable owner and oversight requirements.
- Use ISO standards and the GPAI Code within their proper scope; do not equate certification or adherence with full legal compliance.
- Require recurring board reporting on material risks, incidents, training and corrective actions.
Related Use Cases
From the Session
Governing the Seen and Unseen: Why Boards Must Act on AI Standards Now