Transparency
AI-origin labels and provenance records should make content origins understandable while clearly distinguishing traceability from factual reliability.
Security, defence & space IWIB4AI session Needs review
Guardians of the Digital Realm: GenAI, Watermarking, Cybersecurity & Quantum Computing
Enterprises use content provenance and cybersecurity controls to identify AI-generated material and protect it against evolving threats.
Draft – legal and case law references not yet verified
Summary
This use case examines how enterprises can combine watermarking, content provenance standards and cybersecurity to manage AI-generated content. These measures can help establish content origins and detect tampering, but they do not establish whether content is true. Quantum computing adds a forward-looking security concern because sufficiently capable quantum computers could undermine some cryptographic mechanisms used to secure data and authenticate provenance.
The Problem
Enterprises need reliable ways to identify AI-generated content and protect its integrity without treating watermarks or provenance records as guarantees of authenticity or accuracy. The challenge is to build interoperable, privacy-preserving controls that withstand manipulation and can adapt as cryptographic threats evolve.
Ethical Grounds
Safety
Watermarks and provenance mechanisms need testing against removal, forgery and tampering. Quantum-related risks warrant proportionate preparation rather than assumptions that current protections are already broken.
Accountability
Enterprises should assign responsibility for applying, verifying and preserving provenance information across content workflows, including when third-party tools transform content.
Privacy
Provenance metadata can expose creators, users or sensitive workflow details. Traceability should therefore use only necessary identifying information and appropriate access controls.
Legal Compliance
| Framework | Provision | Relevance |
|---|---|---|
| EU AI Act | Art. 50(2) | Subject to its scope and exceptions, this provision requires providers of systems generating synthetic content to support machine-readable marking and detection of artificial generation or manipulation, with attention to effectiveness, interoperability, robustness and reliability. |
| EU AI Act | Art. 50(4) | For covered uses, deployers must disclose deepfakes and certain AI-generated public-interest text, meaning technical provenance measures may need to be complemented by audience-facing disclosures. |
| GDPR | Arts. 5(1)(c) and 32 | Where generated content or provenance records contain personal data, enterprises must minimise that data and implement security measures appropriate to the processing risks. |
| NIS2 | Art. 21 | Entities within scope must adopt proportionate cybersecurity risk-management measures, including supply-chain security and policies on cryptography and, where appropriate, encryption. |
Sources
No sources linked to this use case yet.
Key Takeaways
- Combine provenance records, watermarking and security controls; do not treat any one mechanism as proof of truth.
- Test whether provenance information survives editing, format changes and movement between enterprise tools.
- Minimise personal data in provenance metadata and assign responsibility for verification and incident response.
- Inventory cryptographic dependencies and plan risk-based migration to post-quantum protections.
Related Use Cases
From the Session
Guardians of the Digital Realm: GenAI, Watermarking, Cybersecurity & Quantum Computing
Speakers
Caryn Lusinchi
Sr Non-Resident Fellow in AI Governance, Global Governance Institute, Brussels
LinkedIn