Safety
Chatbots accessible to children should anticipate harmful or age-inappropriate responses and provide routes to human support. Safety should be built into the service rather than depend solely on teenagers recognising every risk.
Education & youth IWIB4AI session Needs review
Youth Chapter Talk – Opportunities and Risks of AI: Chatbots
Helping teenagers use AI chatbots safely while building the skills to question answers and protect personal information.
Draft – legal and case law references not yet verified
Summary
This use case examines how children and teenagers can benefit from AI chatbots while recognising their limitations and risks. Chatbots can support learning, creativity and exploration, but may also produce misleading answers, expose users to inappropriate content or encourage disclosure of personal information. Youth-centred AI literacy can help young people make informed choices without placing the entire burden of safety on them.
The Problem
Conversational systems can sound trustworthy and understanding even when their answers are unreliable or their responses are not suitable for children. The challenge is to preserve useful opportunities for young users while providing age-appropriate safeguards and avoiding excessive reliance on chatbot advice.
Ethical Grounds
Autonomy
AI literacy should help teenagers question chatbot outputs and decide when to seek other sources or trusted people. Conversational design should not exploit age-related vulnerabilities or encourage dependence.
Privacy
Young users may reveal sensitive details through seemingly informal conversations. Services should minimise data collection and make privacy choices understandable and accessible.
Transparency
Children and teenagers should understand that they are interacting with AI, not a person or an inherently reliable authority. Explanations of limitations and data use should suit their level of understanding.
Legal Compliance
| Framework | Provision | Relevance |
|---|---|---|
| EU AI Act | Art. 50(1) | Establishes a duty for providers of AI systems intended to interact directly with people to ensure users are informed that they are interacting with AI, unless this is obvious in the circumstances. |
| EU AI Act | Art. 5(1)(b) | Prohibits AI that exploits vulnerabilities due to age where it materially distorts behaviour in a manner that causes, or is reasonably likely to cause, significant harm. |
| GDPR | Arts. 5(1)(c) and 12(1) | Where chatbot services process personal data, they must minimise collection and provide accessible, clear information, particularly when addressing children. |
| GDPR | Art. 8 | Where consent is the legal basis for an information society service offered directly to a child, parental authorisation is required below the applicable national age threshold, which ranges from 13 to 16. |
Sources
Court & Regulator Decisions
Garante per la protezione dei dati personali – Replika (Luka Inc.) (opens in a new tab)
The companion-chatbot intervention directly illustrates risks to minors and emotionally vulnerable users, supporting safeguards beyond user literacy alone.
Garante per la protezione dei dati personali – ChatGPT (OpenAI) limitation (opens in a new tab)
The ChatGPT limitation highlights age-verification, transparency and lawful-processing requirements relevant to teenagers’ disclosure of personal information during chatbot use.
Key Takeaways
- Teach young users to verify important chatbot answers and recognise that confident language does not guarantee accuracy.
- Explain what personal information should not be shared and provide privacy-protective defaults.
- Make AI identity, limitations and safety controls understandable to children and teenagers.
- Provide clear routes to trusted adults or qualified support when conversations become harmful or concern serious personal problems.
Related Use Cases
From the Session
Youth Chapter Talk – Opportunities and Risks of AI: Chatbots
Speakers
