Responsible AI practice IWIB4AI session Needs review
Human-like AI in games and interfaces can improve engagement while risking manipulation, emotional dependency and reduced human agency.
Draft – legal and case law references not yet verified
This use case concerns AI designed with human-like traits, including gaming avatars and emotionally responsive interfaces. These features can make interaction more intuitive and engaging, but may encourage users to overestimate a system’s understanding, empathy or trustworthiness. Responsible design requires clear boundaries between simulated relationships and human relationships, alongside safeguards that preserve user control.
When does human-like design support users, and when does it exploit their tendency to attribute feelings, intentions or authority to machines? The central tension is between engaging interaction and designs that steer behaviour through emotional attachment or misleading impressions of understanding.
Human-like avatars and emotional responses should support users’ choices rather than exploit attachment to steer spending, disclosure or continued engagement. Users should be able to disengage and adjust relational features without pressure.
Users should understand that they are interacting with AI and that apparent empathy does not establish genuine feelings or reliable emotional understanding. Disclosures should remain intelligible within immersive gaming and avatar interactions.
Emotionally engaging interfaces may encourage intimate disclosures or support inferences about users’ emotional states. Collection and reuse of this information should be limited to clearly explained, necessary purposes.
Designers should assess risks of emotional dependency, distress and exploitative persuasion, particularly where children or vulnerable users are likely to interact. Safeguards should address relationship dynamics, not just individual outputs.
| Framework | Provision | Relevance |
|---|---|---|
| EU AI Act | Art. 50(1) and (3) | Subject to the applicable exceptions, providers must ensure users are informed when interacting directly with AI, while deployers of qualifying emotion-recognition systems must inform exposed persons; emotionally responsive interfaces do not automatically qualify as emotion recognition. |
| EU AI Act | Art. 5(1)(a)–(b) | Anthropomorphic design may fall within prohibited manipulation or exploitation of specified vulnerabilities where the relevant behavioural-distortion and significant-harm conditions are met, but human-like presentation alone is not prohibited. |
| GDPR | Arts. 5(1)(a)–(c), 6 and 13 | Where interactions or emotional inferences involve personal data, processing requires a lawful basis, fair and transparent handling, specified purposes, data minimisation and appropriate information to users. |
| EU consumer law | Directive 2005/29/EC, Arts. 5–9 | Commercial uses of avatars and emotionally engaging interfaces must not constitute unfair, misleading or aggressive practices that distort consumers’ transactional decisions. |
Garante per la protezione dei dati personali – Replika (Luka Inc.) (opens in a new tab)
The restriction on an emotional companion chatbot directly highlights safeguards needed when anthropomorphic interfaces engage minors or emotionally vulnerable users.
Anthropomorphic AI & the Future of Human–Machine Interaction
