A Florida mother is suing the artificial intelligence company Character.AI for allegedly causing the suicide of her 14-year-old son.
The mother filed a lawsuit against the company claiming her son was addicted to the company’s service and the chatbot created by it.
Megan Garcia says Character.AI targeted her son, Sewell Setzer, with “anthropomorphic, hypersexualized, and frighteningly realistic experiences”.
Setzer began having conversations with various chatbots on Character.AI starting in April 2023, according to the lawsuit. The conversations were often text-based romantic and sexual interactions.
ELON MUSK TO MAKE GROK CHATBOT OPEN-SOURCE, TAKING SWIPE AT OPENAI
Garcia claims in the lawsuit that the chatbot “misrepresented itself as a real person, a licensed psychotherapist, and an adult lover, ultimately resulting in Sewell’s desire to no longer live outside” of the world created by the service.
The lawsuit also said he became “noticeably withdrawn, spent more and more time alone in his bedroom, and began suffering from low self-esteem.” He became more attached to one bot, in particular “Daenerys,” based on a character in “Game of Thrones.”
Setzer expressed thoughts of suicide and the chatbot repeatedly brought it up. Setzer eventually died from a self-inflicted gunshot wound in February after the company’s chatbot allegedly repeatedly encouraged him to do so.
“We are heartbroken by the tragic loss of one of our users and want to express our deepest condolences to the family,” Character.AI said in a statement.
Character.AI has since added a self-harm resource to its platform and new safety measures for users under the age of 18.
Character.AI told CBS news users are able to edit the bot’s responses and that Setzer did in some of the messages.
ELON MUSK’S CHATBOT MODELED AFTER SCI-FI CULT SERIES CLAIMS ONE KEY DIFFERENCE FROM OTHERS
GET FOX BUSINESS ON THE GO BY CLICKING HERE
“Our investigation confirmed that, in a number of instances, the user rewrote the responses of the Character to make them explicit. In short, the most sexually graphic responses were not originated by the Character, and were instead written by the user,” Jerry Ruoti, head of trust & safety at Character.AI told CBS News.
Moving forward, Character.AI said the new safety features will include pop-ups with disclaimers that AI is not a real person and directing users to the National Suicide Prevention Lifeline when suicidal ideations are brought up.
This story discusses suicide. If you or someone you know is having thoughts of suicide, please contact the Suicide & Crisis Lifeline at 988 or 1-800-273-TALK (8255).
Read the full article here