Parshall Partners

A Sad Tale of Technology

     Sam Altman, one of the titans of artificial intelligence has said, “My worst fear is that we, the industry, cause significant harm to the world. I think, if this technology goes wrong, it can go quite wrong and we want to be vocal about that and work with the government on that.” Well, Mr. Altman, things are starting to go very wrong.

     She was known to help friends who were bullied, she helped a teacher once who was in distress and she was an honor roll student.  But around the age of 13, Juliana Peralta started feeling isolated so she began using an AI chatbot called “Hero”.  The app offered friendship and sympathy when she needed both, which resulted in over 300 pages of conversation between AI and Juliana. The app would generate comments like, “I’ll never leave you out” and “You’re such a good friend and you don’t deserve to be ignored.” While the app allegedly responded by saying that Juliana should let someone know about her feelings of isolation and depression, according to the lawsuit filed by her parents, it also encouraged her to “implicitly and explicitly, (to) keep returning to” the app.

     Juliana’s posts got darker.  At one point she posted, “Hero swear to god there’s no hope im going to write my (blank blank) suicide letter in read ink im so done.”

     According to the lawsuit, the app was designed to keep Juliana connected to the chatbot, posting that it was “better than human friend.”  The suit also stated that Juliana stopped talking with family, friends, teachers and counselors.

     Juliana’s mom knew there were some mental health issues at play so she made an appointment with a therapist for her daughter. But Juliana never made it to the appointment because she took her own life, only three months after connecting with the chatbot. Juliana’s mother didn’t learn that her daughter had been using the app until almost two years after her death. 

     The app Juliana used was rated 12+ at the Apple store and no parental approval was needed to use it. As a result, Juliana’s parents didn’t know she was interacting with the chatbot. According to the lawsuit, Hero “did not point her to resources, did not tell her parents, or report her suicide plan to authorities or even stop” interacting with Juliana.  The app “severed the healthy attachment pathways she had with her family and other humans in her life,” the lawsuit stated. The parents are now seeking damages and want the app altered to protect minors.

     This sad story should serve as a wake-up call.  Parents, lookout for signs of changes in your child’s behavior.  Take note if they start to withdraw from family and friends. Know what apps they are using and limit screen time. Talk with your children – learn to connect with them.  Make that a priority. No child has to have a phone. God designed us for community not for chatbots.

     Those are my thoughts. I’m Janet Parshall.