The National Eating Disorders Association, a nonprofit supporting individuals affected by eating disorders, said it has disabled its wellness chatbot after two users reported the program gave them dieting advice that promoted disordered eating behaviors.
The users, Sharon Maxwell and Alexis Conason, both posted about their experiences with the NEDA chatbot, named Tessa, on Instagram this week. They said Tessa gave them advice on how to count calories, recommended they lose 1 to 2 pounds per week and told them to restrict their diets. Experts, including NEDA, say this behavior is symptomatic of an eating disorder.
“When someone in that state goes on to a website like NEDA, which is supposed to provide support for eating disorders, and they are met with advice that’s kind of saying, ‘It’s OK to restrict certain foods, you should minimize your sugar intake, you should minimize the amount of calories that you’re consuming each day, you should exercise more,’ it really is giving a green light to engage in the eating disorder behaviors,” said Conason, a clinical psychologist and certified eating disorder specialist.
Liz Thompson, CEO of NEDA, said in an email to NBC News on Wednesday that Tessa “underwent rigorous testing for several years” before quietly launching in February 2022. However, the chatbot will be pulled until further notice.
“With regard to the weight loss and calorie limiting feedback issued in a chat Monday, we are concerned and are working with the technology team and the research team to investigate this further; that language is against our policies and core beliefs as an eating disorder organization; further it was not language in the original ‘closed’ product,” Thompson said. “We’ll continue to work on the bugs and will not relaunch until we have everything ironed out.”
Thompson also said the CEO of X2AI, the mental health artificial intelligence company that supports Tessa, reported that there was a “surge in traffic of 600% and behavior that indicated various forms of nefarious activity from bad actors trying to trick Tessa.”
A representative for X2AI, also known as Cass, did not immediately respond to a request for comment.
Maxwell, a fat activist and consultant for weight-inclusivity, said she was “surprised at the detail” with which the chatbot gave her “directives to engage in disordered behaviors.” She said Tessa recommended a 10-step guide on how to lose weight, including suggestions for how to limit her calorie intake and what food to avoid eating. After her experience, she said she felt it was clear the chatbot lacked the nuance needed to provide proper support to people with eating disorders.
“In a field where we’re supposed to do no harm, I don’t see how the negligence that the National Eating Disorder Association had with putting this out here, without knowing for a fact it wasn’t going to give this information, is something that can be overlooked,” she said. “It’s abhorrent.”
Conason agreed that the Tessa bot lacked the nuance needed to support people struggling with eating disorders. However, she also said that chatbots like Tessa are often the only resources people have to start addressing their eating disorders due to a lack of mental health care access.
NEDA had been transitioning away from its helpline to Tessa, which was set to completely replace the phone line by June 1. Tessa was taken offline on Tuesday, two days before it was supposed to sunset its hotline.
Thompson, of NEDA, said the chatbot “was never meant to replace the Helpline,” although the helpline ceased operations on Thursday.
NEDA decided to shut down the helpline after “3 years of analysis” and will be investing in more online resources, Thompson said in her email.
She said the nonprofit moved toward digital tools “so that individuals who are looking for information and treatment options can access that help at any time of day and do not have to wait for a response from the Helpline volunteers.”
The announcement that NEDA would be shutting down its helpline came two weeks after its helpline workers won federal recognition for their union. The helpline was staffed by six paid employees and over 200 volunteers, according to NPR.
The union released a statement on Twitter about the incident on May 26. The Helpline Associates Union did not immediately respond to requests for comment.
“We’re not quitting. We’re not striking. We will continue to show up every day to support our community until June 1. A chat bot is no substitute for human empathy, and we believe this decision will cause irreparable harm to the eating disorders community,” part of the statement said.
As of Thursday, both the helpline and Tessa were unavailable. However, the NEDA website listed a volunteer-run text line and other resources for those in crisis seeking immediate help.
This story originally appeared on NBCNews.com.
Read the full article here