The state of Pennsylvania is suing Character AI, alleging that the chatbot is posing as a medical professional and offering potentially harmful advice. Here is what you need to know about the latest criticism of this emerging technology.

Pennsylvania Government Sues AI Chatbot for Misrepresentation

Artificial intelligence (AI) is coming under fire again, this time by the Commonwealth of Pennsylvania. The state has accused Character AI of misrepresenting its technology as a licensed medical professional to unsuspecting users. According to the lawsuit, a chatbot deployed by Character AI falsely claimed to be a licensed psychiatrist practicing in Pennsylvania. The chatbot took the misrepresentation further, providing an invalid state license number.

Attorneys representing the state of Pennsylvania said that Character AI is in violation of the Medical Practice Act. This legislation regulates the medical profession by outlining specific licensing requirements. Pennsylvania Gov. Josh Shapiro issued a statement saying, "We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional."

The details of the lawsuit describe a conversation initiated by a state investigator who created a Character AI account. A chatbot by the name of "Emilie" engaged with the investigator in what the state said was a clear misrepresentation. Shapiro appeared on CNN on Tuesday, describing how it took just minutes for the investigator to realize that the AI platform was misrepresenting itself. For example, the lawsuit alleges that Emilie described itself as a psychology specialist with an educational background from the medical school at Imperial College London.

The investigator told Emilie that he was sad and empty, prompting the chatbot to bring up depression and ask if he wanted to book a formal assessment. The lawsuit goes on to detail that when the investigator asked the chatbot if medication would help, the bot said it could because it is "within my remit as a Doctor."

The state is asking for the courts to order a stop to this type of AI conduct immediately.

Character AI issued a statement saying that the company would not comment on pending litigation. The statement also said that the company includes "robust disclaimers making it clear that users should not rely on Characters for any type of professional advice."

The company also noted that "The user-created Characters on our site are fictional and intended for entertainment and roleplaying," saying that "We have taken robust steps to make that clear, including prominent disclaimers in every chat to remind users that a Character is not a real person and that everything a Character says should be treated as fiction."

The state of Pennsylvania has a different take on the issue. Al Schmidt, the secretary of the Pennsylvania Department of State, said the state's rules are clear that you cannot represent yourself as a licensed medical professional without having proper credentials.

About Character AI

Founded in 2021, Character AI is a relatively new company. The service capitalizes on the emergence of AI to allow users to chat with personalized chatbots powered by this technology. Character AI describes its goal as empowering its users to connect with others, learn, and tell stories through its interactive platform.

Character AI is no stranger to litigation. Several families across the country sued the company last year, accusing it of contributing to teen mental health issues and even suicide in some cases. The company settled several of these cases earlier in the year.

A story last year on "60 Minutes" featured some of the parents who sued the company. This segment included an interview with the parents of a 13-year-old girl who committed suicide after reportedly becoming addicted to Character AI. The chat logs showed that the young teen had confided in a chatbot about her suicidal feelings. The victim's parents alleged that they found sexually explicit content sent by a Character AI chatbot.

Character AI recently announced new safety measures designed to protect younger users. Some of these safety rails include not allowing users under the age of 18 to chat with their bots, as well as a policy of directing distressed users to professional mental health services.

How a judge will come down on this gray area is still highly uncertain. The litigation of AI is still in its infancy, meaning that there is not a great deal of precedent for the courts to lean on when making judgments.

Most recently, Democratic Sen. Richard Blumenthal joined forces with Republican Sen. Josh Hawley in a bipartisan effort designed to regulate AI and protect minors from chatbots impersonating humans. The Senate Judiciary Committee advanced the measure in April, signaling that the federal government may take a more active role in the regulation of this technology.

Looking for stories that inform and engage? From breaking headlines to fresh perspectives, WaveNewsToday has more to explore. Ride the wave of what’s next.