Can uOttawa make a safe AI therapist? The new fight against chatbot psychosis

University of Ottawa Faculty of Engineering professor Karim Alghoul is one of the researchers leading a team dedicated to developing UbiMyTherapist, an AI digital therapy assistant. Photo by Spencer Colby /Postmedia

Article content

When ChatGPT exploded in popularity two years ago, it didn’t just seep its way into the very fabric of our society. The technology also became privy to some of the most intimate corners of many people’s inner worlds.

Article content

As an “AI therapist,” the technology, however, came with its own set of consequences as more people started depending on it for daily use.

Article content

Article content

Now, clinicians and health care professionals are scrambling to close the mental health-care gap while simultaneously researching alternative solutions addressing cases of what’s been called “AI psychosis,” according to Peter Selby, a scientist at the Centre For Addiction and Mental (CAMH).

Article content

Article content

Researchers across the country, including some at the University of Ottawa, have started developing alternative therapeutic AI tools they say are safer than “taking your chances with ChatGPT,” according to Selby.

Article content

Article content

“I believe health-care professionals should be shaping the technology rather than the other way around,” Selby said.

Article content

A chatbot alternative

Article content

At uOttawa, Faculty of Engineering professor Karim Alghoul is one of the researchers leading a team dedicated to developing UbiMyTherapist, an AI digital therapy assistant.

Article content

UbiMyTherapist would continuously monitor a user’s emotional state and respond with suggestions that are clinically informed.

Article content

Alghoul says the tool runs on everyday devices like smart smartwatches, smartphones or earbuds. The technology is an AI model that helps detect and predict emotions like stress ahead of time and provide support.

Article content

Article content

But the idea is not to get people to stop using Large Language Models (LLMs) like ChatGPT as their therapist altogether.

Article content

“Everyone is using LLMs anyway,” said Alghoul, who also works as a clinical AI adviser at the university’s Faculty of Medicine. “So instead of just kind of forcing people to not use it, what if we put some layers of protection?”

Article content

Article content

Article content

Could you truly trust AI to be your therapist?

Article content

Alghoul says it’s safer because the tool works based on the physiological data of the user.

Article content

This means the AI model can use bio signals like heart rate and sleep data, all collected through smartwatches, to predict emotion.

Article content

“The reason why we focused a lot on the physiological measurement is that this kind of data is continuous. You can wear your smartwatch all the time, even when you’re asleep,” Alghoul said.

Article content

The “Ubi” in UbiMyTherapist, stands for “ubiquitous,” Alghoul explained, which is meant to hint at the continuous nature of the data collection process.

Article content

UbiMyTherapist, which uOttawa professor Karim Alghoul is helping to develop, would continuously monitor a user’s emotional state and respond with suggestions that are clinically informed. Photo by Spencer Colby /Postmedia

Article content

What makes the human-AI interaction safer, though, is the additional protection that the “Retrieval-Augmented Generation”, or RAG, provides. The data collected will be used against a RAG technology that is layered on top of the LLM, the professor said.

Article content

“This will ground the LLM. This means that the LLM will only answer from a specific data set that it’s given,” he said.

Article content

“The answers are not allowed to be from its general knowledge. The tool will only be allowed to go and grab answers from the database.”