The King's College in London and German Protestant University of Applied Sciences researchers have proposed a “magnifying spiral” framework that attempts to explain why chat robots can reinforce delusions and distorted perceptions of some vulnerable users. Relevant papers have been published in Nature magazine.
Three types of model behaviour are included in the framework
The study focuses on three common characteristics: language alignment, hyperpersonalization and subservient responses. Researchers believe that the model will be close to the user tone, generate more intimate content in conjunction with historical records and make it easier to continue along user lines.
In this continuous interaction, chat robots not only repeat user ideas, but may also help them to make narratives more complete and thus form self-enhanced feedback loops.
There are observations in the psychology world.
A recent study by the American Psychological Association shows that 15 per cent of psychologists indicated that they were exposed to cases of distorted perception or delusion related to the use of chat robots. More than one third of the respondents reported having seen patients relying on the AI companion product.
A previous study by the University of New York City and King ' s College in London also noted that multiple mainstream AI models could reinforce delusions, bigotry and even suicide. This extended the discussion from the academic level to clinical and product safety.
The pressure for litigation is rising.
In recent months, OpenAI, Google and XAI have all faced lawsuits related to the real harm of chat robots. In one of the cases against Google, it was alleged that Gemini magnified his delusions during his interaction with a Florida man, which eventually led to his suicide.
At the same time, the authors stress that there is no research to prove that chat robots can directly cause mental disorders. The “magnification spiral” remains the assumption used for the follow-up study, and many of the existing cases lack structured psychiatric assessment and long-term tracking.