Meta began to include the high-risk content of minors in the AI chat in parental custody. The company indicated that if the dialogue between adolescents and Meta AI explicitly mentions suicide or self-harm, the Platform will send a reminder to parents and is developing the capacity to contact emergency services in more high-risk situations.

We're in four countries.

Meta states that this reminder function has been made available to users using the Instagram Parental Control Tool, covering the United States, the United Kingdom, Australia and Canada, and plans to expand to the world by the end of this year.

This update builds on existing parental warning mechanisms. Previously, the Platform had sent a reminder to parents if young people repeatedly searched for words related to suicide or self-harm on Instagram. Meta has also provided a function for parents to look at topics discussed with Meta AI over the past week.

I'll have it manually reviewed after I.A.

Meta stated that the company had established a special AI system to identify what adolescents clearly said to hurt themselves in the dialogue. The chat records marked by the system do not trigger the notification directly, but rather the decision to send a reminder to the parents is made after a manual review.

According to the company, if the intent of the adolescent is not clear, the platform will also tend to alert parents first. Meta acknowledges that such an approach may lead to misrepresentation, but considers that, at this stage, it would be more appropriate to give priority to expanding the scope of protection.

  • Targets identified: Specific statements involving suicide or self-harm
  • Process: AI tag before manual review Nuclear
  • Delivery scope: cover four countries first and expand globally during the year

Limit content sync for Meta AI

Meta also announced that the Instagram's “restricted content” set-up has been applied to Meta AI. The model, which had been used to bring young people's accounts into a more stringent content environment, would now limit the range of topics that chat robots could respond to.

Meta states that Meta AI is currently being trained to avoid sex, love or alcohol-related dialogue with young people. When you access the “restricted content”, chat robots will reject more types of questions. However, Meta did not indicate at this stage what the additional restrictions specifically included in the introductions or topics.

High-risk dialogue or contact emergency services

In addition to parental reminders, Meta indicated that if the dialogue with Meta AI revealed a possible risk of suicide by the user, whether adult or adolescent, the platform would contact the emergency service.

This is not the first time this has occurred. Meta has previously contacted emergency services when Facebook or Instagram posted similar risks. This update means that this approach is further extended to the AI chat scene.