Goldman Sachs analyst told ChatGPT about the rape and murder plan

A 25-year-old man in the United States was detected by the developer OpenAI and reported to the Federal Bureau of Investigation (FBI) for detailing his plans to rape and kill his ex-girlfriend during his work with ChatGPT. Artificial intelligence companies often use multiple labeled hazard content classification models to identify precisely different types of hazardous content, such as violence, hatred, sexual assault, and to complement the level of risk with emotional strength, speech and intent. According to OpenAI ' s corporate policy, the content is manually audited when the back office monitors that the interlocutor may be planning to harm others, and if the risk is determined to be high, the company may also provide the law enforcement with dialogue content as a lead. According to court documents, the man was named Darren Zhou and appeared to be of Asian origin. He was a Goldman Sachs analyst at West Palm Beach, Florida. The FBI agents linked the contents of the artificial intelligence dialogue to the information received by the woman and concluded that Darren was already a sufficient threat to his personal security and arrested him in May. Darren was released on parole after paying $100,000 in bail. After being informed of these allegations, Goldman Sachs terminated his employment relationship with him in June. Darren was charged with, inter alia, making threats to kill or harm others, and serious stalking with a clear physical threat. On 13 August, he pleaded guilty and was sentenced to five years ' probation. The case also led to industry discussions on the AI chat privacy boundary: how the platform should define the measure of monitoring private user dialogue and in what circumstances the user dialogue should be referred to law enforcement。