OpenAI introduces new ‘Trusted Contact’ safeguard for cases of possible self-harm
Key takeaways
- On Thursday Open AI announced a new feature called Trusted Contact, designed to alert a trusted third party if mentions of self-harm are expressed within a conversation.
- OpenAI has faced a wave of lawsuits from the families of people who have committed suicide after talking with its chatbot.
- OpenAI currently uses a combination of automation and human review to handle potentially harmful incidents.
Why this matters: a development in AI with implications for how people work, create, and decide.
On Thursday Open AI announced a new feature called Trusted Contact, designed to alert a trusted third party if mentions of self-harm are expressed within a conversation. The feature allows an adult Chat GPT user to designate another person as a trusted contact within their account, such as a friend or family member. In cases where a conversation may turn to self-harm, Open AI will now encourage the user to reach out to that contact. It also sends an automated alert to the contact, encouraging them to check in with the user.
OpenAI has faced a wave of lawsuits from the families of people who have committed suicide after talking with its chatbot. In a number of cases, the families say ChatGPT encouraged their loved one to kill themselves—or even helped them plan it out.
OpenAI currently uses a combination of automation and human review to handle potentially harmful incidents. Certain conversational triggers alert the company s system to suicidal ideations, which then relay the information to a human safety team. The company claims that every time it receives this kind of notification, the incident is reviewed by a human. We strive to review these safety notifications in under one hour, the company says.