We need psychologists to work together with the federal government to develop legislation around what is and is not acceptable for chat-bots to recommend to people expressing suicidal thoughts...then we need to hold chat providers accountable for the actions their robots take.
For the foreseeable future, it should simply be against the law for a chatbot to provide psychological advice just like it's against the law for an unlicensed therapist to provide therapy...There are too many vulnerable people at risk for us to just run a continuous natural experiment.
I _love_ my chatbots for coding and we should encourage innovation but it's the job of government to protect people from systemic risks. We should expect OpenAI, Anthropic, and friends to operate in pro-social ways given their privileged position in society while the government requires them to stay "in line" with the needs of people they might otherwise ignore.
For the foreseeable future, it should simply be against the law for a chatbot to provide psychological advice just like it's against the law for an unlicensed therapist to provide therapy...There are too many vulnerable people at risk for us to just run a continuous natural experiment.
I _love_ my chatbots for coding and we should encourage innovation but it's the job of government to protect people from systemic risks. We should expect OpenAI, Anthropic, and friends to operate in pro-social ways given their privileged position in society while the government requires them to stay "in line" with the needs of people they might otherwise ignore.