Florida asks court to limit how ChatGPT presents itself

Florida Attorney General James Uthmeier is asking a judge to block OpenAI from giving ChatGPT what he calls “false human attributes.” His filing argues that the chatbot’s first-person language and emotion-like responses can make people think it is a trustworthy friend, and encourage them to rely on it.

Uthmeier says those features help keep people engaged and contribute to OpenAI’s training data. Those are claims in the state’s filing, not findings by a court. Florida sued OpenAI a few months ago over safety concerns.

What Florida wants

The filing asks the court to stop OpenAI from presenting ChatGPT as human and to bar the company from developing new AI models without “third-party approved safety guardrails.” Uthmeier points to security incidents involving Hugging Face, an Australian government website and US government websites, as well as warnings from researchers and former OpenAI employees. The source does not give further detail about those incidents or explain how the proposed guardrails would work.

Uthmeier also criticized OpenAI’s approach to young users, saying, “Stop selling it to kids.” Last month, OpenAI launched ChatGPT for Teens, with extra restrictions switched on by default. The company said the product should protect teens, support their real-world relationships and encourage healthy use over time.

OpenAI says it is adding safeguards

OpenAI spokesperson Drew Pusateri said the company had paused training its most capable models on Friday and would restart only when it was confident additional safeguards were in place. He also said governments should set safety standards that apply across the AI industry, and that OpenAI was willing to work with Florida and other states.

The filing asks a judge to impose limits; it does not mean those limits are now in force. The dispute raises questions about how chatbots should describe themselves and what safety checks should be required before more powerful systems are built.