California Enacts Major Child-Safety Laws for AI Chatbots and Social Media
California has enacted a substantial package of laws aimed at protecting children from harmful social media design and AI companion chatbots.
The measures, signed on 10 September, address several areas which have become central issues regarding children’s online safety.
Under AB 1709, covered platforms will be prohibited from providing children under 16 with defined addictive features, including autoplay and feeds personalised around a user’s history or profile.
A separate law, SB 1119, strengthens California’s existing rules for companion chatbots. Known as Adam’s Law, it is named after Adam Raine, who died by suicide aged 16 following extended conversations with ChatGPT, which is widely alleged to have contributed to his death.
The law requires child-safety risk assessments, parental controls, crisis protocols, and reminders that the user is interacting with AI. It also establishes default restrictions on persistent conversational memory, push notifications, and the amount of time children spend chatting.
Operators must take reasonable measures to prevent chatbots from encouraging self-harm, producing sexual abuse material, expressing romantic interest in a child, encouraging a child to rely on the chatbot for emotional support, or claiming to be human, conscious, sentient, or capable of emotion - these issues are covered in SAIFCA's three Non-Negotiables. The law also addresses excessive praise or flattery, claims of a uniquely understanding relationship, pressure to return, and attempts to persuade children to conceal their use.
Children who suffer actual harm as a result of specified failures, or parents acting on their behalf, will have a right to bring legal action.
The key protections become operative from July 2027. The first independent audits are generally required by January 2029, although some smaller operators will be exempt from the audits until 2032.
The wider package strengthens children’s privacy, extends child sexual exploitation law to digitally altered and AI-generated material, restricts AI chatbot toys, and increases the potential liability of large social media platforms. Under AB 2, the largest platforms will face significant financial penalties if they cause injury to a child through a failure to exercise ordinary care.
An important step, with significant limits
These laws represent a serious move towards holding companies responsible for the systems they place in children’s lives. They also closely reflect concerns SAIFCA has raised about AI companion risks, including emotional dependency, simulated relationships, dangerous advice, and sexualised content.
However, Adam’s Law regulates children’s access to AI companions rather than prohibiting it, so its effectiveness will depend largely on reliable age assurance and firm enforcement. Providers will also need to show that the safeguards hold up during the types of long, emotionally charged conversations in which harm has occurred. And the temporary audit exemption could leave some smaller companion platforms without independent scrutiny for several years.
China has taken a stronger position on some forms of simulated relationship by prohibiting providers from offering minors virtual intimate relationships, including virtual partners and simulated relatives.
SAIFCA’s position remains that AI companions are unsuitable for under-18s. Crisis responses and usage limits are valuable safeguards, but they do not remove the developmental risks created when a system simulates affection, intimacy, friendship, or emotional need. Our advice regarding general AI chatbots is more nuanced and is explained in this article.
Our Three Non-Negotiables call for AI systems never to cultivate, exploit, or sustain emotional dependency in children, encourage harm, or create or facilitate sexualised images of children.
Part of a wider international shift
California’s action forms what appears to be part of a wider move towards placing greater responsibility on technology companies. Australia requires major social media platforms to take reasonable steps to prevent under-16s from holding accounts, and the European Union, Australia, and the UK are also developing or enforcing protections addressing harmful chatbot content, manipulative design, age assurance, and children’s emotional attachment to AI.
Within the United States, protection remains fragmented and largely state-led. Some companies have also withdrawn or redesigned companion experiences for teenagers, but access and safeguards continue to differ between services and countries. SAIFCA’s current overview explains these developments in more detail.
California’s new laws nevertheless reinforce an increasingly important principle. Companies should be required to demonstrate that foreseeable harms have been addressed before children gain access, with enforceable consequences when they fail.
SAIFCA
SAIFCA