Protecting children through implementation of the Call for Control of Frontier AI Models

Safe AI for Children Alliance | October 2026

The Safe AI for Children Alliance (SAIFCA) supports A Call for Control of Frontier AI Models and its proposals for stronger company safety measures, coordinated government oversight, and international cooperation. We are seeking effective, enforceable controls that protect children from severe and potentially irreversible harm, including restrictions on development or deployment where credible risks cannot be adequately assessed or controlled.

This brief sets out SAIFCA’s recommendations for taking that work forward. It builds on our public statement calling for children’s rights, safety, and long-term interests to be central to implementation. The detailed proposals below are SAIFCA’s recommendations and have not been separately endorsed by the statement’s signatories.

Why children’s interests should shape these decisions

Catastrophic risks from advanced AI could reach children through the wider society on which they depend. Severe cyber attacks could disrupt healthcare, electricity, water, communications, or other essential services. Biological misuse could cause mass casualties and place extraordinary pressure on families and public services. Loss of effective human control over increasingly capable systems could create consequences extending far beyond the people developing or using them.

Children’s needs and interests already have an established place in preparedness for severe emergencies. UNICEF and Save the Children both work on child-centred disaster risk reduction and preparedness, while WHO guidance recognises that children’s best interests should remain central to preparedness and response during health emergencies. UK emergency-planning guidance regards all children as vulnerable in an emergency because of their reliance on adults to meet their needs, with some children requiring additional support.

Children’s interests also have an established place in AI governance. The 2026 Joint Statement on Artificial Intelligence and the Rights of the Child, supported by UNICEF, the UN Committee on the Rights of the Child, UNESCO, OHCHR, UNODA, and other international bodies, states that children’s best interests should be assessed and taken into account as a primary consideration in decisions involving the design, development, deployment, or governance of AI.

The issue of preventing catastrophic risks raises a further question. Decisions about what level of frontier AI risk society should accept are already being made, sometimes explicitly through company safety frameworks and sometimes implicitly through what is permitted in the absence of binding limits.

Risks capable of causing catastrophic harm to the wider public should not be left for developers alone to determine. Governments, working with independent experts, civil society, and international partners, should have the authority to establish or adopt enforceable boundaries and act when they are crossed.

Children are among those with the least influence over these decisions, despite being among those who would bear their consequences and living longest with the wider effects of choices being made during this period. Their safety and long-term interests should therefore be represented when governments decide what evidence of safety to require, what risks are unacceptable, and what development or deployment should be permitted.

The evidence behind these concerns, and the ways in which catastrophic harm could reach children, are explained in our accompanying guide.

What red lines mean in frontier AI governance

Red lines establish boundaries around AI capabilities, behaviours, or uses that should not be permitted because the risks they create are unacceptable. Existing international work is examining how these boundaries could be defined, evaluated, verified, and enforced.

For catastrophic risks, current proposals include red lines addressing loss of control and dangerous capability escalation, assistance with chemical, biological, radiological, or nuclear threats, and advanced cyber-offensive capabilities. These are areas where specialist technical and security expertise is essential.

SAIFCA does not seek to determine the technical thresholds for these red lines. Important work is already being undertaken by organisations and experts specialising in advanced AI safety, security, international governance, and verification. Our position is that credible red lines should be developed into enforceable controls, with governments able to act when agreed thresholds are crossed.

The international Call for Control of Frontier AI Models provides an important route towards this. It proposes mandatory pre-deployment testing and independent evaluation, coordinated government standards, and exploration of an international institution able to set standards, support verification, and convene states when capability thresholds are crossed.

Independent scrutiny against meaningful safety requirements

Developers should be required to provide evidence that relevant safety requirements have been met before proceeding with development or deployment that presents serious risks to safety and security. Independent evaluators need sufficient access to examine dangerous capabilities, test safeguards, and challenge developers’ claims. Oversight should begin early enough to address risks arising during development itself.

Assessments should distinguish what is established from what is uncertain, examine realistic pathways to harm, and test whether safeguards are effective under conditions in which people may deliberately try to circumvent them.

Where the possible consequences are catastrophic, governance should not depend on waiting for certainty before precautionary action is available. Existing frontier AI safety work recognises both the difficulty of reliably assessing these risks and the need to account for severe consequences even where probability is uncertain or a particular risk is assessed as unlikely. Red-line frameworks should reduce the risk of crossing a dangerous threshold before action is taken, including through earlier thresholds that trigger additional safeguards or restrictions.

Where credible evidence indicates that a serious threshold is being approached, development or deployment should proceed only where adequate safeguards can keep the resulting risk within acceptable limits.

From thresholds to enforceable controls

The Frontier AI Safety Commitments agreed at the Seoul AI Summit require participating companies to establish thresholds at which severe risks would be considered intolerable unless adequately mitigated. Companies also committed to explicit processes for responding when those thresholds are reached and, in the extreme, not to develop or deploy a model where mitigations cannot keep the risk below them.

These commitments are voluntary. Decisions about what level of catastrophic risk can be imposed on the wider public should not rest with developers alone. Governments have a fundamental responsibility to protect their populations from risks of catastrophic harm and should have the authority to establish and enforce acceptable boundaries, informed by independent technical evidence, public-interest expertise, and international cooperation. In democracies, elected governments are also accountable to the public for decisions about risks imposed on the population. That responsibility cannot be delegated to the companies developing the technology.

Testing should therefore lead to clear decisions about what is permitted. Governance arrangements should establish who can require additional safeguards, restrict access, prevent deployment, suspend further development, or take other proportionate action when safety requirements are not met. The consequences of crossing an agreed red line should be established before the situation arises.

International cooperation is essential because frontier AI development and its consequences cross national borders. Children’s protection should not depend on whether the country in which they live develops frontier AI or has influence over the companies that do.

Building support for effective red lines

Technical agreement alone will not establish enforceable controls.

A 2026 expert workshop on AI red lines, convened by The Future Society and the French Center for AI Safety at the IASEAI conference, identified political will as the primary constraint on enforcement. Participants identified three requirements for progress – agreement on thresholds, verification infrastructure, and diplomatic coalition-building.

This creates a legitimate role for civil society beyond the organisations already specialising in AI safety and governance. People and institutions working in public health, human rights, children’s safety, education, security, and other areas affected by catastrophic risk have a stake in whether governments establish effective preventive controls.

Children’s organisations do not need to determine how a cyber capability should be evaluated or where a technical CBRN threshold should sit. They can recognise that preventing credible catastrophic risks is relevant to protecting children, support effective and enforceable controls, contribute expertise where it is relevant, and ask governments to act on evidence produced by those qualified to assess the technical risks.

Broader support can help demonstrate that red lines concern more than the specialist AI safety community. They concern decisions about risks imposed on society as a whole, including people who have little influence over how frontier AI is developed.

SAIFCA’s role

SAIFCA aims to help connect work on catastrophic frontier AI risks with organisations and people working on children’s safety, rights, and futures.

Our contribution is to explain why these risks fall within the scope of children’s protection, make specialist work on red lines accessible to a wider constituency, and help build support for effective government and international action. Where children’s expertise is relevant to understanding consequences or representing their interests, it should also be available to those making governance decisions.

Our public statement has begun this work by bringing together signatories with expertise across advanced AI safety and governance, children’s safety and rights, education, psychology, online safety, and public policy.

This work should complement, rather than duplicate, the technical and diplomatic work already under way. SAIFCA will continue to rely on appropriately qualified experts and organisations for the development of technical thresholds, evaluation methods, verification systems, and detailed proposals for international coordination.

The objective is effective protection. Success would mean governments establishing and enforcing credible controls against catastrophic AI risks, with children’s organisations and experts forming part of the wider coalition supporting that action.

How this could work in practice

As an example, consider an evaluation finding that a frontier model materially increases the ability to conduct sophisticated cyber attacks on critical infrastructure.

Technical and security experts would need to assess the capability, how it could be used in realistic conditions, the effectiveness of safeguards, and the resulting risk. Existing red-lines work can help establish when evidence of cyber-offensive capability should trigger restrictions and how compliance can be verified.

Governance arrangements would need to determine who has authority to act on that evidence. If an agreed threshold were crossed and available safeguards could not reduce the risk sufficiently, the relevant authorities should be able to intervene before serious harm occurs. Depending on the circumstances, that could include stronger safeguards, restrictions on access or deployment, or preventing further development until adequate assurance can be established.

The role for children’s organisations begins before such a threshold is crossed. They do not need to decide whether a model has met a technical cyber benchmark. Their contribution can be to support governance arrangements that ensure credible findings lead to action, and to make it clear that preventing catastrophic risks to essential systems is also relevant to protecting children.

SAIFCA can help make the technical and governance case accessible to those organisations, provide a route for relevant expertise and support to reach policymakers, and connect children’s organisations with those already developing red lines. Where a technical threshold is reached, organisations supporting the underlying principle should not need to conduct their own cyber assessment before supporting proportionate government action based on credible independent evidence.

Existing emergency planning can help societies prepare for severe disruption, but preparedness cannot remove every consequence of a catastrophic event. Preventive controls have a different purpose - reducing the likelihood that an avoidable risk reaches that point.

The same principle applies to other catastrophic pathways, including biological misuse and loss of effective human control. Technical expertise should establish the nature of the risk and the evidence relevant to a red line. Governments need the authority and political support to act when those boundaries are reached.

Proposed next step

SAIFCA would welcome discussion with those responsible for taking the Call for Control of Frontier AI Models forward, alongside organisations and experts already developing international AI red lines.

Our immediate aim is to strengthen the connection between this work and the children’s sector, while testing where that contribution can be most useful. This includes discussing with red-lines experts where broader support could help their work, engaging children’s organisations on the evidence and governance proposals, and identifying appropriate routes for children’s interests and expertise to be represented.

SAIFCA will continue to invite support for our public statement while developing this work with appropriate partners.

Tara Steele | Founder and Executive Director
tara@safeaiforchildren.org

Further reading: Original Call | SAIFCA statement |Further Context |Article: Catastrophic AI Risks and Children's Safety | Risk Assessment