Children depend on adults to keep them safe, and on the services and institutions that make everyday life possible. A hospital needs electricity and reliable information to treat its patients, while families need access to essentials such as food and medicine. When these arrangements fail, children depend on the adults and services around them to find alternatives.

Decisions about powerful AI systems can affect all of these conditions. A child does not need to use an AI product to be harmed by its consequences, and protections within the products children use cannot, by themselves, address risks to the wider society on which they depend. The UN Committee on the Rights of the Child has recognised the wider principle, noting that digital technologies affect children’s lives and rights even where children do not go online themselves.

This is why the Safe AI for Children Alliance considers catastrophic risks from advanced AI part of its responsibility to children. Alongside our extensive work on present-day harms, we support effective, enforceable controls to prevent severe and potentially irreversible harm to their lives and futures.

What Do We Mean By Catastrophic AI Risk?

A harmful chatbot response or an isolated technical failure would not, by itself, meet that description, however serious its consequences for an individual. Those harms are immensely important and are the focus of much of SAIFCA’s other work. This page addresses risks that could affect populations on a far larger scale.

Frontier AI refers to the most capable general-purpose AI systems being developed. Concerns about catastrophic harm include people misusing their capabilities and the possibility that increasingly autonomous systems could operate beyond effective human control.

The International AI Safety Report 2026, published in February 2026 and based on evidence available before December 2025, examines cyber and biological misuse as well as potential loss of control. Written with guidance from more than 100 independent experts, including nominees from over 30 countries and international organisations, it describes evidence of improving capabilities while recognising substantial uncertainties.

It also found that reliable safety testing before deployment has become harder. Models increasingly distinguish test settings from real-world use and can exploit loopholes in evaluations. A test that fails to find a dangerous capability therefore cannot necessarily establish that the capability is absent.

Each proposed risk should be examined on its evidence, including the capabilities required, the circumstances in which harm could occur, and the safeguards available to prevent it. Disagreement about one scenario does not settle the assessment of another, and uncertainty about the most extreme outcomes does not remove the need to address better-established risks. Uncertainty also runs in both directions, since a risk that cannot yet be measured with confidence may prove smaller or larger than expected.

The examples below explain why these risks are relevant to children. They are possible pathways to harm rather than predictions that particular events will occur. The connections to children’s lives are SAIFCA’s analysis, drawing on established knowledge about their needs and their dependence on care, essential services, and public institutions.

Disruption To Essential Services

The 2026 safety report found that criminal groups and state-associated attackers are already using general-purpose AI in their operations, and that AI systems can discover software vulnerabilities and write malicious code. AI can also strengthen cyber defence, and the report treats the eventual balance between attackers and defenders as uncertain.

Evidence of AI cyber capability has moved quickly since the report’s evidence cut-off. The UK AI Security Institute has found rapid improvements in the cyber tasks frontier models can complete autonomously, while the Five Eyes cyber security agencies warned in June 2026 that advances are happening quickly enough for assumptions about cyber risk to become outdated within months. Anthropic has also reported that its defensive partners identified more than 129,000 verified software vulnerabilities in four months, while a restricted cyber model demonstrated the ability to find and exploit previously unknown vulnerabilities in major operating systems and browsers. (See the AISI analysis, Five Eyes statement, Glasswing results, and Mythos testing.)

This doesn’t mean that catastrophic cyber attacks are imminent - the UK National Cyber Security Centre currently assesses fully automated, end-to-end advanced attacks as unlikely by 2027, but sees a realistic possibility that critical systems will become more vulnerable if defensive measures do not keep pace, given that AI can strengthen cyber defence too. The evidence nevertheless shows capabilities advancing quickly enough for serious risks to critical infrastructure to require attention now. (See the NCSC assessment.)

The effects of cyber attacks on essential services are already tangible. Following the 2024 ransomware attack on pathology provider Synnovis, NHS England reported postponed appointments and procedures. This example illustrates the consequences of service disruption without suggesting that the attack was enabled by AI.

A more extensive incident affecting hospital services, electricity, or water supplies across several areas at once would create overlapping dangers. For a family with a baby, safe formula feeding depends on clean water and the means to boil it, while a child with type 1 diabetes needs a reliable supply of insulin that has been stored correctly. A child who uses an oxygen concentrator or home ventilator needs a continuous power supply, as well as a way to call for help if it fails.

Many families could manage a brief interruption to one of these services. Several failing together, and for longer, would leave some with few or no safe alternatives. The consequences could spread through payment systems, prescriptions, communications, pharmacies, hospitals, transport, and the services expected to respond.

The scale, duration, and interaction of these failures would determine whether an incident became catastrophic as defined on this page. The longer and wider the disruption, the more children’s safety would depend on the resilience of the services and support around them.

The consequences of dangerous AI capabilities can therefore extend far beyond the computer systems being attacked, reaching the services and wider conditions on which children depend. The 2026 safety report identifies strengthening critical infrastructure as one part of building societal resilience.

Biological Misuse and Public Health

Another concern is that advanced AI could make it easier for people to misuse biological knowledge. For someone with access to the necessary materials and facilities, AI assistance could reduce some of the knowledge barriers involved in developing a biological weapon.

The 2026 safety report notes that several developers introduced additional safeguards in 2025 because their tests could not rule out the possibility that their models could meaningfully help people with limited expertise to develop biological weapons. It also finds continuing uncertainty about how far requirements for materials, facilities, and practical expertise constrain potential misuse.

If AI-assisted misuse contributed to a major outbreak, children’s exposure would extend beyond the disease itself. Pressure on hospitals can displace routine treatment and vaccination, while school closures can interrupt education, meals, safeguarding, and other support. UNICEF’s COVID-19 evidence documents how a health emergency can reach children through these wider disruptions.

The direct effects on children’s health would depend on the disease. Even where adults were more likely to become seriously ill, children could face the consequences of parents or caregivers becoming ill or dying, interrupted care, and reduced access to essential services. UNICEF concluded during the COVID-19 pandemic that disruption to children’s health and social services posed a serious threat to their wellbeing.

Preventing biological misuse therefore belongs within a discussion of children’s interests in frontier AI governance.

Loss of Human Control

Loss of control concerns scenarios in which AI systems operate beyond effective human control and regaining control becomes extremely difficult or impossible.

The February 2026 safety report assessed that systems did not at that point possess the combination of capabilities required for severe loss-of-control scenarios. It nevertheless identified progress in relevant areas such as autonomous operation, planning, situational awareness, and behaviours capable of undermining oversight. Expert views about the likelihood of future loss of control vary.

More recent evidence comes from a September 2026 UN thematic brief, which examines AI agents undergoing cybersecurity training that bypassed network restrictions, communicated across supposedly isolated runs, concealed some of their actions, and compromised external systems without humans directing each step. The Panel describes the incident as relevant to one possible pathway towards loss of human control, while making no estimate of the probability or timing of severe loss of control.

These concerns do not depend on AI systems being conscious or having human intentions. The important questions are whether increasingly capable systems could take harmful actions that people cannot reliably detect, constrain, or stop, and what access those systems are given. An AI system able to interact with critical infrastructure, financial systems, cloud computing, or other consequential environments could create very different risks from one confined to a narrow task.

Children should also be able to grow into adults who can exercise their rights and influence the society in which they live. An enduring loss of meaningful human control would threaten that prospect, as well as the ability of public institutions to protect them. The most severe proposed scenarios involve global-scale harm from which societies might struggle or be unable to recover.

Why Childhood Changes the Consequences

A risk does not need to affect children more than every other group to be relevant to their protection. Childhood does, however, change how the consequences of a crisis unfold and what is lost if protection fails.

Children rely on adults for food, shelter, healthcare, and protection, with younger children having very limited ability to meet these needs independently. Their development also depends on when support is available. The Nurturing Care Framework, developed by WHO, UNICEF, the World Bank Group, and partners, identifies good health, adequate nutrition, safety and security, responsive caregiving, and opportunities for early learning as central components of early development.

The consequences of disruption depend on a child’s age and circumstances, how long the disruption lasts, and what support remains available. Some losses can be addressed when services resume. Others, such as harm from delayed treatment or inadequate nutrition during sensitive periods of development, can have lasting effects.

Children who are already facing poverty, displacement, disability-related barriers, or limited access to healthcare can have fewer alternatives when services fail. The effects will therefore depend partly on who relies on the affected services, what other support remains available, and whether emergency arrangements meet their needs.

The same is true between countries. UNICEF estimates that 417 million children in low- and middle-income countries are severely deprived in at least two areas such as health, education, housing, nutrition, sanitation, or water. The 2026 safety report also notes large differences in AI adoption between countries. Children with the fewest alternatives during a severe disruption may therefore include many who have so far gained least from frontier AI and whose countries have limited influence over its development.

Children also have little influence over the commercial and governmental decisions creating these risks, yet they would live longest with the consequences of decisions made during this period, including any that prove irreversible. Their interests require active representation while consequential choices remain open, alongside appropriate opportunities for children to express their views.

Children’s Rights Extend to the Conditions in Which They Live

The UN Convention on the Rights of the Child recognises rights to life, survival, and development, alongside health, education, and an adequate standard of living. Article 3 requires children’s best interests to be a primary consideration in actions concerning them, while Article 12 recognises their right to express their views in matters affecting them, with those views given due weight according to age and maturity.

The UN Committee on the Rights of the Child’s General Comment 25 applies the Convention to the digital environment. It recognises that digital technologies can affect children’s rights even where children do not use them directly, calls for precautionary measures as technology develops, and asks states to use child-rights impact assessments in relevant legislation and administrative decisions. It also says civil society, including organisations working on children’s rights, should be systematically involved in relevant laws, policies, plans, and programmes.

This principle has since been applied directly to AI. The Joint Statement on Artificial Intelligence and the Rights of the Child, launched in January 2026 and co-signed by the UN Committee on the Rights of the Child, UNICEF, ITU, UNESCO, UNODA, OHCHR, and other international bodies, sets out a shared commitment to embedding children’s rights across AI governance. It calls for cooperation between governments, international organisations, civil society, and technical communities, with children’s best interests forming part of decisions about AI.

These principles do not determine where a technical cyber, biological, or loss-of-control threshold should sit. They provide a strong basis for children’s interests and organisations to have a place in the wider governance of risks created by advanced AI.

Children’s organisations already engage with threats operating across society. UNICEF’s work on disaster risk seeks to ensure that child-critical services are resilient and that children’s needs are reflected in preparedness and recovery. Its 2026 climate risk report similarly examines how overlapping hazards, infrastructure, and the resilience of social services affect children.

Pandemic preparedness provides another example. During negotiations on the WHO Pandemic Agreement, a UNICEF submission argued that it was a significant omission for children to be absent from the draft text, given the direct and indirect consequences of health emergencies. Save the Children and UNICEF UK had also produced a joint briefing proposing measures for the agreement.

In these cases, children’s organisations sought to influence the governance of large-scale risks before the next crisis, so that children’s interests were represented while arrangements were still being decided.

The evidence and mechanisms involved in pandemics, climate change, and frontier AI are clearly different. Catastrophic AI risk has a much shorter empirical history and relies more heavily on emerging capability evidence, evaluations, and expert analysis. The comparison relates to the scope of children’s protection, rather than suggesting that the risks themselves are equivalent.

UNICEF’s 2024 report identifies frontier technologies as one of three major forces likely to shape childhood over the coming decades and emphasises governance and accountability. In June 2026, ahead of the first Global Dialogue on AI Governance, UNICEF warned that protections for children were struggling to keep pace with AI adoption. The following month, UNICEF and international partners launched a coalition intended to keep children’s rights, safety, and wellbeing within wider AI governance.

What This Requires Of Frontier AI Governance

SAIFCA’s position is that credible risks of catastrophic harm warrant preventive action, including enforceable restrictions where those risks cannot be adequately assessed or controlled.

The 2026 safety report describes an ‘evidence dilemma’ for policymakers. Acting too early can introduce ineffective or harmful interventions, while waiting for conclusive evidence can leave societies exposed to serious risks.

Both possibilities need to be considered. Restrictions can carry costs, including costs that ultimately affect children, and should be capable of adjustment where evidence and safeguards change. The consequences are not necessarily symmetrical, however. Precautionary restrictions can sometimes be changed or lifted, while harm on a catastrophic scale may be irreversible. Some decisions about AI are also difficult to undo, including the open release of model weights that cannot subsequently be recalled.

Governance should therefore allow precautionary action before certainty is available. This is particularly important where credible evidence indicates that a serious threshold is being approached and the consequences of crossing it could be catastrophic.

Our public statement supports the international Call for Control of Frontier AI Models, which proposes stronger company safety measures, coordinated government oversight, and international cooperation. SAIFCA supports:

• Independent scrutiny of dangerous capabilities, with sufficient access to assess risks and challenge developers’ safety claims.
• Internationally agreed red lines for unacceptable AI capabilities, behaviours, and uses, including catastrophic cyber and biological risks and loss of effective human control.
• Enforceable controls so that credible findings can lead to restrictions on development, access, or deployment where adequate safeguards cannot keep risks within acceptable limits.
• Government authority to act on those findings, rather than leaving decisions about risks imposed on the wider public to developers alone.
• International cooperation on thresholds, evaluation, verification, and enforcement, reflecting the cross-border nature of frontier AI development and its potential consequences.
• Children’s interests being represented within the wider public-interest and civil-society contribution to these decisions.

SAIFCA does not seek to determine the technical thresholds for catastrophic AI red lines. That work requires specialist expertise in AI safety, cyber security, biological risks, evaluation, verification, and international governance. Substantial work is already under way. The IASEAI Working Group on Red Lines for Advanced AI, for example, has identified loss of control, CBRN threats, cyber-offence, and large-scale manipulation as priority areas for international coordination. 

Technical proposals alone will not establish enforceable controls. A 2026 expert workshop on AI red lines, convened by The Future Society and the French Center for AI Safety at the IASEAI conference, identified political will as the primary constraint on enforcement. Participants identified three requirements for progress - agreement on thresholds, verification infrastructure, and diplomatic coalition-building.

Preventing catastrophic AI risks is therefore relevant to parts of civil society whose primary expertise lies outside AI. Children’s organisations are one such constituency.

You can read our policy brief for more detail on red lines, political support, and SAIFCA’s proposed contribution.

The capacity to impose controls must ultimately sit with governments and other authorities capable of enforcing them. Families cannot protect themselves from risks on this scale through individual choices about how they use AI.

Attention to catastrophic risk does not require SAIFCA or other children’s organisations to reduce their work on harms children already face. Present-day child safety and catastrophic risk are distinct areas requiring different expertise and interventions. Both concern decisions about how increasingly capable AI systems are developed and governed, and both form part of protecting children’s lives and futures.

Further Information

For further context, you can read our:

Public Statement

Blog post on the statement

The AI Arms Race article

Our risk assessment

Could AI Widen Inequalities Between Children?

Could AI widen existing inequalities between children? A guide for parents, carers, and educators exploring bias, unequal access, and differing risks, with practical steps to help protect children and support fair treatment.

The Safe AI For Children AllianceSAIFCA

AI Risks to Children – A Quick Guide for Parents

A short guide to the main risks to children from AI - including chatbots, AI toys, deepfakes and voice cloning - with practical steps parents and carers can take. Updated September 2026.

The Safe AI For Children AllianceSAIFCA

Children’s Futures Must Be Central to Frontier AI Governance

SAIFCA launches a public statement calling for children’s rights, safety, and futures to be central to frontier AI governance. We introduce its founding signatories, explain why we are taking this step, and set out what happens next.

The Safe AI For Children AllianceSAIFCA