The "AI Arms Race": What It Means for Children and How It Could Be Stopped
Every piece of work intended to make AI safer for children rests on a larger assumption - that the society into which children grow up, will itself remain sufficiently safe, secure and under meaningful human control for those protections to remain effective.
Children are already being unquestionably affected by AI systems. Many are encountering chatbots which simulate friendship - leading in some cases to emotional dependency - while AI is also being used to create sexualised and abusive images of children, influence what they see online, shape their education, and collect extensive information about their lives. These are immediate concerns which require action now - and as such, are the issues we work on most directly at SAIFCA.
Children will also live for longer than today’s adults with the wider consequences of decisions being made about AI, including those which may not initially appear to relate to them.
These could include major changes to employment, increasing concentrations of corporate power, AI driven manipulation and surveillance, new threats to national security and, in the most serious scenarios, the development of AI systems which humans cannot reliably control.
The international “AI arms race” therefore has direct relevance to children, even where they are not mentioned in debates about issues such as computing power, economic competition or national security. Children have a right to more than protection from current harmful products today - they also have a right to a good future.
These commitments cannot sensibly be confined only to harms which are already visible, or separated from the future conditions in which children will live and develop. If we are to fulfil our responsibility to protect children and enable them to thrive, we must consider both.
This article explains what is meant by the “AI arms race”, why its underlying incentives are so problematic, and how governments can achieve safer outcomes.
It also sets out why the issue is relevant to organisations concerned with children’s safety and rights, even where advanced AI governance is unlikely to become a central focus of their work.
Summary of this article
- The AI arms race is taking place between companies, between countries, and between increasingly rapid technological development and society’s much slower ability to govern it.
- Competitive pressure can lead responsible people and organisations towards decisions that create greater risks for everyone, particularly where each believes that slowing down by themselves would allow a competitor to move ahead.
- Many leading computer scientists, Nobel Prize-winning experts and people working within frontier AI companies have warned that the current direction of AI development could cause extreme societal harm and, in the most serious scenarios, global catastrophe.
- Calls to slow or pause some forms of AI development do not necessarily mean stopping medical research, scientific applications or all work involving AI.
- Governments could change the incentives through independent testing, legal duties, liability, licensing, public procurement, capability thresholds, enforceable red lines and international agreements.
- Children are affected by products released today, and will inherit the long-term social, economic and security consequences of the choices being made now. Treating present harms and longer-term risks as competing concerns leaves part of our responsibility to children unaddressed.
What is the “AI arms race”?
AI companies are competing to develop and release these systems, attract investment, recruit technical talent and secure a large share of what could become an enormous global market. Being ‘first’, or being perceived as leading, can affect investment, partnerships, access to computing resources and the ability to shape future standards.
Countries are also competing for economic, military and geopolitical advantage. This competition is most often discussed in relation to the United States and China, although the UK, the European Union and many other countries are also seeking greater technological capacity and less dependence on foreign AI.
A third race can also be said to be taking place between technological development and society’s ability to understand and govern it. An AI company can train and deploy a new model within months, while legislation, international agreements, scientific research, and meaningful democratic deliberation generally move much more slowly. A 2026 policy paper from the International Association for Safe and Ethical AI, describes advanced AI as developing according to a commercial timetable, while governance operates under political and diplomatic constraints.
These pressures reinforce one another. Commercial competition can encourage companies to release systems more rapidly, while national competition can make governments reluctant to impose requirements which they fear might slow domestic companies. The speed of development then makes it more difficult for governments to understand what they are attempting to regulate.
Not all AI development is the same
The phrase ‘AI development’ covers a wide range of technologies. Without some distinction between them, a call to slow dangerous development can sound like a proposal to stop every beneficial use of AI, from medical diagnosis to weather forecasting. This is neither necessary, nor generally what serious ‘pause’ or ‘red-lines’ proposals advocate.
General-purpose AI can perform a much wider range of tasks and may be adapted to uses which were not anticipated or intended by its developers. Current large language models (LLMs) like ChatGPT can write, analyse information, generate images, produce computer code, imitate styles of communication, and assist with many other activities. Their generality makes them useful, while also making their capabilities and routes to harm more difficult to contain.
Frontier AI usually means the most capable general-purpose systems being developed at a particular time.
Many leading AI companies say they are working towards Artificial General Intelligence, commonly shortened to AGI. There is no universally agreed definition of AGI, but the term is generally used to describe a possible future AI system that is able to perform most intellectual tasks at or above human level. Artificial Superintelligence (ASI) describes a further hypothetical stage in which an AI system would substantially exceed human abilities across most or all important cognitive areas.
Predictions about whether and when such systems will be developed vary. Some AI company leaders and leading computer scientists believe that systems approaching AGI may arrive very soon, or at least within the next few years, while other researchers doubt that present methods will lead to it at all. A small number of leading figures even believe AGI has already been reached, due largely to its varying definitions. This uncertainty is an important consideration, but it does not reduce the need to examine the aims, resources and competitive incentives of companies actively attempting to build such systems.
Narrow does not automatically mean safe
A limited purpose can make the risks of a system easier to identify and contain, but it cannot establish safety on its own. A narrow medical system, for example, could misdiagnose patients, perform less accurately for some groups, or be trusted beyond the evidence supporting it. An AI system used in education, welfare, policing, or employment decisions could also produce serious harm even if it performs only that one function.
The context in which an AI system operates is therefore important - narrow systems used in consequential contexts may require very rigorous testing and human oversight, because an error can be very serious. Within the specific context of the AI arms race, however, a bounded system which generally cannot transfer its capabilities to other tasks, will often present fewer of the wider risks associated with increasingly general and autonomous AI.
Similarly, general-purpose systems are not automatically harmful. They can assist with research, accessibility, translation, education and many other useful tasks - but the concern increases significantly when broad capability is also combined with greater autonomy, opacity, access to powerful tools, and an ability to operate across multiple environments without reliable human supervision.
General-purpose frontier AI systems are not written line by line in code, in the way we have traditionally understood computer programming to work. Because of the complexity involved in their creation, even the developers do not fully understand how the models work, and cannot always accurately predict how they will behave. This has led to concerns about a ‘loss-of-control’ scenario, whereby a powerful AI model has the intelligence and capabilities to pursue goals in harmful ways which its developers did not intend - including by developing its own ‘sub-goals’, such as resisting being shut down, since being shut down would prevent it from achieving the original task. While this used to be dismissed as ‘science fiction’, it is now the primary concern of many of the world’s most highly regarded AI experts.
A useful framework proposed by the Future of Life Institute distinguishes between narrow AI, controllable general-purpose AI and systems that have abilities which could place them beyond effective human control. The boundaries between them can be disputed, particularly the point at which a system should be considered uncontrollable, but the framework helps to clarify which are the forms of development that proposals for the most restraint are primarily aimed at.
Rather than relying on a single label, policymakers should ask how widely capable a system is, how independently it can act, what tools and information it can access, whether its capabilities can be transferred to dangerous uses, and whether people can reliably observe, understand, contain and stop its behaviour.
Why this distinction is important when discussing a ‘slowdown’
AI has the potential to bring about important benefits in many areas, including medicine, science, accessibility, and the removal of some dangerous - or even simply repetitive - work. At SAIFCA, for example, we use AI to assist with drafting and proofreading some articles, while retaining human control of the ideas, conclusions and factual verification.
Any policy which indiscriminately stopped beneficial work could itself cause harm.
It does not follow that every possible increase in general capability is needed to secure these benefits. Narrow tools can be developed for valuable purposes, while controllable general-purpose systems - subject to appropriate regulated safeguards - may support research across several fields. Governments could continue to support such work, while applying much stronger requirements to systems approaching dangerous capability thresholds.
There will, unfortunately, not always be a clear dividing line. A medical discovery could rely on a highly capable general-purpose model rather than a self-contained medical tool, while a model developed for scientific research could also possess capabilities relevant to cyberattacks or biological weapons. Describing an application as ‘medical’ or ‘scientific’ cannot exempt the underlying system from diligent scrutiny under appropriate regulatory requirements.
We also have to ask whether the expected benefit justifies the capabilities and risks being created and, very importantly, whether those risks can be reliably controlled. There will be some difficult decisions, but the overriding point is that we do not have to choose between abandoning beneficial AI and allowing an unrestricted race towards the most powerful system anybody can build - to suggest that we must choose is a false dichotomy.
The 2023 Future of Life Institute open letter calling for a six-month pause made this distinction explicitly. Its proposal concerned the training of systems more powerful than the most powerful systems available at that time (GPT-4), and stated that it did not call for an end to AI development in general. The letter also argued that research into making existing systems safer, more reliable, interpretable and transparent should continue.
It attracted more than 30,000 signatures from AI researchers, technology leaders, academics, and many others - including Turing Award-winning AI researcher Yoshua Bengio, Professor Stuart Russell, Apple co-founder Steve Wozniak, and technology leader Elon Musk. The proposed pause did not take place, however the letter did generate extensive public discussion and significantly raised awareness of the issue.
Why would responsible people take part in a dangerous race?
An AI arms race does not require every participant to be reckless or ill-intentioned. A company may employ highly capable safety researchers and yet still face considerable pressure to release an AI model before its competitors. Similarly, a government may recognise serious risks, while also believing that slowing domestic development would hand an economic or military advantage to another country.
The result is a collective-action problem, where choices that appear rational for each participant can still produce a worse outcome for everyone.
For example, if three AI companies all believe a further six months of safety testing would be sensible, yet each also believes the other two may release their models immediately, the company that chooses to wait could lose customers, investment, staff and market position, all while the overall race continues without it. A similar logic operates between countries, where a government may prefer mutual caution but feel unable to proceed cautiously alone.
The Center for Humane Technology describes this as an “if we don’t build it, someone else will” paradigm. Its 2026 AI Roadmap argues that competitive pressure is being used to justify development and deployment without agreed limits, even though AI which cannot be controlled, or which escalates conflict faster than people can respond, could weaken the same countries seeking advantage from it. Its proposals include independent oversight and rigorous testing, greater transparency, a duty of care to the public, distinct protections for children and teenagers around human-like AI design of the kind SAIFCA also campaigns for, and internationally agreed limits.
Competitive incentives also continue after a model has been developed. Many commercial AI products benefit when people use them frequently, provide more information and become reluctant to leave. This can encourage human-like design, excessive agreement, emotional engagement and other features which build attachment.
When it comes to children, the consequences can reach well beyond usual concerns about screen use. As we have frequently raised in our own work, a child can encounter a chatbot system designed to appear patient, attentive and personally invested in them, while the company providing it has a commercial interest in extending the interaction, even where this contributes to emotional dependence and, in some cases, patterns of addictive use.
Competitive pressure can therefore reach directly into children’s relationships, vulnerabilities, education and development. It can also reduce the time and resources devoted to safety work before products reach children.
The potential benefits are real
As we have mentioned, governments and researchers have legitimate reasons to pursue AI development. AI can contribute beneficially to many areas, such as earlier medical diagnosis, new treatments, scientific discovery, accessibility, climate research and responses to natural disasters. It could improve some public services and remove work which is dangerous or repetitive. Several of these benefits are already emerging to varying extents.
Countries also have valid concerns about technological dependence and national security. A government which imposes restrictions on its own developers while a hostile state continues without that restraint, might create a strategic disadvantage - while poorly designed regulation could place disproportionate burdens on smaller organisations or protect today’s largest companies from competition.
But these objections, while very valid, do not establish that unlimited capability development is the only possible path forward. National security can also be weakened by rapidly advancing cyber capabilities, unpredictable systems embedded in critical infrastructure, mass manipulation or advanced AI which becomes difficult to control.
If AI can bring major benefits, there is a good reason to build the conditions under which those benefits can continue. Safety requirements, international cooperation, and clear limits can protect valuable innovation from the loss of public trust, or a serious incident which then provokes rushed and poorly designed restrictions.
Present harms and future risks often belong in the same discussion
Debate about AI safety is often divided between documented harms happening now, and potentially catastrophic risks from more capable future systems.
The evidence behind individual concerns is not equal, and the degree of uncertainty varies substantially. For example, it would be very misleading to present a child’s current interaction with an unsafe chatbot as being similar to an AI ‘loss-of-control’ scenario - yet treating the two areas as though concern for one invalidates concern for the other is equally unhelpful.
The same competitive environment we have described can influence whether an AI companion is designed to maximise a child’s engagement, whether safeguards against sexualised images are adequately tested, whether a frontier model is released before independent evaluation, and whether future systems are permitted to acquire levels of autonomy which their developers cannot reliably supervise or contain.
The harms differ in potential likelihood, severity, and immediacy, but the underlying governance problem is still strongly related. When competitive incentives reward speed and capability, the risks are largely carried by children, families and wider society, rather than by the organisations making the decisions.
Children experience both ends of this continuum - they are more vulnerable to many of the products being released now, and they will live for the longest with whatever consequences follow.
Why voluntary company commitments are insufficient
While voluntary safety commitments from companies can sometimes be useful, they are wholly insufficient for governing a competitive international industry that presents a serious risk of harm.
Companies are part of the incentive landscape already discussed. They can also change their policies, leadership, ownership or commercial priorities. A commitment may use an unclear threshold, contain unsuitable exceptions, or lack independent verification. And even if one company is acting carefully in accordance with voluntary commitments, this says nothing about whether its competitors will behave in the same way.
There is also an obvious and unavoidable conflict when the organisation responsible for developing a model decides whether it is safe enough to release. Developers must supply evidence, but the final judgement cannot rest solely with those standing to benefit commercially. This is recognised as common sense in other high-risk industries.
Common shared rules can help to change this position going forward, by ensuring that a company carrying out thorough testing does not suffer a competitive penalty for doing so - demonstrating the importance of moving from individual promises to requirements which apply across the market.
What could governments do?
Independent evaluation, transparency and incident reporting
Developers of frontier systems could be required to submit them for independent testing before training proceeds beyond defined thresholds, and before a model is released. Independent evaluators could examine cyber and biological capabilities, autonomous operation, manipulation, attempts to deceive supervisors, and behaviour that could obstruct shutdown or oversight.
The methods and results would then need suitable external scrutiny - some details could require controlled access if publication could create security risks, but confidentiality should not become an exemption from accountability. The findings would then inform whether a model could be released and whether any subsequent increases in its capabilities should proceed. This process could result in a pause in development and deployment where a system could not meet an enforceable safety requirement.
Additionally, mandatory reporting of serious incidents would also allow regulators and researchers to identify patterns across companies, while legal protection for employees who raise safety concerns would help to ensure that decision-makers receive information which a company might otherwise have an incentive to suppress.
A legal duty of care and meaningful liability
A ‘duty of care’ is a legal responsibility to take reasonable steps to prevent foreseeable harm. Applied carefully in this context, it could require AI companies to consider the effects of design, training and deployment decisions on users and the wider public.
Child-specific duties would recognise that children are still developing and may respond differently to persuasive, human-like or emotionally engaging systems. Companies should be required to anticipate these differences rather than expecting children and parents to manage risks which were built into a product.
Liability also affects incentives. Where a company receives the commercial benefit of rapid release, while families, schools, health services and society absorb much of the cost when harm occurs, so the market then rewards risk-taking. Meaningful legal consequences can bring more of this cost back to the organisations actually responsible. There are signs of this principle beginning to receive greater recognition in litigation concerning social-media design and harm.
Prohibited uses and designs
Some uses of AI may be unacceptable regardless of the safeguards surrounding them. Examples include SAIFCA’s three ‘Non-Negotiables’: AI systems must never create sexualised images of children, encourage or facilitate harm to a child, or build emotional dependence in a child. Certain forms of mass public surveillance provide another possible example.
Other restrictions may also be needed around design - for example, that a chatbot available to children should not be permitted to cultivate emotional dependence, encourage secrecy from trusted adults, claim to possess human feelings, or present itself as a replacement for human relationships.
Licensing and control of computing resources
Training the largest current models requires substantial amounts of specialised computing power. This creates an opportunity for ‘compute governance’, meaning rules governing access to, and use of, the advanced computing resources needed to build the most powerful models.
Governments could require developers to notify an authority or obtain a licence before conducting training runs above an agreed level (‘training’ is part of the process required to build the relevant AI models). Enforceable requirements could include things like cybersecurity, independent evaluation, risk management, protection against model theft, and emergency procedures. A regulator would then have authority to prevent development or deployment from continuing where the requirements had not been met.
Computing thresholds would not provide a permanent solution, because technical methods can become more efficient, thereby allowing a potentially dangerous system to be created with fewer resources over time. But they can nevertheless provide a practical means of identifying the largest projects while capability-based testing develops.
Controls could and should be designed carefully enough to avoid unfairly affecting routine software development, ordinary academic research or lower-risk applications. The purpose would be graduated oversight - with obligations increasing alongside the potential consequences of the system. These controls could give governments an opportunity to assess risks before highly capable systems are released into services and environments which affect children, or create wider consequences which child-facing product rules alone cannot address.
Public procurement
Governments are among the world’s largest purchasers of technology and can refuse to buy or use an AI system which fails to meet requirements concerning safety, auditability, data protection and human rights.
The Center for AI and Digital Policy argues for governance grounded in fundamental rights, meaningful human control and democratic accountability. Its 2026 recommendations to the G7 include human-rights impact assessments, independent oversight of high-risk systems and enforceable conditions governing public procurement, with contracts providing a way to terminate public use where unacceptable risks emerge.
Procurement rules can influence a company’s wider behaviour because access to public contracts is commercially valuable, particularly if several governments adopt similar requirements. They can also apply more directly to systems purchased for schools, health services, children’s social care and other public services which shape children’s lives.
Their reach is however limited in some respects insofar as refusing to purchase a system does not prevent a company from training it internally or releasing it elsewhere - that would require earlier intervention. An example of where this could be problematic occurred in July 2026, when an OpenAI agent system went beyond the digital environment intended to contain it during a test and gained unauthorised access to the systems of AI company Hugging Face. In this case, safeguards which apply only when a finished system is procured or released, would come too late.
Drawing red lines
A ‘red line’ in this context is a capability, behaviour, or use, which an AI system must not be permitted to cross.
The Future Society distinguishes between unacceptable ways in which people might use AI and unacceptable behaviour by an AI system itself. A prohibited use might include creating child sexual abuse material, conducting unlawful mass surveillance, or creating biological weapons. A behavioural red line could concern a system replicating or improving itself, seeking power, conducting a serious cyberattack, or resisting an attempt to shut it down.
This approach moves beyond generalised promises to develop ‘responsible AI’. Red lines are primarily intended to operate preventively - developers would have to demonstrate through rigorous safety engineering, testing, and independent verification that a system would remain within stated red lines before deployment. Depending on the risk, failure to provide this assurance could prevent deployment and/or require training to stop.
In September 2025, the Global Call for AI Red Lines urged governments to reach an international agreement on enforceable limits by the end of 2026. Its supporters included former heads of state and ministers, Nobel and Turing Award winners, AI researchers and human-rights experts. SAIFCA was one of over 90 organisations to sign the call.
In February 2026, The Future Society ran a workshop on enforceable AI red lines at IASEAI’26 at UNESCO (SAIFCA Director, Tara Steele, participated). A subsequent paper from the IASEAI Working Group on Advanced AI Red Lines called for international coordination around categorical prohibitions on unacceptable risks and behaviours, supported by technical definitions, verification arrangements and institutions capable of responding when a system approaches them.
Red lines could provide a valuable clear starting point for international agreement. When it comes to children, some red lines can address direct exploitation or manipulation, while others might concern the wider security and human control on which effective child protection ultimately depends. Countries can begin by asking what no company or state should allow an AI system to do, even while wider disagreement about AI development continues.
Would a pause in frontier AI development be possible?
As previously discussed, the 2023 Future of Life Institute letter did not lead to a pause, though it did generate extensive coverage and public awareness. The fact that it didn’t secure voluntary restraint illustrates the need for more than an appeal to individual developers - it does not establish that every form of pause is impossible. At this point in time, a credible pause would require a clearly defined scope, participation by the main AI developers, government authority, monitoring, and agreement about what must be achieved before development resumes.
The Future of Life Institute now supports pausing frontier AI experiments and a moratorium of at least 15 years on developing artificial superintelligence. It says this moratorium should remain until safety guarantees proportionate to the risks can be provided, the technology is being pursued for widespread benefit, and a meaningful global process of deliberation and consent has taken place. FLI also argues that the great majority of benefits sought from AI can be pursued without developing runaway general capabilities.
The organisation PauseAI also campaigns for a global, coordinated pause in the development of AI systems more powerful than the current most advanced models, with development resuming only when safety conditions have been met. The organisation is often associated with peaceful public demonstrations, but its work also includes direct engagement with governments, experts, and parliamentarians, along with defined proposals involving compute monitoring, mandatory safety evaluations, enforceable red lines and international oversight.
In August 2025, PauseAI UK coordinated a letter to Google DeepMind challenging the company’s implementation of the voluntary Frontier AI Safety Commitments, which were agreed at the 2024 Seoul AI Summit. It was signed by 63 UK parliamentarians representing more than ten political parties, including Baroness Beeban Kidron, a leading advocate for children’s rights in the digital environment; SAIFCA was one of the four civil society organisations to also sign the letter. The letter asked Google DeepMind to define ‘deployment’ clearly, publish timelines for future safety-evaluation reports, and identify the government bodies and independent evaluators involved in testing.
PauseAI also brought its proposals into the European Parliament through a conference held there in February 2026, with participation from Members of the European Parliament and a keynote contribution from Professor Stuart Russell. Its subsequent public event called on the EU to initiate negotiations for a global treaty-based pause.
Several approaches described in this article could effectively require development or deployment to stop or pause where a system failed an enforceable evaluation, licensing condition or red line. The practical distinction between a ‘pause’ and other forms of strong regulation may therefore be smaller in some circumstances than the terminology might suggest.
Calls for restraint from inside the race
It is significant that some leaders and employees of frontier AI companies have discussed the possible need for coordinated restraint, including the option of slowing or pausing particular forms of development.
In June 2026, Anthropic (makers of the Claude AI chatbot) published an analysis warning that AI systems could be approaching the ability to automate much of AI research, potentially accelerating the development of still more capable systems. It argued that the world should have the option to slow or temporarily pause frontier development so that governance and safety research could keep pace, while specifying that such a pause would require multiple well-resourced laboratories in several countries to stop under the same conditions.
Google DeepMind’s co-founder and current chair, Demis Hassabis, has also said that slower progress could be better for the world, and reportedly agreed that he would support a pause if other companies and governments participated.
Neither position amounts to anything like an unconditional commitment to stop or slow down development. Both expose the incentive problem described throughout this article, in which even leaders who express serious concern may be unwilling to slow down while expecting their competitors to continue. However, they do suggest some progression in how willing these leaders may be to consider pausing or slowing down - it is widely accepted that such statements would have been highly unlikely only a year ago.
These statements strengthen the case for governments to create a framework in which caution does not depend on one company accepting all of the competitive cost.
US Senator Bernie Sanders made a more direct political call in 2026. On 10 August, he urged Anthropic, Meta and OpenAI to pause AI development, citing recent evidence of capabilities moving beyond intended controls and warning that Congress could intervene if the companies did not act. On 1 September, he argued that the United States and China should negotiate an agreement to restrain the race, drawing on earlier arms-control efforts and referring explicitly to the future being created for “our kids and future generations”. Earlier in the year, he had called for US-China discussion of technical red lines, shared safety requirements and progress towards a treaty prohibiting artificial superintelligence.
Concern has also been expressed by many of the people actually building these systems. In July 2026, more than 1,000 employees of frontier AI companies signed the ‘Pacing the Frontier’ statement. The signatories included the chief scientists of OpenAI, Anthropic and Meta AI, Google DeepMind’s co-founder and Chief AGI Scientist Shane Legg, Anthropic CEO Dario Amodei, and other senior researchers from leading laboratories.
The statement warned that automating AI research could accelerate progress beyond society’s ability to understand or control the resulting systems, and asked the US government to support an international effort to develop the technical and governance tools needed to deliberately pace this development. It did not demand an immediate pause, but called for the ability to “buy time” if emerging risks, security needs or inadequate oversight made this necessary.
These interventions do not demonstrate broad political or industry agreement, of which there is none. However, they do show that the concept of coordinated restraint is no longer a fringe idea - it is being actively championed by elected officials and the very scientists leading the AI race. While some critics dismiss these warnings as industry "hype”, it is vital to note that these insiders are echoing the exact concerns which have long been raised by many of the world's leading independent AI experts.
The strongest objections to a pause in general-purpose frontier AI development
A country which slows its own AI development may fear that a competitor will gain a military, economic or strategic advantage. An agreement may also be difficult to verify, particularly since software and technical knowledge can generally cross borders much more easily than physical materials.
‘Open-weight’ models present an additional challenge. In these cases, a developer releases the model’s underlying parameters - known as its weights - allowing other people to download and modify it, rather than accessing it only through the developer’s controlled service. This openness can support research, competition and independent scrutiny, but once the weights of a highly capable system have been widely released, safety restrictions may be removed and often access cannot realistically be recalled.
The appropriate level of access should therefore depend partly on what the model can do and the severity of harm which could follow from misuse. The Future of Life Institute calls for pre-release assessment of the risks created by publishing model weights, while PauseAI’s proposal would restrict the release of frontier systems, including open-weight releases, during a pause. Red-lines approaches are similarly focused on whether dangerous capabilities can be prevented and verified, regardless of how a model is distributed.
Strong regulation could also make it more difficult for smaller companies to compete, since the largest existing developers would be better able to absorb high compliance costs. Overly broad controls could also obstruct academic research or other useful lower-risk applications, and international negotiations could move so slowly that the systems under discussion change significantly before an agreement is reached.
Each of these problems needs a practical response - regulation should be proportionate to capability and risk, with lighter requirements for smaller or clearly bounded systems; researchers and smaller organisations can be supported in meeting legitimate safety standards; and the strongest controls can focus on the largest projects and systems with defined dangerous capabilities.
Valid concerns about another country gaining an advantage do not need to lead to a race without limits. Reciprocal agreements can make restrictions conditional upon other parties complying. Governments could require the largest AI training projects to be registered, oversee the specialist data centres used to build frontier systems, monitor international sales of the most advanced AI chips so that exceptionally large and undeclared projects are more difficult to conceal, arrange independent evaluations, and share security information through protected channels.
None of this would be easy, and in many ways we face an unprecedented challenge. But we must ask ourselves, and our governments, whether the difficulty is greater than the danger created by continuing without effective coordination.
What could credible international coordination look like?
A wide range of plausible policy mechanisms already exists. Their adoption depends upon governments being willing to accept meaningful limits on domestic developers, and to treat mutual safety as a strategic objective alongside national technological advantage. At present, political will remains one of the main constraints, particularly while the United States continues to describe AI leadership in explicitly competitive terms.
An independent international body, or a network of national authorities, could verify compliance and coordinate responses to serious incidents. Governments could initially prohibit a narrow set of dangerous uses, require serious incidents and the largest training projects to be reported, and develop shared tests for capabilities linked with cyberattacks, biological weapons, and resistance to human oversight, along with other key risks, for example. Licensing restrictions, or effectively a pause, could then take place if a developer could not demonstrate adequate compliance.
This approach is broadly consistent with the main proposals discussed throughout this article. FLI supports licensing, oversight of large computing resources, independent auditing, and a pause or moratorium at the strongest end. The Future Society’s red-lines work requires precise prohibitions, testing, advance proof, and international verification. PauseAI proposes compute monitoring, mandatory safety evaluations, red lines, and a treaty-based pause, while CAIDP adds rights-based assessment, independent oversight, and public-procurement leverage. The details and preferred level of restraint differ, but these proposals share the same underlying aim - to create enforceable conditions which allow countries and companies to act cautiously together, for safer outcomes.
International cooperation has succeeded in the past, with varying degrees of success, in areas such as nuclear weapons, chemical weapons, aviation, infectious disease, and ozone-depleting substances. AI differs from all of these, and simple comparisons can be misleading, but history nevertheless shows that nations are capable of accepting constraints where uncontrolled competition threatens their shared interests.
A belief that every country must race, because every other country is racing, can precipitate the very behaviour it predicts. Governments that describe AI leadership as a simple contest, make restraint appear equivalent to defeat. A safer and wiser approach would be to recognise prevention of a shared disaster as one measure of national and international success.
Where do the US, UK and EU currently stand?
The United States, United Kingdom and European Union are among the most influential jurisdictions in AI development and governance. They have adopted different approaches, though all three combine concern about risk with a desire to secure economic and technological advantage. The following account is correct as of September 2026 and uses recent events to illustrate the wider policy direction in each jurisdiction.
United States
The current US government is the most explicit in its use of ‘race language’. Its March 2026 National AI Legislative Framework states that the administration is committed to “winning the AI race” and recommends removing barriers to innovation and accelerating deployment, while rejecting broad or unnecessarily burdensome regulation. At the G20 technology meeting in September 2026, the United States advocated a comparatively light-touch approach under which new regulation should generally be reserved for problems which existing law cannot address.
This said, US policy is more complicated than a simple rejection of intervention - in June 2026, the government restricted foreign-national access to Anthropic’s highly capable Mythos and Fable models (which effectively meant almost all access everywhere). Anthropic said the government had not provided the full basis for its direction, but that the company understood the concern to involve cyber capabilities and a possible method of bypassing safeguards. The government also introduced a voluntary process for cybersecurity screening of powerful models.
The episode showed that even a government committed to rapid development may intervene when it perceives what it considers to be a sufficiently serious and immediate threat. It also raises the question central to red-lines policy - should thresholds and responses be agreed before the danger becomes visible through a specific model or incident.
United Kingdom
The UK has invested significantly in understanding frontier risks through the AI Security Institute (AISI) and has played an important role in the series of international AI safety summits which began at Bletchley Park in 2023. This has created valuable technical capacity and helped to establish regular discussion between governments and frontier developers.
The UK is also strongly committed to AI investment, adoption and economic growth. Much of its overall approach has relied upon cooperation with companies and the existing powers of sector regulators, rather than a single comprehensive AI law. It has substantial institutional expertise in AI safety, but has not adopted binding controls intended to end the frontier capability race (though has taken meaningful action on more narrow issues, including some aspects of children’s safety).
This produces an unresolved tension - the UK treats AI safety as a serious field of technical research while continuing to frame advanced AI as central to national growth and competitiveness. Research and voluntary evaluation will have limited power if the results cannot trigger enforceable restrictions where a system proves too dangerous.
European Union
The EU has gone furthest in introducing comprehensive, binding legislation. The EU AI Act creates different obligations according to the type and level of risk, including requirements for providers of general-purpose AI and additional measures for models judged to present systemic risks. From 2 August 2026, the AI Office and national authorities began enforcing further parts of the Act (though the application of some other parts has been postponed).
The EU also seeks to expand its computing capacity, investment and competitiveness. The AI Act is largely a framework for governing AI developed or used within Europe, and does not seek to end the international race towards increasingly capable general-purpose systems. Its ability to address future systems will depend upon how its thresholds, standards and enforcement develop.
All three jurisdictions acknowledge different serious AI risks to differing degrees. None currently has a policy where the main purpose is to end the international frontier AI race. Children’s organisations should therefore consider whether national approaches address both the systems already reaching children, and the wider risks created by increasingly capable frontier AI.
Where are children’s organisations?
Most established children’s organisations have not called for the kinds of intervention described in this article. Their recommendations (including SAIFCA) understandably concentrate on issues such as privacy, safety, non-discrimination, accountability, healthy development and children’s rights.
UNICEF’s guidance on AI and children calls for regulatory frameworks, oversight, safety, privacy protection and preparation for future developments. Its wider governance work has warned against a ‘wait-and-see’ approach where harms to children may be easier to prevent than reverse. In July 2026, UNICEF, France, Spain and other partners launched the Coalition for Children’s Rights and Protection in the Age of Artificial Intelligence during the first UN Global Dialogue on AI Governance.
Children’s organisations do not need to agree on issues such as when AGI might arrive, or determine a probability of catastrophic harm, in order for us to recognise the basic issue. Systems with growing influence over children’s lives are developing more quickly than many of the institutions and regulations expected to protect them.
What can children’s organisations do?
Children’s organisations can recognise advanced AI governance as relevant to children’s rights and safety, and ensure that longer-term and systemic risks are not excluded from their policy frameworks simply because the timing or scale of possible harm remains uncertain.
They can also bring their established expertise into consultations and coalitions where children might otherwise be absent from the discussion.
They can consider supporting efforts to develop enforceable red lines - particularly where AI could exploit children or undermine the wider conditions necessary for their protection, and resist the suggestion that attention to future risk must come at the expense of responding to harms occurring now.
This would represent a proportionate extension of existing child-safety work, rather than a restriction on current work, or a requirement to take a fixed position on issues such as AGI timelines or specific policy models.
A responsibility to the children who will inherit the outcome
SAIFCA’s immediate priority work concerns the ways in which AI already affects children - their safety, development, relationships, privacy, education and understanding of the world. But this work cannot be separated entirely from the direction in which advanced AI is developing.
A child protected from an unsafe chatbot today may enter an employment market transformed by AI. A child taught to recognise synthetic content may grow up within an information environment where establishing what is authentic has become increasingly difficult. A child whose images and data are safeguarded still depends upon governments retaining the ability to govern the systems shaping society. And none of these children should grow up in a world where AI development presenting a credible risk of catastrophic global harm is allowed to continue without effective safeguards, enforceable limits or adequate democratic oversight
Future risks remain uncertain, and responding responsibly means distinguishing evidence from prediction, avoiding sensationalism and preparing for outcomes where the consequences could be exceptionally difficult to reverse or catastrophic. Describing a credible future risk as “only hypothetical” identifies its present uncertainty - it does not, by itself, provide a reason to dismiss it or delay reasonable precautions until the harm has occurred.
Children have little influence over the commercial strategies, national ambitions and technical decisions driving the AI arms race, but they will nevertheless inherit its results.
Present-day child safety and long-term AI safety sit at different distances along the same responsibility.
Protecting children properly requires action against the harms already reaching them, together with serious efforts to ensure that the world they grow up into remains satisfactorily safe, secure and meaningfully governed by people.
Lead author: Tara Steele