Could AI Widen Inequalities Between Children?
Bias, access, and fair treatment - what parents and educators need to know
No child should be treated unfairly because an AI system handles their language poorly, makes assumptions about their background, or fails to accommodate their disability. Nor should a family's ability to pay determine whether a child receives appropriate educational support or can challenge a decision affecting them.
Schools and families are increasingly making important decisions about whether and how AI is used, and potential inequalities need to be part of those decisions. There are circumstances in which particular tools could help remove barriers to learning, but there are also ways in which AI could reinforce existing disadvantages or create new ones. And giving every child access to the same product would not, by itself, resolve these problems.
UNICEF's guidance identifies fairness, non-discrimination, and inclusion as essential requirements for AI that respects children's rights. Putting these principles into practice requires looking at which children a system works for, who bears the consequences if it fails, and whether its use actually improves the support children receive.
Access to what, and for whose benefit?
The ‘digital divide’ describes inequalities in access to technology and people's ability to use it. Factors such as a reliable internet connection, a suitable device, accessible software, and help from a knowledgeable adult can all affect whether a child can take part in an activity involving technology.
These differences need close attention when schools are considering AI. At the simplest level, if an assignment depends on a paid product or assumes that every pupil has their own device, some children will be disadvantaged before they even begin. Providing access also achieves little if a tool cannot accommodate a child's needs or work adequately in the language they use.
However, describing children as being ‘left behind’ because they do not use AI can make an untested assumption about what they are missing. An AI product could be useful, or it could be ineffective, inappropriate for their age, or harmful. Its availability and popularity do not necessarily tell us which of these applies.
UNESCO's education report argues that the suitability and educational value of technology needs to be demonstrated, and that ultimately decisions should focus on learning outcomes. Its warning about digitalisation benefiting already privileged learners sits alongside a clear expectation that technology should support human interaction in education.
At SAIFCA, we start with the basic questions of whether a particular use is safe, appropriate, and supported by evidence. Where it demonstrably helps children, access should be fair. But where it has not met that standard, expanding access cannot be assumed to also advance equality.
AI can treat children differently
AI systems are developed using data and design choices that can reflect existing inequalities. The Information Commissioner's Office explains that unbalanced data, discriminatory patterns, and the way systems are designed and used can result in less favourable treatment of particular groups.
One evidenced example is tools that claim to detect AI-written work. In a 2023 study, researchers tested seven detectors using human-written English essays. Across the detectors, the average false-positive rate for the sample of essays written by non-native English writers was about 61%. Performance was much better on essays by US eighth-grade pupils, which the researchers used as their native-English comparison sample.
This was a specific study of particular tools and samples and cannot be seen as a measure of every detector available today. Additionally, the essays also came from different writing contexts, so the comparison did not isolate language background alone. However, the findings demonstrate how a system intended to uphold fair assessment can expose some learners to a greater risk of being wrongly accused. A detector score should never be treated as sufficient proof that a child has cheated.
Bias can also persist when obvious discriminatory signs become less common. A 2024 Nature study found that the AI systems tested produced prejudiced judgements about speakers of African American English. Training with human feedback made the systems’ explicit statements about African Americans more positive, but did not reduce the prejudice triggered by dialect.
The research examined experimental tasks, including employment and criminal justice scenarios, rather than decisions about school pupils, but its relevance to education includes the mechanism it exposes. A system can produce discriminatory judgements in response to someone's language even when it avoids easily identified, explicitly racist statements. Educational uses therefore need their own testing, including attention to the children and communities affected.
Claims that a model has improved overall cannot answer every question about fairness. A school needs evidence about the task it proposes to use the system for, including whether bias and errors fall more heavily on some pupils than others.
Children whose needs are overlooked
Children with disabilities, children who are neurodivergent, and children who speak a minority language have varied needs and experiences. A tool that helps one child could be unsuitable for another, including another child with the same diagnosis.
For example, a proposed voice-based learning activity needs to accommodate pupils who cannot use spoken responses reliably. A tool used to interpret engagement would require careful scrutiny of the assumptions made about what attention ‘looks like’. These are questions to investigate before adoption, rather than reasons to assume that any named group will always benefit or will always be harmed.
UNICEF's detailed guidance highlights the needs of girls, children with disabilities, minority and Indigenous children, and those whose languages receive less commercial attention. It also warns that efforts to improve representation in AI data must not justify irresponsible collection of children's information.
Schools should ask suppliers which children were included in testing and what limitations were found. Families and pupils should have opportunities to explain practical difficulties, with suitable alternatives available where a product does not meet a child's needs. And if a child has difficulty using a system, that should prompt examination of the system and the arrangements around it.
There should also be room to decline its use. UNICEF recommends that children should be able to opt out of AI systems without losing educational opportunities. This is a useful principle for schools considering what alternatives they provide and how pupils who do not use a particular product will be supported.
Some children could face greater risks
Children's circumstances can also affect why they turn to AI and the risks they encounter. This is especially relevant when chatbots are used for emotional support or companionship.
In its 2025 UK research, Internet Matters found that 23% of children it classified as vulnerable who used AI chatbots said they did so because they had nobody else to speak to, compared with 12% of child chatbot users overall. (For the purposes of the research, ‘vulnerable’ meant children with an Education, Health and Care Plan, those receiving special educational needs support, or those with a physical or mental health condition requiring professional help).
The findings concern reported use, and do not establish how much harm different groups experienced, but raise a significant concern about children turning to chatbots when they feel they lack someone to talk to. A child seeking support in those circumstances deserves strong protection from unsafe advice and products that encourage emotional dependency.
Protection should also be available regardless of what families can afford. UNICEF's guidance calls for safe and responsible practices to be standard in free products as well as paid services. Families should not have to purchase an upgrade to secure essential safeguards for their children.
What evidence of benefit shows
There are specific applications of AI being developed to help with accessibility. UNICEF describes work using AI to help produce accessible textbooks, including narration and image captions. Such applications need assessment for accuracy, accessibility, and suitability, but they illustrate why different uses of AI should be considered separately. Supporting the production of accessible materials involves different decisions from giving a child a conversational chatbot.
There is also emerging evidence about tools designed to assist educators. A Tutor CoPilot trial, reported in a working paper updated in November 2025, involved more than 700 tutors and 1,000 pupils from underserved communities. Pupils whose tutors had access to the system were four percentage points more likely to master the maths topics assessed. The intervention provided suggestions to human tutors, who continued to work with the pupils.
This is evidence about a particular form of support within a tutoring programme - it does not establish that children benefit from unrestricted chatbot access, that AI can replace tutors, or that the same results would occur elsewhere. The study also does not resolve questions about longer-term learning or safety.
A separate high-school mathematics trial, published in PNAS in 2025, shows why those distinctions are necessary. Nearly 1,000 pupils at a school in Türkiye took part in a study comparing two GPT-4-based tools with a group receiving no AI assistance. Pupils using a tool designed to resemble ordinary chatbot assistance performed better during assisted practice, but worse when subsequently tested without it. A version with safeguards developed using teacher input largely avoided that harm, although it did not produce a statistically significant improvement in unassisted exam performance over the control group.
These findings suggest that completing work successfully with AI assistance should not be assumed to mean that a child has learnt more. Evaluations need to examine what children understand and can do independently, alongside the effects on their wellbeing and development.
Protecting access to human support
There is a further inequality risk in how services are funded and organised. If schools under financial pressure use AI to reduce access to teachers or specialists, while better-resourced settings preserve that support, children could receive increasingly different kinds of education.
This risk should be considered whenever a proposal promises savings through automation. Schools and policymakers need to establish what children would gain, what provision would be reduced, and whether the evidence supports the change.
The same scrutiny should apply when a tool is promoted specifically for disadvantaged pupils.
An unmet need creates a responsibility to provide effective help, but it does not lower the standard of evidence or protection those children deserve.
Human review also needs to be meaningful. Where AI contributes to a judgement about a pupil's work, behaviour, or support needs, the person reviewing it should have enough information and authority to question the result. And families should know how to raise concerns without having to understand the technology themselves.
What parents and schools can do
Parents can ask what AI tools a school uses, whether they affect decisions about their child, and who is responsible for reviewing those decisions. It is also reasonable to ask whether a tool is being introduced alongside existing teaching or specialist support, or whether that provision will change.
If a child is accused of using AI to complete their work, parents can ask which detection tool was used, what other evidence was considered, and how the child can explain their work. Drafts, notes, and previous work can help inform a review where they are available. A child should have a fair opportunity to respond, including where they have not kept those materials.
If a system repeatedly misunderstands a child or cannot accommodate their needs, parents can ask for an appropriate alternative and a review of whether the tool is suitable. It helps to identify the particular output or decision causing concern and explain its effect on the child. Children should also know that they can tell an adult if a tool produces upsetting outputs or stereotypes, or makes them feel uncomfortable.
Before adopting a system, school leaders and governors should be able to explain:
- Why it is being considered - Identify the educational or accessibility need, the evidence that the proposed use meets it, and the alternatives considered.
- Who it has been tested with - Look for evidence relevant to pupils' ages, languages, disabilities, and other needs, including known gaps in testing.
- What participation requires - Check costs, devices, connectivity, accessibility, and expectations of support at home, with suitable provision for pupils who cannot use it or whose families decline its use.
- How unfair treatment will be addressed - Establish a clear route for pupils and families to raise concerns, and ensure staff can correct or set aside an AI-supported judgement.
- How its effects will be assessed - Examine learning, wellbeing, and differences between groups, and be prepared to change or stop its use if the evidence is poor or problems arise.
Responsibility also lies with developers, suppliers, and governments. Individual schools often lack the resources and technical expertise to evaluate AI systems thoroughly, and families and teachers cannot be expected to uncover every weakness in products they did not design and cannot inspect. Suppliers should provide credible, usable evidence about performance and limitations, while governments should support independent evaluation, clear guidance, and effective oversight.
SAIFCA's position is that children's access to effective education, appropriate support, and fair treatment should guide decisions about AI. This requires scrutiny both when children are excluded from something that could help them, and when technology is introduced in ways that could disadvantage them. Every child deserves that protection.
SAIFCA
SAIFCA
SAIFCA
References
- UNICEF (2025). Guidance on AI and Children, version 3.0. Report.
- UNESCO (2023). Global Education Monitoring Report: Technology in education - A tool on whose terms? Recommendations.
- Information Commissioner's Office. What about fairness, bias and discrimination? Guidance. Cited for its explanation of bias, rather than as a summary of legal duties.
- Liang and colleagues (2023). GPT detectors are biased against non-native English writers. Paper.
- Hofmann and colleagues (2024). AI generates covertly racist decisions about people based on their dialect. Nature. Paper.
- Internet Matters (2025). Me, myself and AI: Understanding and safeguarding children's use of AI chatbots. Report.
- Wang and colleagues (2025 revision). Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise. EdWorkingPapers. Working paper.
- Bastani and colleagues (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. PNAS. Paper.