What Educate Ventures Research Independent Evaluation Found About High-Impact AI Maths Tutoring

Ask any school leader what they think of AI in education right now, and you’ll get a careful, qualified answer. There’s interest, but also a reasonable level of scepticism. Schools have seen enough overpromising tech to want real evidence before they commit pupil time to anything new.

AI tutoring is still new, which means independent evaluations of it are few and far between. So when one does land, it’s worth reading carefully.

Educate Ventures Research recently published its findings on Skye, Third Space Learning’s AI maths tutor. They looked at two questions: does Skye actually align with what the research says about high-impact tutoring, and is there evidence that it’s working for pupils?

Whether you’re already using AI in your school or still deciding whether to bring it in, these findings give school leaders something concrete to weigh up.

What the evaluation set out to do

Educate Ventures Research is an independent research and evaluation organisation working across the EdTech sector. They aim to give schools an evidence-based view of education technology, so leaders can make better-informed decisions about what they bring into their classrooms.

For this case study, Educate Ventures Research focused on Skye, Third Space Learning’s AI maths tutor for primary and secondary pupils, in two parts:

1. A rapid evidence review, looking at how closely Skye aligns with the core principles that research identifies as essential for high-impact tutoring.
2. An early-stage impact review, drawing on session-level data from pupils using Skye in school, to see what the early evidence is telling us about pupil outcomes.

Educate Ventures Research findings

The findings from the research span four areas:

1. Pedagogical alignment
2. Evidence of impact
3. Who benefits most
4. The wider question of ethical AI in schools

1. Skye aligns with the evidence for high-impact tutoring

For tutoring to do real work, the research is reasonably clear about what needs to be in place. Pupils need to be working at the edge of what they can do, with the kind of structured support that breaks complex problems into manageable parts. They need quick, useful feedback when they get stuck. And the tutor needs to guide rather than just hand over answers.

Educate Ventures Research looked at AI tutoring with Skye against those principles and found strong alignment across four areas.

1. Scaffolded learning 

Skye structures each session around an “I do, we do, you do” model. The tutor models a method, then works through a problem alongside the pupil, then gradually steps back as the pupil takes over. It’s a familiar approach for any teacher, because it sits at the heart of how scaffolded maths instruction is supposed to work.

2. Immediate, targeted feedback 

When a pupil gets something wrong, Skye doesn’t just say so and move on. It uses probes, helps and corrections to walk the pupil back to the point of confusion, then forwards again. It’s the same kind of in-the-moment, diagnostic feedback that effective one-to-one tutors give – it just happens to come from an AI.

3. Guidance over answer generation 

This is a distinction Educate Ventures Research draws out carefully. Skye is built to help pupils reach the answer themselves, rather than handing it to them. That keeps the cognitive load with the pupil, where the learning happens.

4. Teacher agency 

Topic selection and pupil participation stay in the teacher’s hands. Skye doesn’t decide what pupils are working on – the teacher does.

2. The early evidence of impact on pupils

Alignment with the research is one thing. Pupils actually learning more is another. The second half of the evaluation looked at session-level data from pupils using Skye, and the early signals are encouraging.

Within-session learning gains

Each Skye session begins with a short diagnostic check-in and ends with a check-out assessment. Across the sessions, the evaluators looked at:

• Pupils answered 34% of check-in questions correctly at the start of the session
• The same pupils answered 92% of the check-out questions correctly at the end

That’s a substantial within-session shift. It suggests the targeted practice and feedback during the session is doing what it’s designed to do – moving pupils from shaky to secure on the specific concept they came in working on.

Confidence gains during and across sessions

The evaluators also looked at how pupils’ self-reported confidence changed during and between sessions. Two findings stood out:

• 59.4% of pupils ended their sessions with higher confidence than they started
• 66% of pupils showed session-on-session confidence increases over time

For schools running maths intervention groups, those within-session gains line up with the case for short, focused, targeted practice. The session-on-session confidence pattern is the more interesting finding, though. A pupil who comes out of a session feeling more capable than they went in – and who keeps feeling that way week on week – is a pupil more likely to engage with maths in their main lesson too.

3. The pupils benefiting most from AI tutoring

Two groups of pupils stood out in the Educate Ventures Research evaluation as particular beneficiaries of Skye.

Anxious or quiet pupils

The first is the group every teacher recognises – the pupils who don’t put their hand up. They might be quietly capable, or quietly struggling, or somewhere in between. Either way, they hesitate to ask for help in a busy classroom, and especially in maths, where maths anxiety and the fear of looking like you don’t understand can be loud.

These pupils, the evaluators found, often respond differently to Skye. There’s no audience. No one to overhear them getting an answer wrong. They can take the time they need to work something out, ask for help, try again, and get it right – without the social cost that comes with doing the same in a classroom.

For SLT and maths leads, thinking about how to reach the “middle pupils” who often slip through intervention plans, this is the part of the evaluation worth sitting with.

Pupils needing immediate consolidation after teaching

The second group is also familiar: pupils who’ve just been taught something new in a maths lesson and need to practise it before it slips. Working memory in maths is a known constraint, and the time between input and consolidation is when learning is most fragile.

Skye is well-suited to that gap. Adaptive, targeted practice straight after teaching gives pupils a low-stakes way to consolidate what they’ve just learned, with the kind of corrective feedback a busy teacher can’t always provide pupil by pupil.

See how schools like yours are using AI tutoring to close the maths attainment gap.

4. Ethical AI in schools: strengths and ongoing work

It would be easy for a case study like this to skip over the harder questions, but Educate Ventures Research doesn’t. The evaluation looks at how Skye performs against the things school leaders are right to be cautious about with AI, and gives a balanced picture.

Where Skye is doing the right things

The evaluation found Skye aligns well with several core ethical AI principles:

• Curriculum-appropriate content, anchored to UK maths content rather than generated freeform
• User agency preserved, with teachers retaining professional judgement over topic selection and pupil participation
• Age-appropriate dialogue, with the tutor’s tone and language calibrated to the pupils it’s working with
• Human-centred design throughout – Skye is built by teachers to support teaching, not replace it

For a school weighing up AI use in maths, those are useful confirmations. Pupils aren’t being taught by a generic chatbot. The teacher is still the teacher.

Where there’s ongoing work

The evaluators were also honest about where Skye doesn’t yet work as well as it could. Two groups in particular face challenges:

• Pupils learning English as an Additional Language (EAL), who can find it harder to communicate with Skye in spoken English
• Pupils with speech recognition difficulties, whose voices the system doesn’t always pick up reliably

This is the kind of honesty that’s missing from a lot of AI marketing. Inclusion in AI tools is an open challenge, and pretending otherwise doesn’t help anyone. For schools with significant EAL cohorts or pupils with speech needs, it’s worth weighing this up when planning how Skye fits into the wider intervention picture.

What one independent evaluation can and can’t tell us

The honest answer is: one evaluation isn’t enough to settle the AI tutoring evidence question for any school, even with plans from the DfE to implement AI tutoring to 450,000 disadvanatged pupils. And it shouldn’t be. AI in education is still new, and the wider conversation needs lots more independent evidence before anyone can sit comfortably with a definitive view.

But this evaluation does shift the conversation a little. If you’ve been waiting for someone other than the people selling AI tutoring tools to look properly at whether they work, this is a useful place to start. The pedagogy lines up. The early pupil-level data is showing real gains. The ethical questions are being taken seriously, including the ones that haven’t yet been solved.

For school leaders weighing up where AI fits in their maths intervention picture, that’s the kind of grounded, evidence-led starting point worth having.