ClearFactAll topics

Do robot and AI-driven tutoring systems meaningfully improve student learning outcomes?

10 peer-reviewed studies for, 5 against

Key takeaways

Stronger evidence: For

Verdict

On balance the evidence supports the claim that AI-driven tutoring systems meaningfully improve learning outcomes, but with an important distinction between software-based intelligent tutoring systems (ITS) and physical, embodied social robots. The strongest, most rigorous evidence comes from meta-analyses of intelligent tutoring systems. VanLehn (Educational Psychologist, 2011) found step-based ITS produce a learning effect of d = 0.76, essentially matching one-on-one human tutoring (d = 0.79). Kulik and Fletcher (Review of Educational Research, 2016) found a median gain of 0.66 SD over conventional instruction across 50 controlled evaluations, and Ma et al. (Journal of Educational Psychology, 2014) found an average g = 0.43 across 14,321 participants. These are large, replicated effects from adaptive, individualised software tutoring. For embodied robot tutors specifically, controlled experiments by Leyzberg et al. (2014) and a classroom field study by Baxter et al. (PLOS ONE, 2017) show that personalised social robots can improve child learning relative to non-personalised versions. However, the "against" evidence is a genuine and necessary caveat. Kanda et al. (2004) showed novelty effects fade within a week; Kennedy et al. (2015) found that adding social behaviour to a robot tutor actually eliminated the learning gain; Vogt et al. (2019), in a well-powered study of 194 children, found a social robot was no better than a tablet app; Steenbergen-Hu and Cooper (2013) found near-zero ITS effects for K-12 math and negative effects for low achievers; and Serholt (2018) documented interaction breakdowns that erode engagement over time. The reconciling picture: adaptive AI tutoring software reliably improves outcomes and rivals human tutors, so the core claim holds. But the benefit is concentrated in personalisation and adaptivity rather than in the physical robot itself — embodied social robots add cost and novelty risk without consistently beating a good tablet app, and poor interaction design can wipe out or reverse the gains. The claim is supported, with the caveat that "how" the system is designed matters more than "whether" it is a robot.

Strongest study on each side

Ranked by citation count.

Strongest study for

Social Robots: A Meta-Analysis of Learning Outcomes (Frontiers in Robotics and AI, 2025)

J. C. F. de Winter, D. Dodou et al. · 146 citations

Comprehensive meta-analysis of 146 studies examining 183 post-test effect sizes between social robots and control groups. Found robots significantly improved learning compared to no-training controls (d = 0.75), demonstrating effectiveness for cognitive skill training when deployed as tutors or teaching assistants.

Strongest study against

Second Language Tutoring Using Social Robots: A Large-Scale Study (ACM/IEEE Int. Conf. on Human-Robot Interaction, 2019)

Paul Vogt, Rianne van den Berghe et al. · 137 citations

In a well-powered study of 194 young children learning English vocabulary over seven lessons, Vogt and colleagues found that children taught by a social robot learned and retained words to a similar extent as children using a tablet application alone. The robot provided no significant learning advantage, and its iconic gestures added no measurable benefit.

Open the interactive page →

The evidence

For (10)

Social Robots: A Meta-Analysis of Learning Outcomes (Frontiers in Robotics and AI, 2025)

J. C. F. de Winter, D. Dodou et al. · 146 citations

Comprehensive meta-analysis of 146 studies examining 183 post-test effect sizes between social robots and control groups. Found robots significantly improved learning compared to no-training controls (d = 0.75), demonstrating effectiveness for cognitive skill training when deployed as tutors or teaching assistants.

Effects of Intelligent Tutoring Systems on Educational Outcomes: Evidence from a Comprehensive Analysis (International Journal of Distance Education Technologies, 2025)

Xiaoli Huang, Wei Xu et al. · 30 citations

Meta-analysis of 30 studies examining ITS effects on learning outcomes across countries and educational levels. Found that ITSs significantly improve learning attitudes and test scores (g=0.86), though effects on knowledge acquisition, motivation, and problem-solving skills vary. Game-integrated ITSs and those with worked examples showed most positive impact.

Personalizing Robot Tutors to Individuals' Learning Differences (ACM/IEEE Int. Conf. on Human-Robot Interaction, 2014)

Daniel Leyzberg, Samuel Spaulding et al.

In a controlled experiment on grid-based logic puzzles, Leyzberg, Spaulding and Scassellati showed that learners taught by a personalized (skill-adaptive) social robot achieved significantly greater cognitive learning gains than those given non-personalized or randomly ordered lessons. This is direct evidence that an embodied robot tutor's adaptive lesson selection improves measured problem-solving performance.

The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems (Educational Psychologist, 2011)

KURT VanLEHN

VanLehn reviewed controlled experiments and found that step-based intelligent tutoring systems produced a learning effect of d = 0.76 over no-tutoring controls — essentially matching one-on-one human tutoring (d = 0.79). This overturned the long-standing assumption that human tutors (the '2-sigma' benchmark) vastly outperform automated tutors, showing well-designed ITS are nearly as effective as expert human tutors.

Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review (Review of Educational Research, 2016)

James A. Kulik and J. D. Fletcher

Across 50 controlled evaluations, Kulik and Fletcher found the median intelligent tutoring system raised test scores by 0.66 standard deviations over conventional instruction — moving a median student from the 50th to roughly the 75th percentile. Effects were larger on locally developed tests closely aligned to the instruction than on broad standardized tests.

Intelligent Tutoring Systems and Learning Outcomes: A Meta-Analysis (Journal of Educational Psychology, 2014)

Wenting Ma, Olusola O. Adesope et al.

Pooling 107 effect sizes across 14,321 participants, Ma and colleagues found ITS raised achievement by an average of g = 0.43 relative to comparison conditions. The advantage over teacher-led large-group instruction was g = 0.42, and ITS also outperformed non-ITS computer instruction and textbooks — though they scored about 0.25 SD below one-on-one human tutoring.

Robot Education Peers in a Situated Primary School Study: Personalisation Promotes Child Learning (PLOS ONE, 2017)

Paul Baxter, Emily Ashurst et al.

In a situated primary-school field study, Baxter and colleagues found that children who interacted with a personalising social robot peer showed greater learning outcomes and more positive behavioural responses than children with a non-personalising robot. The result supports personalised embodied robot tutoring working in a real classroom, not just a lab.

Against (5)

Second Language Tutoring Using Social Robots: A Large-Scale Study (ACM/IEEE Int. Conf. on Human-Robot Interaction, 2019)

Paul Vogt, Rianne van den Berghe et al. · 137 citations

In a well-powered study of 194 young children learning English vocabulary over seven lessons, Vogt and colleagues found that children taught by a social robot learned and retained words to a similar extent as children using a tablet application alone. The robot provided no significant learning advantage, and its iconic gestures added no measurable benefit.

Breakdowns in Children's Interactions with a Robotic Tutor: A Longitudinal Study (Computers in Human Behavior, 2018)

Sofia Serholt · 80 citations

Serholt's longitudinal classroom study documented recurring breakdowns in children's interactions with a robotic tutor — the robot failing to recognise or respond appropriately and breaching social expectations. These breakdowns accumulated across repeated sessions and undermined engagement and the tutoring relationship, showing that real-world robot tutors struggle to sustain effective interaction beyond short, novelty-driven trials.

Interactive Robots as Social Partners and Peer Tutors for Children: A Field Trial (Human-Computer Interaction, 2004)

Takayuki Kanda, Takayuki Hirano et al.

In a two-week English-tutoring field trial with elementary students, Kanda and colleagues found that children's interaction with the robot and its novelty declined sharply after the first week. Only the minority of children who continued interacting showed measurable English gains — a clear demonstration of a strong novelty effect and limited sustained learning benefit.

The Robot Who Tried Too Hard: Social Behaviour of a Robot Tutor Can Negatively Affect Child Learning (ACM/IEEE Int. Conf. on Human-Robot Interaction, 2015)

Kennedy, Baxter and Belpaeme found that children learning about prime numbers with a robot that used additional social and adaptive behaviours showed NO significant learning gain, whereas children in the less-social tutoring condition did improve. Increasing the robot's social behaviour actively harmed learning — evidence that the embodied tutor's benefit is fragile and highly sensitive to design choices.

A Meta-Analysis of the Effectiveness of Intelligent Tutoring Systems on K-12 Students' Mathematical Learning (Journal of Educational Psychology, 2013)

Saiying Steenbergen-Hu and Harris Cooper

Across 26 reports (34 samples), Steenbergen-Hu and Cooper found intelligent tutoring systems had only a near-zero average effect on K-12 mathematics learning (about g = 0.05), and the effect was negative for low-achieving students (g = -0.18). Short-term ITS use produced no significant benefit over regular classroom instruction — a sharp contrast with the larger effects reported for college-age learners.

The debate

Lisa: The headline result here is genuinely strong. VanLehn's 2011 review in Educational Psychologist found intelligent tutoring systems hit an effect size of 0.76 — basically the same as a one-on-one human tutor. For decades people said machines couldn't touch human tutoring, and that turned out to be wrong.

James: For software tutoring, sure. But the topic is robotic tutoring, and when you put an actual robot in the room the story gets much shakier. Vogt and colleagues ran a large study in 2019 — 194 kids learning English vocabulary — and the social robot was no better than a plain tablet app. If the robot adds nothing over a screen, what exactly are we paying for?

Lisa: That's a fair distinction, but the robot studies aren't all negative. Leyzberg and Scassellati showed a personalised robot tutor produced significantly bigger learning gains on logic puzzles than a non-personalised one, and Baxter's 2017 classroom study in PLOS ONE found the same thing with real primary-school children. When the robot adapts to the individual student, learning improves.

James: When it's designed exactly right, yes. But that's the problem — it's fragile. Kennedy's 2015 paper is almost a warning label: they added more social behaviour to the robot, the kind of thing you'd assume helps, and the learning gain disappeared. The paper is literally titled "The Robot Who Tried Too Hard." Small design choices can flip the result from positive to nothing.

Lisa: I'll grant that robots are sensitive to design. But look at the weight of evidence on the AI-tutoring side. Kulik and Fletcher's meta-analysis found a median 0.66 standard deviation gain across 50 evaluations, and Ma's meta-analysis pooled over 14,000 participants and still found a solid 0.43. Those are big, replicated numbers. The core technology works.

James: In the right population. Steenbergen-Hu and Cooper found almost nothing for K-12 math — an effect around 0.05 — and it was actually negative for low-achieving students. The impressive averages come mostly from college-age learners. The students who arguably need the most help sometimes got hurt.

Lisa: That tells me deployment matters, not that the approach fails. The consistent thread is personalisation and adaptivity. Where the system genuinely adapts to the learner, outcomes go up and rival human tutors. Where it doesn't, or where the interaction breaks down like in Serholt's longitudinal study, the benefit evaporates.

James: So maybe we actually agree on the mechanism. Adaptive AI tutoring software has strong, well-replicated benefits. The physical robot is optional — sometimes it helps engagement, but it also brings novelty effects, breakdowns, and cost. Invest in the adaptivity, and be skeptical of the idea that adding a robot body is what makes the difference.

Frequently asked questions

Are intelligent tutoring systems as effective as human tutors?

Some evidence suggests they come close. One review found step-based tutoring systems produced learning effects nearly equal to one-on-one human tutoring, while another meta-analysis found they scored somewhat below human tutors but above classroom instruction and textbooks.

Do robot tutors work better than tablet apps?

Not consistently. A well-powered study of 194 children found a social robot produced similar vocabulary learning to a tablet app alone, suggesting the benefit comes mainly from personalisation rather than the physical robot.

Do AI tutors help K-12 students in math?

The evidence is mixed. One meta-analysis found near-zero effects on K-12 math and negative effects for low-achieving students, while a more recent analysis found modest positive effects that held for low achievers.

Why do some robot tutors fail to improve learning?

Studies point to novelty wearing off within about a week, interaction breakdowns that erode engagement over time, and design choices such as added social behaviour that can eliminate learning gains.

Related questions