Simulating student learning for better AI tutoring

Simulating student learning for better AI tutoring - Microsoft Research

Skip to main content

Research

Publications Code, datasets and models

Artificial intelligence Audio & acoustics Computer vision Graphics & multimedia Human-computer interaction Human language technologies Search & information retrieval

Data platforms and analytics Hardware & devices Programming languages & software engineering Quantum computing Security, privacy & cryptography Systems & networking

Algorithms Mathematics

Ecology & environment Economics Medical, health & genomics Social sciences Technology for emerging markets

Academic programs Events & academic conferences Microsoft Research Forum

Microsoft Research Blog Microsoft Research Podcast Microsoft Research Stories

About Microsoft Research Careers & internships People News & awards

Asia-Pacific Cambridge Catalyst Lab India Montreal New England New York City Redmond Other research groups

Register: Research Forum

Microsoft Security Azure Dynamics 365 Microsoft 365 Microsoft Teams Windows 365

Microsoft AI Azure Space Mixed reality Microsoft HoloLens Microsoft Viva Quantum computing Sustainability

Education Automotive Financial services Government Healthcare Manufacturing Retail

Find a partner Become a partner Partner Network Microsoft Marketplace Software companies

Blog Microsoft Advertising Developer Center Documentation Events Licensing Microsoft Learn Microsoft Research

View Sitemap

Return to Blog Home Microsoft Research Blog

At a glance

StudentSim is a research system for evaluating and improving AI tutors before they are tested with real learners; it is not a product, is not deployed in classrooms, and is not intended to replace teachers

AI tutoring tools need feedback from students to improve, but collecting that feedback through human studies is slow, costly, and difficult to scale

StudentSim trains personalized AI student simulators from publicly available, de-identified learner records, with one simulator adapted to each student

The work evaluates simulators along two education-grounded dimensions: whether they respond like a specific student, and whether they update after tutor guidance

Across chess, second-language English writing, and mathematics, StudentSim outperforms leading baselines and provides a useful feedback signal for training an AI tutor

In research settings, teams are exploring how AI-powered tutoring systems can help explain a math problem, suggest an edit to a writing assignment, or guide a learner through a chess position. Yet the quality of teaching cannot be measured only from the tutor’s answer. A tutor needs to know what happens after the guidance reaches a student. Did the student correct the original mistake? Did the hint help them reason toward the answer, or did it give away too much? Would a different student need a different kind of support? In today’s AI tutor research, answering these questions often requires real learner studies. Those studies are valuable, but they are slow and costly to run, which makes it hard to use student feedback during fast model iteration.

StudentSim is an AI system that creates personalized simulators of individual students, modeling how they answer questions and how their answers change after receiving a tutor’s guidance. The goal is to make student feedback more available during AI tutor development. This research also introduces StudentSimEval, a benchmark and evaluation protocol spanning chess, second-language English writing, and mathematics.

Importantly, the goal is not to replace teachers or classroom learning. Instead, StudentSim helps researchers evaluate and improve AI tutoring systems during the research and development process before testing them with real learners. The research uses only publicly available learner datasets released for research use with de-identified records; no personally identifying information is used at any stage.

Figure 1: AI tutors need feedback from students to improve. Student simulators can approximate part of that feedback during research-stage training and evaluation. They provide an early-stage research signal, not a replacement for testing with real learners.

Measuring what a student simulator needs to do

A simple way to build a student simulator is to ask a large language model to role-play a learner. For example, a prompt might say, “You are an elementary-school student who struggles with math.” The model can change its tone and express uncertainty. The harder part is preserving the learner’s actual ability boundary. A model may speak in a young student’s voice while still relying on knowledge and reasoning far beyond that student’s level. For training an AI tutor, this can produce misleading feedback: the tutor may appear effective because the simulated student is too capable.

Traditional student modeling approaches face a different limitation. Knowledge tracing and behavior prediction models are grounded in real learning records and can estimate a student’s current knowledge, skills, and learning habits. Many of these models, however, were not designed to take natural-language tutor guidance as input. They can help describe what a student has done, but they do not always tell us how a student would respond to a particular explanation, hint, or Socratic question.

StudentSim defines the target through two dimensions inspired by education research. Educators often distinguish what a learner can do independently from what they can do with support. Vygotsky’s zone of proximal development captures this gap between current independent performance and performance with guidance. StudentSim turns this idea into two evaluation goals. Behavioral fidelity measures whether a simulator matches a specific student’s recorded responses on held-out problems. Guidance responsiveness measures whether the simulator updates its response after reading tutor guidance.

Figure 2: StudentSimEval evaluates whether a simulator matches a student’s independent behavior and whether it responds to guidance.

These two goals are complementary. A simulator that only matches past student behavior may be useful for prediction, but it does not show whether a teaching intervention helped. A simulator that always follows the tutor may respond fluently, but it may fail to represent a particular student’s strengths, weaknesses, and likely mistakes. AI tutor training needs both signals: the learner’s starting point and how guidance affects the...