But there was one main difference. A portion of the students were randomly assigned a sequence of practice problems, progressing from easy to difficult. The other half received a personalized follow-up with an AI instructor who continued to adjust the difficulty of each problem based on how the student performed and interacted with the chatbot.
The idea is based on what educators call “the zone of proximal development.” If the problems are too easy, students get bored. When they are too difficult, students get frustrated. The goal is to keep students in the sweet spot: challenged, but not frustrated.
The researchers found that students in the personalized group did better on the final exam than students in the problem-corrected group. The difference was seen as the equivalent of 6 to 9 months of additional education, an eye-catching claim for an online after-school course that lasted only five months. The founder of the AI tutorial, Angel Chung, a doctoral student at the Wharton School, admitted that his conversion of mathematical units “is not a perfect approximation.” (A draft paper about the experiment was posted online in March 2026, but has not yet been published in a peer-reviewed journal.)
Still, this is early evidence that small tweaks — in this case, balancing the difficulty of practice problems for the student — can make a difference.
Chung said ChatGPT’s answers can feel more personal because they directly answer a reader’s unique questions. But that level of personalization is not enough. “Students usually don’t know what they don’t know,” said Chung. “The student does not have the ability to ask the right questions to get the best teaching.”
To address this, Chung’s team combined a large language model with a unique machine learning algorithm that analyzes how students interact with the online course platform — how they answer practice questions, how often they revise or edit their code, and the quality of their conversations with the chatbot — and uses that information to decide which problem to assign next.
How different students interact with the chatbot instructor
In other words, personalization is not just about combining meanings. It’s about integrating the learning process itself.
That idea is not new.
Long before productive AI tools like ChatGPT were invented, educational researchers were building “intelligent tutoring programs” that tried to do the same thing: estimate what a student knows and deliver the next relevant problem. These previous systems could not generate natural conversations, but they could provide advice and quick feedback. Extensive research has found that well-designed translations help students learn more.
Their Achilles heel was marriage. Many students did not want to use them.
Today’s AI tools can help solve that problem. Readers may feel more interested in a chatbot that interacts with them in an almost human way.
In a University of Pennsylvania study, students in the personalized group spent more time practicing, about three minutes more per problem, adding about an hour per module to the Python course, compared to half as much time (half an hour or less) for comparison students. The researchers think that these students did better because they were more engaged in their practice work.
Students’ prior knowledge of the subject influenced how well the personal sequence worked. Students who were new to Python scored better than those who already had Python experience, who performed well on a structured sequence of practice problems. Students in low-performing secondary schools also appeared to benefit the most.
How student background affected results
All Taiwanese students in this study volunteered to take an elective computer programming course that would strengthen their college applications. Many were motivated, had highly educated parents, and many had prior coding experience.
It is not clear whether the chatbot can work with less motivated students who are behind in school and many of whom need extra help.
One possible solution: combining the new with the old.
Ken Koedinger, a professor at Carnegie Mellon University and a pioneer of intelligent tutoring systems, is trying to use new AI models to alert remote human teachers who can motivate struggling students from far away. “We’re having more success,” Koedinger said.
People aren’t out of time – yet.
This story is about AI tutors was produced by The Hechinger reporta non-profit, independent media organization covering education. Sign up Evidence Points and so on Hechinger newsletters.
