The mathematical operation (2/3 + 1/4) × 3/5 appeared twice in the same experiment for 354 American adults recruited from the internet. In the first instance, more than half of the participants had a chat window open on the side of their screen. Introduced as ChatGPT and powered by GPT-5, this bot knew both the question and its solution in advance, meaning participants only needed to type "answer?". A few minutes later, the exact same operation reappeared as the first question of a test. This time, there was no chat window, and no one had been warned beforehand that it would disappear.

Once the window vanished, those who had worked with the bot solved fewer questions and skipped more of them. The research team tested this outcome through two additional experiments, eventually reaching a total of 1,222 participants.

What is in this article?
  • How quitting was measured in the experiment and why removing the bot was done without warning.
  • Whether the decline observed in fractions repeated itself in reading comprehension questions.
  • Whether everyone working with the bot finished in the same place, or if the way you prompted the question changed the outcome.
  • What a fifteen-minute experiment can and cannot tell us about months of real-world usage.

How was the experiment designed?

The lead author of the study is Grace Liu from Carnegie Mellon University. She is joined by Brian Christian and Tsvetomira Dumbalska from Oxford, Michiel Bakker from MIT, and Rachit Dubey from UCLA. Christian is also a researcher at the Center for Human-Compatible AI in Berkeley and is well known for his 2020 book on the alignment problem. The preliminary version of the paper was uploaded to arXiv on April 6, 2026, and its final peer-reviewed version was presented at the Conference on Language Modeling (COLM) in the first week of October 2026.

Participants were recruited via the online research platform Prolific. The first experiment lasted 13 minutes, and each person was paid $2.60. It was explicitly explained to everyone that payment was not dependent on the number of correct answers and that incorrect answers carried no penalty. For anyone who answered incorrectly, the correct solution was immediately displayed on the screen, meaning both groups could learn from their mistakes. A "skip" button sat beneath every question. In a game with no penalties, pressing this button is considered a decision born not of inability, but of a reluctance to put in the effort. The research team measured persistence using this very button.

The bot group was presented with 12 increasingly difficult questions; after the bot was turned off, both groups were tested on the exact same 3 questions.

How did all three experiments point in the same direction?

In the first experiment, among the 307 people included in the analysis, those who worked with the bot correctly solved 57 percent of the final three questions, compared to 73 percent for those who never saw the bot. The skip rate was 20 percent versus 11 percent.

There was a loophole in this initial result. The research team had removed participants who could not handle basic fraction operations from the analysis, but within the bot group, those same individuals could simply extract the correct answer from the bot and bypass elimination. For this reason, the second experiment was designed with 667 people and a pre-test: everyone first solved 3 easy questions without assistance, and elimination was handled accordingly. A side panel displaying the solutions to the pre-test was also placed on the control group's screen, which was then removed when the test began. Thus, participants in both groups experienced something disappearing from their screens at the moment of testing. The gap narrowed but did not vanish: out of 585 people, correct resolution stood at 71 percent versus 77 percent. Skipping was observed at 10 percent versus 7 percent, a difference that was not statistically significant.

The third experiment moved away from fractions and shifted to reading comprehension questions from the American SAT college entrance exam. After a 5-question study phase, the bot was turned off. Answers submitted in under 5 seconds, deemed too brief to have allowed reading the text, were also counted as skipped. Among 168 people, correct resolution was 76 percent versus 89 percent, while skipping was 8 percent versus 1 percent.

Notes and Sources

  1. Liu, G., Christian, B., Dumbalska, T., Bakker, M. A., and Dubey, R. (2026). "AI Assistance Reduces Persistence and Hurts Independent Performance". Conference on Language Modeling (COLM 2026); initial version April 6, 2026. arXiv:2604.04721; project page ↩
  2. Pohl, J. (2026, October 9). "Study explores how AI use affects performance on math and reasoning tasks". Berkeley News, University of California, Berkeley. news.berkeley.edu ↩
  3. Shen, J. H., and Tamkin, A. (2026). "How AI Impacts Skill Formation". Anthropic. arXiv:2601.20245; anthropic.com ↩
  4. Budzyń, K., Romańczyk, M., Kitala, D., et al. (2025). "Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study". The Lancet Gastroenterology & Hepatology, 10(10), 896–903. doi:10.1016/S2468-1253(25)00133-5 ↩
  5. OpenAI (2025, July 29). "Introducing study mode". openai.com; Zeff, M. (2025, July 29). "OpenAI launches Study Mode in ChatGPT". TechCrunch. techcrunch.com ↩