Research Summary: Your Programming Students' Cognition with ChatGPT: Higher Performance, Lower Retention, and Reduced Ownership

A recent research study from arXiv investigates the impact of generative AI, specifically ChatGPT-4.5, on undergraduate computer science students' programming performance and learning retention. The study found that students using ChatGPT achieved significantly higher coding performance but exhibited lower immediate and delayed recall of the material compared to a control group using conventional web search. This suggests a trade-off between immediate task success and long-term knowledge retention when leveraging generative AI in technical domains.

Why it matters

This research highlights a critical paradox in the integration of advanced AI tools: while they can significantly enhance immediate operational performance, they may inadvertently hinder knowledge acquisition and retention. For organizations, this necessitates a strategic re-evaluation of how AI is deployed in training, development, and daily workflows, balancing productivity gains with the imperative of fostering deep human expertise and long-term skill development.

Key insights

• Students using ChatGPT-4.5 achieved substantially higher coding scores (89%) compared to those using conventional web search (69%).
• ChatGPT-assisted students demonstrated lower recall scores immediately after tasks and 48 hours later.
• The study involved 55 undergraduate computer science students completing introductory C programming tasks.
• Measurements included task performance, self-reported mental effort and difficulty, pupillary responses, heart rate variability, ownership, and cued recall.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.21194