A Brief AI Literacy Intervention Does Not Significantly Reduce Over-Reliance and Increases Under-Reliance on ChatGPT: A Randomized Study

A recent study investigated the impact of a brief AI literacy intervention on high school students' reliance on Large Language Model (LLM) recommendations. The findings indicate that while widespread over-reliance on incorrect LLM advice was observed (52.1%), the intervention did not significantly reduce this over-reliance. Instead, the intervention led to an increase in under-reliance on LLM recommendations, suggesting a complex and potentially counter-intuitive effect of basic AI literacy efforts.

These findings are crucial for organizations deploying or considering the integration of AI tools, particularly Large Language Models, as they underscore the challenges in effectively training users to interact optimally with these technologies. An understanding of how literacy interventions influence user behavior is vital for developing effective deployment strategies and maximizing AI benefits while mitigating associated risks.

High school students exhibited widespread over-reliance on Large Language Model (LLM) recommendations, adopting incorrect advice in 52.1% of trials.

Forward consideration, not a verified fact.