Research Summary: The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment

Research from arXiv explores how implicit and explicit demographic signals influence the performance of Large Language Models (LLMs) when used for student assessment. The study investigates the potential for both necessary demographic considerations, such as improving readability, and discriminatory outcomes, like lower scores for specific socioeconomic backgrounds. Controlled prompts were used to test these effects across Automated Essay Scoring, Formative Feedback, and Metalinguistic Question Answering tasks.

Why it matters

The widespread adoption of Large Language Models in sensitive domains like student assessment necessitates a thorough understanding of their potential biases and fairness implications. Ensuring equitable and accurate assessment outcomes is critical for maintaining trust in educational technologies and preventing systemic discrimination, which could have long-term societal impacts.

Key insights

• Large Language Models are increasingly common in student assessment.
• The impact of student demographics on LLM-based assessment is not well understood.
• Considering student demographics might be necessary for certain applications, such as improving content readability.
• There is a significant risk of discrimination if demographics lead to biased assessment outcomes, e.g., lower scores for certain socioeconomic groups.
• The study tests both explicit demographic mentions and implicit demographic signals derived from conversation history.
• Evaluation covers three key assessment tasks: Automated Essay Scoring, Formative Feedback, and Metalinguistic Question Answering.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.16993