Research Summary: Fine-Tuning Large Language Models for Codebook-Guided Coding of Students' Mathematics Metaphor Responses

Research indicates that fine-tuning Large Language Models (LLMs) using Low-Rank Adaptation (LoRA) can significantly enhance their ability to automatically code student-generated mathematics metaphors. This approach addresses the scalability challenges and labor intensity associated with traditional expert human assessment of complex qualitative data, such as students' affective orientations and thematic responses.

Why it matters

This development represents a significant step towards automating qualitative data analysis, particularly in educational or social research contexts. It offers a pathway to more efficiently process large volumes of nuanced textual data, thereby enabling broader and more rapid insights into complex human perceptions and attitudes without proportionally increasing human resource allocation.

Key insights

• Expert human assessment of student metaphors is labor-intensive and difficult to scale due to the semantic complexity of the responses.
• The study explored the use of Low-Rank Adaptation (LoRA) for supervised fine-tuning of LLMs to improve their performance in codebook-guided coding tasks.
• A human-coded corpus of 2,265 Grade 6-8 student responses to mathematics metaphor prompts was used for evaluation.
• LLM performance was evaluated on two specific coding tasks: valence-intensity coding for affective orientations and thematic coding of metaphor content.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.10276