Research Summary: Measuring cognitive presence in online discussions: automated detection and instructional insights from a MOOC context

Researchers have developed and validated an interpretable natural language processing model that automatically detects cognitive presence in online discussions across massive open online courses (MOOCs). By fine-tuning the RoBERTa architecture with theoretically derived structural cues from the Community of Inquiry framework, the model successfully classified 227,863 sentences from 4,260 learners across six Social Work MOOCs into four distinct cognitive presence phases and a fifth non-cognitive category with validated pedagogical interpretability.

Why it matters

This research demonstrates that scalable, automated natural language processing can reliably measure higher-order learning and cognitive depth in large-scale digital education environments. Establishing theoretically grounded and interpretable diagnostic models allows institutions to bridge the gap between large-scale learning analytics and practical instructional intervention.

Key insights

• The cue-augmented RoBERTa model achieved strong discourse classification performance with a macro F1 score of 0.714 and a Cohen's kappa of 0.666.
• The model evaluated 227,863 sentences generated by 4,260 learners across six Social Work MOOCs, categorising discourse into four cognitive presence phases and one non-cognitive presence category.
• Incorporating theory-derived structural cues targeting causal, synthesising, and action-oriented language enhanced the model's diagnostic capability.
• SHAP (Shapley Additive exPlanations) analyses confirmed that the internal logic of the model aligns with theory-consistent linguistic markers, supporting instructional transparency.

Source

Educational Technology Research and Development — https://link.springer.com/article/10.1007/s11423-026-10726-8