Research Summary: Behavioural telemetry is a weak substitute for in-course assessment: predicting engineering students’ examination performance from digital laboratory process data

Research investigating the predictive value of fine-grained behavioral telemetry in an engineering digital electronics laboratory found that while learning analytics aims to identify struggling students early, data derived from laboratory process traces proved to be a weak predictor of examination performance. Despite utilizing a robust methodology and a substantial dataset of student interactions, the study concluded that process data alone was not highly effective in forecasting academic outcomes.

Why it matters

This research is strategically important as it challenges assumptions regarding the efficacy of solely relying on behavioral telemetry from digital learning environments for early identification of at-risk individuals. It highlights a potential limitation in current learning analytics approaches, suggesting a need for more nuanced or comprehensive assessment strategies.

Key insights

• The study focused on predicting below-median final examination performance in first-year engineering students using digital laboratory process data.
• A dataset comprising 115 undergraduates across six simulator sessions (230,318 records) and examination outcomes for 93 students was analyzed.
• A pre-specified behavioral logistic model, using eleven features, was employed with a leakage-aware protocol and nested cross-validation.
• The findings indicate that behavioral telemetry from digital laboratory process data is a weak substitute for in-course assessment in predicting examination performance.

Source

Frontiers in Education — https://www.frontiersin.org/articles/10.3389/feduc.2026.1916958