Research Summary: Cite or Decline: A Strict Course-Grounded Chatbot for STEM Lecture Videos

A retrieval-augmented chatbot, developed as part of the VideoPoints platform, has been successfully deployed for a semester to support STEM lecture videos. This chatbot exclusively uses course-specific lecture materials to answer student queries, providing timestamped citations and demonstrating a high rate of declining to answer when no relevant evidence is found, thus maintaining content integrity and relevance.

Why it matters

This development is significant as it demonstrates a practical application of AI in educational resource delivery, addressing the challenge of content veracity and relevance. For organizations relying on large volumes of domain-specific information, it highlights a model for building highly accurate and auditable AI-powered knowledge retrieval systems, mitigating risks associated with untrustworthy or out-of-domain information.

Key insights

• The VideoPoints platform features a retrieval-augmented chatbot for STEM lecture videos.
• The chatbot's knowledge base is strictly limited to the active course's lecture materials.
• It utilizes chapter summaries to enhance transcript ranking for retrieval.
• Answers are provided with clickable, timestamped citations for verification.
• Students primarily used the chatbot for quick lookups and exam review.
• Analysis of 833 messages showed 70.5% included citations and no answers originated from outside the course materials.
• The chatbot generally declined to answer when course lecture evidence was absent, indicating a robust guardrail against hallucination or irrelevant information.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.01846