Auditable Emergency Triage for Maternal and Newborn Care in India

Noora Health's WhatsApp-based service for maternal and newborn care in India handles over 50,000 medical queries monthly, with emergency triage being a critical, time-sensitive function. An AI system using a Large Language Model (LLM) was developed to classify emergency messages and provide rationales. However, this system proved opaque, hindering error analysis and making prompt changes and re-evaluation operationally challenging and costly due to its scale, contrasting with clinicians' traditional decision-tree approach.

This research highlights the significant challenges and strategic implications of integrating advanced AI, specifically LLMs, into critical operational processes like emergency triage in healthcare settings. It underscores the tension between technological advancement, operational efficiency, and the imperative for transparency and auditability, particularly in high-stakes domains such as maternal and newborn care. Addressing these issues is crucial for maintaining trust, ensuring patient safety, and scaling digital health services responsibly.

Noora Health manages a high volume of medical queries (50,000+ per month) for maternal and newborn care via WhatsApp.

Forward consideration, not a verified fact.