Research Summary: Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Models

Research proposes a novel reinforcement learning (RL) fine-tuning framework to enhance the clinical reasoning capabilities of Electronic Health Record (EHR) foundation models. These models, while powerful for prediction, are limited by existing next-token prediction methods on incomplete data. The framework treats EHR models as generative policies over patient trajectories, reformulating clinical predictions as event-conditioned, time-windowed reasoning tasks and utilizing time-aware, rollout-sensitive rewards to overcome data limitations and improve reasoning over patient pathways.

Why it matters

This development is strategically important as it addresses a core limitation in leveraging advanced AI models for critical decision-making in healthcare or related domains. Enhancing clinical reasoning in EHR foundation models could lead to more accurate and nuanced predictions, improving patient outcomes and operational efficiencies by better understanding complex, longitudinal data.

Key insights

• Existing EHR foundation models exhibit strong performance in diverse clinical prediction tasks but are constrained in their clinical reasoning.
• Limitations stem from reliance on next-token prediction and incomplete longitudinal patient data.
• A reinforcement learning (RL) fine-tuning framework is proposed to address these constraints.
• The framework re-conceptualizes EHR foundation models as generative policies operating over patient trajectories.
• Clinical prediction problems (e.g., hospital readmission) are reframed as event-conditioned, time-windowed reasoning tasks.
• Time-aware, rollout-sensitive rewards are designed to handle finite rollout lengths and temporally inconclusive outcomes in RL fine-tuning.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.12277