AusClinBench: a deterministically scored benchmark of large language models against current Australian clinical policy

Researchers introduced AusClinBench, a deterministic benchmark evaluating 28 large language models against current Australian clinical policy across 142 verifiable questions. The study identified wide performance variations in blind tests, ranging from 51.5% to 92.7% accuracy across regulatory, billing, and clinical domains, underscoring risks given that clinical software frequently embeds undisclosed models.

This development matters because generative AI tools are already operating within clinical workflows without jurisdiction-specific validation or transparent model disclosure. Without localised benchmarking, healthcare providers risk regulatory non-compliance, financial errors in billing, and compromised patient safety. Establishing deterministic, jurisdiction-specific evaluations provides an objective baseline for clinical AI governance and procurement standards.

AI models are actively deployed in Australian clinical software to draft notes and support clinical decisions, frequently without disclosing the underlying model to practitioners.

Forward consideration, not a verified fact.