Research Summary: Do Large Language Models Know Colombian Law? A Reliability Benchmark for the Colombian Legal System

A new expert-validated benchmark designed to assess the reliability of Large Language Models (LLMs) within the Colombian legal system has revealed significant variability in their performance. While some models achieve high accuracy on closed questions, their factual consistency and hallucination rates in free-text legal answers remain problematic. This indicates that current LLMs are not yet consistently reliable for complex legal tasks in specific national jurisdictions outside of commonly-studied systems.

Why it matters

The expanding use of LLMs in legal practice, education, and research necessitates robust evaluations of their reliability, especially in diverse national legal frameworks. Demonstrating their capability and mitigating risks like hallucination are critical for their adoption and for maintaining trust in AI-assisted legal processes. This research provides a foundational understanding of LLM limitations in a specific non-US legal context, informing development and deployment strategies.

Key insights

• A benchmark comprising 1,042 items across ten areas of Colombian law and three question formats (closed multiple-choice, semi-open, open-ended IRAC) has been developed through a human-in-the-loop, expert-reviewed pipeline.
• Fifteen proprietary and open-weight LLMs were evaluated using format-appropriate metrics.
• Accuracy on closed questions varied significantly among models, ranging from 0.905 (Gemini 3.1 Pro) to 0.577 for the lowest performing model.
• The study highlights concerns regarding factual consistency and hallucination in free-text legal answers provided by LLMs.
• The research indicates a general lack of documentation regarding LLM reliability in national legal systems outside the United States.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2610.03639