Reference-Distribution Dependence in LLM-Based Synthetic Persona Data: Diagnosis and Post Hoc Adjustment of Demographic Distributions

Research has diagnosed the fidelity of demographic distributions in LLM-based synthetic persona data, specifically identifying that the primary source of error in matching external distributions is attributable to the choice of the reference distribution, rather than the data generator itself. A case study comparing 1,000,000 synthetic records against Korean official statistics revealed a bias bound of 1.81 percentage points across sex, age group, and province variables.

Why it matters

The fidelity of synthetic data is critical for robust analytical applications, strategic planning, and policy development, particularly in areas requiring accurate demographic representation. Understanding that reference distribution choice is a major error source highlights the need for careful selection and validation of ground truth, which can significantly impact the reliability and trustworthiness of insights derived from synthetic datasets.

Key insights

• The accuracy of demographic distributions in LLM-based synthetic persona data is significantly influenced by the chosen external reference distribution.
• The generator's contribution to distribution errors is less pronounced than that of the reference data selection.
• A comparison of Nemotron-Personas-Korea (NPK) data with April 2026 Korean resident-registration statistics showed a bias bound of 1.81 percentage points.
• This bias bound, using total variation distance (TVD), applies to the joint distribution of sex, age group, and province, and is comparable to the margin of error of a survey of roughly 2,900 individuals.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.28668