Research Summary: Multi-dimensional Bias in Modeling Multi-dimensional Preferences: Evaluating the Ability of Synthetic Agents to Replace Human Participants in Conjoint Experiments

Research is exploring the efficacy of synthetic agents, specifically Large Language Models (LLMs), to replace human participants in conjoint experiments. This study replicates existing conjoint studies, comparing synthetic agent-generated results with original human data across representational correspondence, inferential correspondence, and procedural stability to assess their ability to reproduce multi-dimensional human preference patterns. The core aim is to determine if LLMs can enhance robustness or reduce data collection costs in survey experiments.

Why it matters

The potential for synthetic agents to replace human participants in preference modeling could significantly alter data collection methodologies, impacting the speed, scale, and cost of market research, policy preference assessment, and strategic planning. This development offers opportunities for more agile and cost-effective insights, while also introducing new considerations regarding data validity and representativeness.

Key insights

• The research investigates the utility of LLMs as substitutes for human participants in conjoint experiments, a method gaining traction in political science.
• The study focuses on whether synthetic agents can accurately replicate the complex, multi-dimensional human preference patterns that conjoint experiments are designed to measure.
• Existing conjoint studies were replicated, and the results from synthetic agents were compared against the original human data.
• Comparisons were conducted along three specific dimensions: representational correspondence, inferential correspondence, and procedural stability.
• The analysis evaluates both the alignment of choice distributions and the statistical and substantive similarity of estimations derived from synthetic agents versus human data.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.04243