Research Summary: When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generation

A large-scale empirical study investigates how user demographic information, conveyed through personas, influences the technical quality of code generated by Large Language Models (LLMs). The research, which compared proprietary (Gemini 2.5 Pro) and open-weight (GPT-OSS-120B) models across 35,000+ generated programs, found that demographic cues can affect code generation. This suggests potential biases in AI programming assistants.

Why it matters

The potential for Large Language Models to generate varying code quality based on user demographics introduces significant ethical and operational risks. This research highlights the necessity for developers and users of AI programming assistants to understand and mitigate algorithmic bias, ensuring equitable and consistent technical outcomes irrespective of user identity.

Key insights

• The study focused on persona-induced bias in LLM-based code generation.
• It utilized 18 demographic personas covering nationality, gender, and experience level, compared against a neutral baseline.
• Two LLMs, Gemini 2.5 Pro (proprietary) and GPT-OSS-120B (open-weight), were evaluated.
• Over 35,000 programs were generated and analyzed.
• Analysis included demographic marker leakage, functional correctness, maintainability, code style, and security.
• Preliminary results indicate that demographic cues produce effects on generated code quality, though the specific nature of these effects is not detailed in the provided abstract.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.22102