Research Summary: Do Social Patterns Hold in Synthetic Data? Analyzing Cyberbullying Dynamics in LLM-Generated and Authentic Dialogues

Research from arXiv investigates the social realism of Large Language Model (LLM)-generated synthetic cyberbullying conversations compared to authentic dialogues. The study evaluates these synthetic conversations across interactional structure (turn-taking, power dynamics, repair behavior) and linguistic/stylistic realism (pronoun usage, humor) to determine if they faithfully reproduce complex social dynamics beyond supporting downstream task performance.

Why it matters

The fidelity of synthetic data is critical for any application, particularly in sensitive domains like social dynamics or cyberbullying. Reliance on unrealistic synthetic data can lead to flawed insights, ineffective interventions, or misallocated resources, thereby impacting strategic decision-making and operational effectiveness.

Key insights

• LLMs are increasingly used to generate synthetic cyberbullying data for augmentation and benchmarking.
• A key concern is whether this synthetic data accurately reproduces the social dynamics of authentic interactions.
• The research proposes a framework to evaluate the social realism of LLM-generated cyberbullying conversations.
• GPT, Grok, and LLaMA models were used to generate synthetic dialogues for comparison.
• Evaluation criteria include interactional structure (turn-taking, power dynamics, repair behavior) and linguistic/stylistic realism (pronoun usage, humor).

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.17549