LLM Persuasion Is in the Eye of the Evaluation

Research indicates that large language models (LLMs) can match or exceed human experts in persuasion, posing dual-use implications across beneficial sectors like education and health as well as risks involving manipulation and misinformation. However, evaluation of these capabilities remains fragmented across inconsistent definitions and narrow contexts, prompting the development of automated, scalable assessment methods to evaluate high-risk persuasion.

This issue is critical because unstandardized persuasion metrics leave decision-makers and regulators with incomplete visibility into the true influence and manipulation potential of deployed models. Without reliable and scalable evaluations, institutions risk deploying persuasive AI systems in sensitive contexts without understanding their capacity for distortion or harm. Establishing rigorous benchmarks is necessary to govern dual-use generative systems before widespread societal exposure occurs.

Large language models match or surpass human experts in persuasive capability.

Forward consideration, not a verified fact.