Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AI

This research investigates the application of trustworthiness-enhancing techniques in Large Language Models (LLMs) to support the development of ethically aligned AI software. Recognizing that AI systems, including LLMs, widely impact society but also present challenges like misinformation and bias, the study aims to provide practical guidance on AI ethics. It proposes and evaluates a multi-agent prototype LLM-based system incorporating distinct roles, structured communication, and multiple rounds of debate to address real-world AI ethics issues.

Why it matters

Addressing the trustworthiness of AI systems, particularly Large Language Models, is strategically vital as their widespread adoption creates significant societal impact. Organizations must navigate the inherent risks of misinformation, bias, and misuse to maintain public trust and ensure responsible innovation. Developing methodologies for ethically aligned AI is critical for long-term sustainability and regulatory compliance across all sectors.

Key insights

• AI-based systems, including LLMs, significantly impact diverse tasks but are susceptible to issues such as misinformation, bias, and misuse.
• The domain of AI ethics is critical, constantly evolving with new technologies and concerns, yet practical and objective guidance remains a subject of debate.
• Trustworthiness-enhancing techniques identified for LLMs include the use of multi-agents, assigning distinct roles, implementing structured communication, and facilitating multiple rounds of debate.
• The study employs a single exploratory cycle of Design Science Research (DSR) to investigate how these techniques can enhance trustworthiness.
• A multi-agent prototype LLM-MAS (Multi-Agent System) has been designed where agents are tasked with addressing real-world AI ethics issues.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2411.08881