Research Summary: Moral Entropy: Auditing Bias and Uncertainty in Moral Judgment

Recent research introduces 'Moral Entropy', a Bayesian framework designed to model and learn from disagreement in moral judgment annotations, rather than dismissing it as noise. This framework distinguishes between irreducible disagreement (aleatoric uncertainty) and uncertainty due to insufficient or noisy annotation (epistemic uncertainty). It also enables the auditing of heuristic consensus rules against a calibrated ground truth using entropy methods.

Why it matters

This development challenges conventional approaches to handling divergent opinions in data annotation, particularly in fields requiring ethical discernment. Accurately modeling and understanding different types of uncertainty in moral judgments can lead to more robust, transparent, and ethically sound algorithmic decision-making and content moderation systems.

Key insights

• Traditional computational ethics often treats annotator disagreement on moral content as 'noise', collapsing it into majority or 'any-annotator' rules.
• The proposed 'Moral Entropy' framework aims to model and learn from this inherent uncertainty in moral judgments.
• This Bayesian framework maintains a full posterior over the true label and decomposes its entropy.
• Two types of uncertainty are identified: aleatoric uncertainty (irreducible disagreement) and epistemic uncertainty (due to insufficient or noisy annotation).
• The framework allows for auditing existing heuristic consensus rules against a calibrated ground truth using methods like cross-entropy/KL, Brier score, and expected calibration error.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.21992