Research Summary: Humanize: Judgement Engineering for Agentic Coding

A research study introduces Humanize, a multi-agent orchestration framework designed to address the deficiency of agentic code generators acting as judges of their own work. The approach relies on judgement engineering, establishing strict boundaries between planning, execution, and evaluation through human approval, multi-vendor agent segregation, and 72 deterministic mechanical gates.

Why it matters

As generative software engineering scales, single-agent self-validation presents acute reliability and quality risks in technical operations. Implementing multi-vendor separation of duties and deterministic checkpoints provides an architectural foundation for dependable agentic automation. This structural shift moves the operational focus from pure generation capacity to verifiable execution governance.

Key insights

• Agentic code generation is cost-effective but struggles with reliable completion due to agents being poor evaluators of their own output.
• The Humanize framework implements judgement engineering to enforce explicit decisions across planning, implementation, review, and learning phases.
• Workflows require human sign-off on plan contracts, assign implementation to a builder agent, and delegate completion decisions to an independent reviewer agent from a separate vendor.
• Routing and gatekeeping are executed via deterministic hooks rather than AI models, enforcing 72 mechanical gates.
• The system mathematically models workflow progression as a Markov chain over repository states by alternating sampling between two distinct models.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2610.08900