Research Summary: Execution and Evaluation: Task Content, Labor Demand, and the Evaluator Pipeline

A research study introduces a granular methodology for assessing artificial intelligence exposure by distinguishing between execution (producing initial output) and evaluation (assessing output correctness) across 19,265 O*NET task statements. Empirical validation demonstrates an 0.81 correlation with human coders and indicates that employment growth is comparatively lower in execution-heavy white-collar occupations.

Why it matters

This framework matters because traditional AI exposure metrics treat task replacement monolithically, obscuring the specific mechanism by which automation affects labor. By decoupling execution from evaluation, organizations and policymakers can pinpoint where white-collar roles face immediate contraction versus where human oversight remains durable. This distinction clarifies whether labor risk stems from total substitution or the erosion of foundational execution pipelines.

Key insights

• Distinguishes task execution (generating first-pass outputs) from evaluation (verifying correctness) to refine conventional AI exposure metrics.
• Applied a fixed-rubric scoring methodology across 19,265 O*NET task statements, showing high consistency across model coders and dataset vintages.
• Achieved an 0.81 correlation with the mean of four blinded human evaluators in a 200-task audit, outperforming individual coders in predicting consensus.
• Preliminary findings indicate subdued employment growth in execution-heavy white-collar occupations.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2607.20807