# Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation Source: [Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation]() Collection index: [llms.txt]() Data updated: 2026-03-05T05:00:11.275Z Authors: Hongliu Cao, Ilias Driouich, Eoin Thomas Published: 2026-03-03T15:47:41.000Z ## Abstract Large Language Model (LLM)-based agents are increasingly adopted in high-stakes settings, but current benchmarks evaluate mainly whether a task was completed, not how. We introduce Procedure-Aware Evaluation (PAE), a framework that formalizes agent procedures as structured observations and exposes consistency relationships between what agents observe, communicate, and execute. PAE evaluates agents along complementary axes (Utility, Efficiency, Interaction Quality, Procedural Integrity) and applies multi-dimensional gating that categorically disqualifies corrupt outcomes. Evaluating state-of-the-art LLM agents on tau-bench yields findings at the axis, compliance, and benchmark levels. At the axis level, the dimensions capture non-redundant failure modes: utility masks reliability gaps, speed does not imply precision, and conciseness does not predict intent adherence. At the procedural compliance level, 27-78% of benchmark reported successes are corrupt successes concealing violations across interaction and integrity. Furthermore, gating substantially collapses Pass^4 rate and affects model rankings. The analysis of corrupt success cases reveals distinctive per-model failure signatures: GPT-5 spreads errors across policy, execution, and intent dimensions; Kimi-K2-Thinking concentrates 78% of violations in policy faithfulness and compliance; and Mistral-Large-3 is dominated by faithfulness failures. At the benchmark level, our analysis exposes structural flaws in the benchmark design, including task scope gaps, contradictory reward signals, and simulator artifacts that produce accidental successes. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [LLM Agent Evaluation on tau-bench Retail](): LLM Agent Evaluation; Tau-bench retail; Pass@1: not recorded. - [LLM Agent Evaluation on tau-bench Airline](): LLM Agent Evaluation; Tau-bench airline; Pass@4: not recorded. ## Original sources and implementation [Original paper]() [PDF]()