# TADDLE: A Tool-Augmented Agent for Detecting Deficient LLM-Generated Peer Reviews

Source: [TADDLE: A Tool-Augmented Agent for Detecting Deficient LLM-Generated Peer Reviews](<https://www.sota2.com/research/paper/6a167ac4dbd7150506f8016f>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-28T05:00:13.160Z

Authors: Hanqi Duan, Xiang Li

Published: 2026-05-26T12:07:44.000Z

## Abstract

LLM-generated peer reviews are increasingly common at major venues, yet their deficiencies are hard to detect because they are uniformly fluent and well-structured. Existing work either classifies authorship without judging quality, or scores quality with features designed for human-written reviews; no prior system detects deficiencies in LLM-generated reviews at the level of individual defect types. To bridge the gap, we introduce TADDLE, a Tool-Augmented Agent for Detecting Deficient LLM-Generated Peer Reviews, together with the first expert-annotated benchmark for this task. Our benchmark comprises 1,800 reviews on 50 ICLR 2025 papers, multi-label-annotated by 18 domain experts against a taxonomy of six defect categories (plus a non-deficient label). TADDLE decomposes detection into four specialized analysis tools -- Verify, Correct, Complete, and Transform -- orchestrated by an agent; an integrator synthesizes their outputs into binary and multi-label classifications via two-stage semi-supervised learning. Extensive experiments show that TADDLE performs strongly on both binary detection and the multi-label classification task. We release the benchmark and code at https://github.com/AquariusAQ/TADDLE.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Binary deficiency detection on gold-labeled ICLR (test)](<https://www.sota2.com/research/sota/binary-deficiency-detection-on-gold-labeled-iclr-test.md>): Binary deficiency detection; gold-labeled ICLR (test); Accuracy: 86.

- [Persona Discrimination on ICML Cross-conference](<https://www.sota2.com/research/sota/persona-discrimination-on-icml-cross-conference.md>): Persona Discrimination; ICML Cross-conference; Persona Separability (Δ): 0.453.

- [Persona Discrimination on NeurIPS Cross-conference](<https://www.sota2.com/research/sota/persona-discrimination-on-neurips-cross-conference.md>): Persona Discrimination; NeurIPS Cross-conference; Persona Separability (Δ): 0.418.

- [Persona Discrimination on Llama Cross-generator 3.3-70B](<https://www.sota2.com/research/sota/persona-discrimination-on-llama-cross-generator-3-3-70b.md>): Persona Discrimination; Llama Cross-generator 3.3-70B; Persona Separability (Δ): 0.427.

- [Persona Discrimination on MiniMax Cross-generator M2.5](<https://www.sota2.com/research/sota/persona-discrimination-on-minimax-cross-generator-m2-5.md>): Persona Discrimination; MiniMax Cross-generator M2.5; Persona Separability (Δ): 0.281.

- [Fine-grained multi-label classification on ICLR gold-labeled (test)](<https://www.sota2.com/research/sota/fine-grained-multi-label-classification-on-iclr-gold-labeled-test.md>): Fine-grained multi-label classification; ICLR gold-labeled (test); Jaccard Similarity: 74.24.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.26911>)

[PDF](<https://arxiv.org/pdf/2605.26911.pdf>)