# ReviewGuard: Enhancing Deficient Peer Review Detection via LLM-Driven Data Augmentation

Source: [ReviewGuard: Enhancing Deficient Peer Review Detection via LLM-Driven Data Augmentation](<https://www.sota2.com/research/paper/68fa7c541e209131cea8d982>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-11-25T05:03:07.149Z

Authors: Haoxuan Zhang, Ruochi Li, Sarthak Shrestha, Shree Harshini Mamidala, Revanth Putta, Arka Krishan Aggarwal, Ting Xiao, Junhua Ding, Haihua Chen

Published: 2025-10-18T15:45:26.000Z

## Abstract

Peer review serves as the gatekeeper of science, yet the surge in submissions and widespread adoption of large language models (LLMs) in scholarly evaluation present unprecedented challenges. While recent work has focused on using LLMs to improve review efficiency, unchecked deficient reviews from both human experts and AI systems threaten to systematically undermine academic integrity. To address this issue, we introduce ReviewGuard, an automated system for detecting and categorizing deficient reviews through a four-stage LLM-driven framework: data collection from ICLR and NeurIPS on OpenReview, GPT-4.1 annotation with human validation, synthetic data augmentation yielding 6,634 papers with 24,657 real and 46,438 synthetic reviews, and fine-tuning of encoder-based models and open-source LLMs. Feature analysis reveals that deficient reviews exhibit lower rating scores, higher self-reported confidence, reduced structural complexity, and more negative sentiment than sufficient reviews. AI-generated text detection shows dramatic increases in AI-authored reviews since ChatGPT's emergence. Mixed training with synthetic and real data substantially improves detection performance - for example, Qwen 3-8B achieves recall of 0.6653 and F1 of 0.7073, up from 0.5499 and 0.5606 respectively. This study presents the first LLM-driven system for detecting deficient peer reviews, providing evidence to inform AI governance in peer review. Code, prompts, and data are available at https://github.com/haoxuan-unt2024/ReviewGuard

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Binary deficiency detection on gold-labeled ICLR (test)](<https://www.sota2.com/research/sota/binary-deficiency-detection-on-gold-labeled-iclr-test.md>): Binary deficiency detection; gold-labeled ICLR (test); Accuracy: 66.

- [Persona Discrimination on NeurIPS Cross-conference](<https://www.sota2.com/research/sota/persona-discrimination-on-neurips-cross-conference.md>): Persona Discrimination; NeurIPS Cross-conference; Persona Separability (Δ): 0.31.

- [Persona Discrimination on ICML Cross-conference](<https://www.sota2.com/research/sota/persona-discrimination-on-icml-cross-conference.md>): Persona Discrimination; ICML Cross-conference; Persona Separability (Δ): 0.286.

- [Persona Discrimination on MiniMax Cross-generator M2.5](<https://www.sota2.com/research/sota/persona-discrimination-on-minimax-cross-generator-m2-5.md>): Persona Discrimination; MiniMax Cross-generator M2.5; Persona Separability (Δ): 0.18.

- [Persona Discrimination on Llama Cross-generator 3.3-70B](<https://www.sota2.com/research/sota/persona-discrimination-on-llama-cross-generator-3-3-70b.md>): Persona Discrimination; Llama Cross-generator 3.3-70B; Persona Separability (Δ): 0.071.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.16549>)

[PDF](<https://arxiv.org/pdf/2510.16549.pdf>)