Scientific Review Quality Assessment on constructed Rebuttal ICLR and NeurIPS reviews post-2023 (test)
9.18Rigor (C)RebuttalAgent
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RebuttalAgentFramework=RebuttalAgent2026.01 | 9.18 | 8.82 | 9.99 | 9.14 | 8.84 | 9.96 | 9.09 | 8.54 | 9.94 | 9.34 | 9.06 | 9.98 | 9.34 | |
| o3Model=o32026.01 | 8.92 | 8.86 | 9.71 | 8.8 | 8.71 | 9.44 | 8.59 | 8.33 | 9.24 | 9.26 | 8.98 | 9.67 | 9.09 | |
| DeepSeek-R1Model=DeepSeek-R12026.01 | 8.62 | 8.1 | 9.15 | 8.56 | 8.14 | 8.94 | 8.36 | 7.86 | 8.69 | 9.62 | 9.06 | 8.72 | 9.18 | |
| Self-RefinedPrompting Strategy=Self-Refined2026.01 | 8.46 | 7.96 | 8.92 | 8.43 | 7.94 | 8.81 | 8.2 | 7.61 | 8.54 | 9.62 | 8.93 | 8.58 | 8.99 | |
| GPT-4.1Model=GPT-4.12026.01 | 8.27 | 7.73 | 8.67 | 8.22 | 7.69 | 8.51 | 8.06 | 7.36 | 8.33 | 9.4 | 8.74 | 8.38 | 8.74 | |
| Strategy-PromptPrompting Strategy=Strategy-Prompt2026.01 | 8.21 | 7.33 | 8.22 | 8.23 | 7.57 | 8.33 | 8.11 | 7.31 | 7.91 | 9.47 | 8.7 | 8.22 | 8.5 | |
| Qwen3-8BModel=Qwen3-8B2026.01 | 7.91 | 7.18 | 8.01 | 7.79 | 6.99 | 7.66 | 7.69 | 6.75 | 7.35 | 9.07 | 8.39 | 7.87 | 8 | |
| RebuttalFTTraining Protocol=Fine-tuned2026.01 | 6.8 | 5.89 | 6.59 | 6.61 | 5.7 | 6.28 | 6.5 | 5.41 | 5.86 | 6.39 | 5.64 | 6.52 | 6.23 |