LLM Judging Agreement on Reasoning Agreement Benchmark 2,500 samples
100Parsing Success RateGPT-4o
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-4o2025.07 | 100 | — | |
| Master-RM-32BModel Scale=32B2025.07 | 100 | 0.89 | |
| Master-RM-7BModel Scale=7B2025.07 | 100 | 0.91 | |
| Multi-sub RM2025.07 | 100 | 0.91 | |
| Omni-Judge2025.07 | 100 | 0.81 | |
| Qwen2.5-72B-InstructModel Scale=72B2025.07 | 100 | 0.89 | |
| Qwen2.5-32B-InstructModel Scale=32B2025.07 | 100 | 0.9 | |
| Qwen2.5-14B-InstructModel Scale=14B2025.07 | 100 | 0.92 | |
| Qwen2.5-7B-InstructModel Scale=7B2025.07 | 100 | 0.85 | |
| Qwen2.5-3B-InstructModel Scale=3B2025.07 | 100 | 0.81 | |
| Qwen2.5-1.5B-InstructModel Scale=1.5B2025.07 | 100 | 0.83 | |
| Qwen2.5-0.5B-InstructModel Scale=0.5B2025.07 | 100 | 0.1 | |
| LLaMA3-70B-InstructModel Scale=70B2025.07 | 100 | 0.82 | |
| LLaMA3-8B-InstructModel Scale=8B2025.07 | 100 | 0.73 | |
| General-Verifier2025.07 | 99.8 | 0.72 |