List your company
ResearchFormerly WizWand

LLM Judging Agreement on Reasoning Agreement Benchmark 500-sample human-annotated

0.91Cohen's KappaMulti-sub RM
0.06760.28630.5050.7237Jul 11, 2025
Updated 4mo ago

Evaluation results

MethodLinks
Multi-sub RM2025.070.91
GPT-4o2025.070.9
Master-RM-7BModel Scale=7B2025.070.9
Qwen2.5-72B-InstructModel Scale=72B2025.070.88
Qwen2.5-32B-InstructModel Scale=32B2025.070.88
Qwen2.5-14B-InstructModel Scale=14B2025.070.88
Master-RM-32BModel Scale=32B2025.070.87
Qwen2.5-1.5B-InstructModel Scale=1.5B2025.070.83
Qwen2.5-3B-InstructModel Scale=3B2025.070.82
Omni-Judge2025.070.81
LLaMA3-70B-InstructModel Scale=70B2025.070.81
Qwen2.5-7B-InstructModel Scale=7B2025.070.8
LLaMA3-8B-InstructModel Scale=8B2025.070.73
General-Verifier2025.070.7
Qwen2.5-0.5B-InstructModel Scale=0.5B2025.070.1