Reward Modeling on WorkArena
84.33Pairwise AccuracyGPT-4o
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-4oEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 84.33 | 55.19 | |
| DeepSeek-R1Evaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 84.12 | 63.89 | |
| Gemini-2.5-FlashEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 83.3 | 56.13 | |
| Claude-3.7-SonnetEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 82.1 | 60.6 | |
| WebArbiter-7BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=7B2026.01 | 82.09 | 70.19 | |
| GPT-4o-miniEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 81.43 | 46.7 | |
| GPT-5Evaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 81.14 | 64.62 | |
| WebArbiter-3BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=3B2026.01 | 81.01 | 54.81 | |
| Llama-3-70B-InstructEvaluation Protocol=LLM-as-judge, Model Source=Open-source2026.01 | 79.08 | 40.09 | |
| Qwen2.5-7B-InstructEvaluation Protocol=LLM-as-judge, Model Source=Open-source2026.01 | 77.58 | 35.85 | |
| Qwen2.5-3B-InstructEvaluation Protocol=LLM-as-judge, Model Source=Open-source2026.01 | 64.45 | 19.34 | |
| WebShepherd-8BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=8B2026.01 | 54.56 | 25.53 | |
| WebShepherd-3BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=3B2026.01 | 50 | 21.23 |