Reward Modeling on WebArena
88.43Pairwise AccuracyWebArbiter-7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| WebArbiter-7BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=7B2026.01 | 88.43 | 68.66 | |
| GPT-5Evaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 84.83 | 71.64 | |
| GPT-4oEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 84.58 | 66.67 | |
| Claude-3.7-SonnetEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 82.8 | 64.1 | |
| Gemini-2.5-FlashEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 82.71 | 62.19 | |
| DeepSeek-R1Evaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 82.04 | 60.21 | |
| WebArbiter-3BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=3B2026.01 | 81.97 | 56.22 | |
| GPT-4o-miniEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 78.23 | 56.72 | |
| Llama-3-70B-InstructEvaluation Protocol=LLM-as-judge, Model Source=Open-source2026.01 | 77.36 | 50.75 | |
| Qwen2.5-7B-InstructEvaluation Protocol=LLM-as-judge, Model Source=Open-source2026.01 | 74.88 | 42.79 | |
| WebShepherd-8BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=8B2026.01 | 68.33 | 43.88 | |
| WebShepherd-3BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=3B2026.01 | 68.16 | 41.29 | |
| Qwen2.5-3B-InstructEvaluation Protocol=LLM-as-judge, Model Source=Open-source2026.01 | 60.32 | 15.42 |