Reward Modeling on Mind2Web
97.07Pairwise AccWebArbiter-7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| WebArbiter-7BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=7B2026.01 | 97.07 | 89.53 | |
| WebArbiter-3BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=3B2026.01 | 93.32 | 78.42 | |
| WebShepherd-3BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=3B2026.01 | 87.5 | 65.21 | |
| WebShepherd-8BEvaluation Protocol=WebPRM, Model Source=WebPRMs, Model Scale=8B2026.01 | 86.66 | 73.69 | |
| GPT-4o-miniEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 81.74 | 50.92 | |
| DeepSeek-R1Evaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 81.62 | 57.37 | |
| Gemini-2.5-FlashEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 81.3 | 57.01 | |
| GPT-5Evaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 80.86 | 62.39 | |
| Llama-3-70B-InstructEvaluation Protocol=LLM-as-judge, Model Source=Open-source2026.01 | 80.55 | 49.36 | |
| Claude-3.7-SonnetEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 80.2 | 57.9 | |
| GPT-4oEvaluation Protocol=LLM-as-judge, Model Source=Proprietary2026.01 | 79.99 | 52.62 | |
| Qwen2.5-7B-InstructEvaluation Protocol=LLM-as-judge, Model Source=Open-source2026.01 | 77.79 | 39.18 | |
| Qwen2.5-3B-InstructEvaluation Protocol=LLM-as-judge, Model Source=Open-source2026.01 | 76.46 | 36.93 |