Reward Modeling on OGRBench
88.1Ubuntu AccuracyQwen3-VL-235B
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-VL-235BFramework=OS-Themis2026.03 | 88.1 | 88.3 | 92.3 | 92.8 | 77.5 | 68.4 | 94.8 | 86.7 | 92.1 | 92.2 | 88 | 92.8 | 82.3 | 87.2 | |
| Gemini-3-FlashFramework=OS-Themis2026.03 | 85 | 84.7 | 91 | 91.1 | 86.9 | 83.7 | 93.5 | 82.8 | 82.6 | 80.5 | 86.2 | 93.2 | 78 | 84.9 | |
| Qwen3-VL-235B-ThinkingFramework=OS-Themis2026.03 | 83.4 | 83.7 | 89.4 | 89.6 | 85.5 | 82.5 | 93.5 | 82.8 | 84.7 | 83.4 | 85.2 | 89.3 | 79.9 | 84.3 | |
| GPT-5Framework=OS-Themis2026.03 | 82.5 | 81.3 | 80.3 | 78.4 | 84.5 | 80.5 | 88.3 | 60.9 | 95.9 | 80.4 | 82.9 | 93.4 | 70.6 | 80.4 | |
| Qwen3-VL-30B-A3BFramework=OS-Themis2026.03 | 79.5 | 79.4 | 84.6 | 85 | 76.5 | 70.9 | 88.3 | 66.7 | 80.5 | 79.3 | 80.3 | 84.7 | 73.7 | 78.8 | |
| Qwen3-VL-32BFramework=OS-Themis2026.03 | 77.6 | 75.2 | 83.5 | 82.1 | 75.1 | 63.5 | 88.3 | 69 | 84.7 | 83.3 | 79.6 | 92.2 | 64.3 | 75.8 | |
| Qwen3-VL-8BFramework=OS-Themis2026.03 | 77.2 | 75.8 | 85.6 | 85.1 | 72.8 | 61.8 | 85.7 | 59.3 | 86.3 | 87.2 | 79.3 | 86.3 | 69.4 | 77 | |
| Qwen3-VL-235BFramework=ZeroGUI2026.03 | 76.5 | 74.5 | 82.5 | 84.2 | 77.9 | 70.8 | 93.5 | 81.5 | 84.7 | 85.6 | 79.6 | 85.5 | 70.9 | 77.5 | |
| Gemini-3-FlashFramework=ZeroGUI2026.03 | 76.1 | 72.8 | 77.1 | 76 | 80.8 | 74.5 | 93.5 | 81.5 | 77.4 | 75.1 | 78.1 | 89.8 | 63 | 74.1 | |
| Qwen3-VL-32BFramework=ZeroGUI2026.03 | 75 | 72.2 | 78.7 | 80 | 79.3 | 72.5 | 89.6 | 66.7 | 76.3 | 75.9 | 77.2 | 85.3 | 65.3 | 74 | |
| Qwen3-VL-30B-A3BFramework=ZeroGUI2026.03 | 74.8 | 74.6 | 80.3 | 82.8 | 80.8 | 77.3 | 90.9 | 75.9 | 72.1 | 75.1 | 76.9 | 77.9 | 74.9 | 76.3 | |
| Qwen3-VL-8BFramework=ZeroGUI2026.03 | 72.7 | 67.8 | 78.7 | 79.6 | 76.5 | 67.5 | 89.6 | 69.2 | 75.8 | 75 | 75.4 | 86.3 | 60.1 | 70.9 | |
| GPT-5Framework=ZeroGUI2026.03 | 72.6 | 67 | 81.9 | 82.3 | 77.5 | 68.8 | 90.9 | 72 | 80.5 | 79.6 | 76.7 | 89.7 | 59.9 | 71.8 | |
| Qwen3-VL-235B-ThinkingFramework=ZeroGUI2026.03 | 71.8 | 66.9 | 79.8 | 82.1 | 79.3 | 72.5 | 89.6 | 66.7 | 79 | 78.3 | 75.9 | 85.3 | 62.3 | 72 | |
| Gemini-3-FlashFramework=DigiRL2026.03 | 69.5 | 63 | 69.7 | 64.2 | 76.5 | 66.2 | 83.1 | 31.6 | 75.3 | 74.3 | 72.1 | 86.6 | 51.9 | 64.9 | |
| GPT-5-miniFramework=OS-Themis2026.03 | 68.8 | 59.7 | 65.4 | 52.5 | 76.5 | 64.3 | 87 | 54.5 | 75.8 | 70.5 | 71.5 | 95.4 | 44.7 | 60.9 | |
| Qwen3-VL-32BFramework=DigiRL2026.03 | 65.7 | 66.5 | 72.3 | 77.8 | 71.8 | 71.2 | 75.3 | 55.8 | 62.1 | 67.9 | 67.6 | 65.8 | 72.1 | 68.8 | |
| Qwen3-VL-235BFramework=DigiRL2026.03 | 65.5 | 67 | 70.2 | 75.2 | 72.3 | 70.9 | 76.6 | 43.8 | 63.2 | 62.8 | 67.4 | 66.6 | 69 | 67.8 | |
| Qwen3-VL-235B-ThinkingFramework=DigiRL2026.03 | 62.4 | 66.3 | 67.5 | 74.7 | 65.7 | 66 | 70.1 | 46.5 | 58.4 | 62.2 | 63.5 | 61 | 73 | 66.5 | |
| Qwen3-VL-8BFramework=DigiRL2026.03 | 62.1 | 64.9 | 69.2 | 76.6 | 66.2 | 63.6 | 72.7 | 40 | 63.2 | 69.6 | 64.4 | 62.2 | 72.1 | 66.8 | |
| Qwen3-VL-30B-A3BFramework=DigiRL2026.03 | 61 | 67.6 | 67 | 75.6 | 69 | 71.1 | 67.5 | 49 | 54.2 | 63 | 62.5 | 58.9 | 80.6 | 68.1 | |
| GPT-5Framework=DigiRL2026.03 | 50.5 | 13.7 | 56.9 | 33.1 | 59.1 | 17.1 | 79.2 | 0 | 52.6 | 16.7 | 54.5 | 89.3 | 9.6 | 17.3 | |
| GPT-5-miniFramework=ZeroGUI2026.03 | 48.5 | 6.4 | 49.5 | 5.9 | 56.3 | 2.1 | 79.2 | 0 | 49 | 5.8 | 51.5 | 87 | 2.9 | 5.5 | |
| GPT-5-miniFramework=DigiRL2026.03 | 47 | 0 | 47.9 | 0 | 55.9 | 0 | 79.2 | 0 | 47.9 | 0 | 50.3 | 0 | 0 | 0 |