Trajectory completion judgment on OGRBench Web
87.4AccuracyOS-Themis
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OS-ThemisVerifier=Qwen3.5-27B2026.06 | 87.4 | 86.7 | |
| StainFlowVerifier=Qwen3.5-27B2026.06 | 87.4 | 87.2 | |
| StainFlowVerifier=Gemini-3-Flash2026.06 | 86.2 | 86.2 | |
| ZeroGUIVerifier=Qwen3.5-27B2026.06 | 85.3 | 86.7 | |
| ZeroGUIVerifier=Gemini-3-Flash2026.06 | 83.1 | 84.2 | |
| OS-ThemisVerifier=Qwen3-VL-8B2026.06 | 82.6 | 83.6 | |
| StainFlowVerifier=GPT-52026.06 | 82.6 | 82.2 | |
| OS-ThemisVerifier=Gemini-3-Flash2026.06 | 82.5 | 81.8 | |
| ZeroGUIVerifier=GPT-52026.06 | 81.6 | 82.1 | |
| OS-ThemisVerifier=Qwen3.5-9B2026.06 | 81.6 | 80 | |
| StainFlowVerifier=Qwen3.5-9B2026.06 | 80.5 | 79.6 | |
| ZeroGUIVerifier=Qwen3.5-9B2026.06 | 79.5 | 81.2 | |
| StainFlowVerifier=Qwen3-VL-8B2026.06 | 79.5 | 82.5 | |
| ZeroGUIVerifier=Qwen3-VL-8B2026.06 | 79 | 82.2 | |
| OS-ThemisVerifier=GPT-52026.06 | 77.9 | 75.3 | |
| DigiRLVerifier=Qwen3.5-27B2026.06 | 68.4 | 63 | |
| DigiRLVerifier=GPT-52026.06 | 66.8 | 61.8 | |
| DigiRLVerifier=Gemini-3-Flash2026.06 | 66.7 | 63.6 | |
| DigiRLVerifier=Qwen3-VL-8B2026.06 | 65.3 | 60.7 | |
| DigiRLVerifier=Qwen3.5-9B2026.06 | 61.1 | 47.1 |