Trajectory completion judgment on OGRBench Windows
85.5AccuracyOS-Themis
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OS-ThemisVerifier=Gemini-3-Flash2026.06 | 85.5 | 83.2 | |
| StainFlowVerifier=Qwen3.5-27B2026.06 | 85 | 83.7 | |
| OS-ThemisVerifier=Qwen3.5-27B2026.06 | 84 | 81.1 | |
| StainFlowVerifier=Gemini-3-Flash2026.06 | 84 | 83 | |
| ZeroGUIVerifier=Qwen3.5-27B2026.06 | 82.2 | 78.9 | |
| StainFlowVerifier=Qwen3.5-9B2026.06 | 82.2 | 81 | |
| DigiRLVerifier=Gemini-3-Flash2026.06 | 81.7 | 76.4 | |
| DigiRLVerifier=GPT-52026.06 | 80.8 | 75.7 | |
| ZeroGUIVerifier=GPT-52026.06 | 79.8 | 77.5 | |
| ZeroGUIVerifier=Gemini-3-Flash2026.06 | 78.9 | 77.2 | |
| StainFlowVerifier=Qwen3-VL-8B2026.06 | 78.4 | 77.2 | |
| StainFlowVerifier=GPT-52026.06 | 78.4 | 77 | |
| DigiRLVerifier=Qwen3.5-27B2026.06 | 77.5 | 68.8 | |
| DigiRLVerifier=Qwen3-VL-8B2026.06 | 77 | 70.7 | |
| OS-ThemisVerifier=Qwen3.5-9B2026.06 | 77 | 71 | |
| OS-ThemisVerifier=GPT-52026.06 | 76.1 | 67.5 | |
| OS-ThemisVerifier=Qwen3-VL-8B2026.06 | 75.1 | 68.3 | |
| DigiRLVerifier=Qwen3.5-9B2026.06 | 74.2 | 62.1 | |
| ZeroGUIVerifier=Qwen3.5-9B2026.06 | 73.7 | 71.7 | |
| ZeroGUIVerifier=Qwen3-VL-8B2026.06 | 73.2 | 74.9 |