Physical Task Execution on Virtual-Tool (test)
33.3Pass@1Human
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Human2026.07 | 33.3 | 66.1 | 77.8 | |
| +EG+VAORAbase model=Qwen3VL-8B-SFT2026.07 | 16.7 | 22.2 | 33.3 | |
| Gemini-3.1-Flashtype=Closed-Source API2026.07 | 11.1 | 27.8 | 27.8 | |
| Gemini-3.1-Protype=Closed-Source API2026.07 | 11.1 | 27.8 | 38.9 | |
| Qwen3-VL-8B-Instructtype=Open-Sourced Baseline2026.07 | 5.6 | 16.7 | 16.7 | |
| DQN-expert (phyre)2026.07 | 0 | 5.6 | 5.6 | |
| Qwen3-VL-8B (SFT)type=Open-Sourced Baseline2026.07 | 0 | 16.7 | 22.2 |