Cross-task physical reasoning on PHYRE Set 2 (test)
0.472Pass@1Human
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HumanCategory=Oracle2026.07 | 0.472 | 0.822 | 0.92 | |
| DQN-expertCategory=Oracle2026.07 | 0.26 | 0.368 | 0.418 | |
| Gemini-3.1-ProCategory=Closed-Source API2026.07 | 0.22 | 0.368 | 0.42 | |
| +EG+VAORABase Model=Qwen3VL-8B-SFT, Protocol=Expert-Guided + VAORA2026.07 | 0.198 | 0.39 | 0.472 | |
| Gemini-3.1-FlashCategory=Closed-Source API2026.07 | 0.192 | 0.302 | 0.352 | |
| +EG (Expert-Guided)Base Model=Qwen3VL-8B-SFT, Protocol=Expert-Guided2026.07 | 0.178 | 0.312 | 0.362 | |
| Qwen3VL-8B (SFT)Category=Open-Sourced Baselines, Protocol=SFT2026.07 | 0.146 | 0.344 | 0.452 | |
| GPT-5.4Category=Closed-Source API2026.07 | 0.048 | 0.096 | 0.144 | |
| Qwen3VL-8B (SFT+GRPO)Category=Open-Sourced Baselines, Protocol=SFT+GRPO2026.07 | 0.042 | 0.124 | 0.172 | |
| Claude-Sonnet-4.6Category=Closed-Source API2026.07 | 0.036 | 0.088 | 0.122 | |
| InternVL-3.5-8BCategory=Open-Sourced Baselines2026.07 | 0.028 | 0.084 | 0.13 | |
| Qwen3VL-8BCategory=Open-Sourced Baselines2026.07 | 0.018 | 0.044 | 0.078 | |
| Qwen3VL-8B (GRPO)Category=Open-Sourced Baselines, Protocol=GRPO2026.07 | 0.016 | 0.038 | 0.056 |