Cross-task physical reasoning on PHYRE Set 3 (test)
52Pass@1Human
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HumanCategory=Oracle2026.07 | 52 | 73.2 | 82 | |
| DQN-expertCategory=Oracle2026.07 | 33.8 | 41.2 | 46.6 | |
| +EG+VAORABase Model=Qwen3VL-8B-SFT, Protocol=Expert-Guided + VAORA2026.07 | 30 | 45.6 | 51.2 | |
| Gemini-3.1-ProCategory=Closed-Source API2026.07 | 27.8 | 48.4 | 55.4 | |
| +EG (Expert-Guided)Base Model=Qwen3VL-8B-SFT, Protocol=Expert-Guided2026.07 | 21.8 | 34 | 39.2 | |
| Qwen3VL-8B (SFT)Category=Open-Sourced Baselines, Protocol=SFT2026.07 | 17.6 | 36.8 | 45.2 | |
| Gemini-3.1-FlashCategory=Closed-Source API2026.07 | 14.2 | 29.4 | 37.6 | |
| InternVL-3.5-8BCategory=Open-Sourced Baselines2026.07 | 5 | 10.4 | 15.4 | |
| GPT-5.4Category=Closed-Source API2026.07 | 4.4 | 8.6 | 14 | |
| Claude-Sonnet-4.6Category=Closed-Source API2026.07 | 3 | 7.4 | 10.8 | |
| Qwen3VL-8B (GRPO)Category=Open-Sourced Baselines, Protocol=GRPO2026.07 | 2.4 | 6 | 10.8 | |
| Qwen3VL-8B (SFT+GRPO)Category=Open-Sourced Baselines, Protocol=SFT+GRPO2026.07 | 2.2 | 7.8 | 10.2 | |
| Qwen3VL-8BCategory=Open-Sourced Baselines2026.07 | 1.6 | 4 | 7.4 |