Cross-task physical reasoning on PHYRE Set 1 (test)
52Pass@1Human
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HumanCategory=Oracle2026.07 | 52 | 75.8 | 80 | |
| +EG+VAORABase Model=Qwen3VL-8B-SFT, Protocol=Expert-Guided + VAORA2026.07 | 38.2 | 52.4 | 59.4 | |
| DQN-expertCategory=Oracle2026.07 | 32.6 | 43.4 | 47.6 | |
| Gemini-3.1-ProCategory=Closed-Source API2026.07 | 27.8 | 47.2 | 55.2 | |
| +EG (Expert-Guided)Base Model=Qwen3VL-8B-SFT, Protocol=Expert-Guided2026.07 | 27 | 43.6 | 51.2 | |
| Qwen3VL-8B (SFT)Category=Open-Sourced Baselines, Protocol=SFT2026.07 | 17.6 | 38 | 47 | |
| Gemini-3.1-FlashCategory=Closed-Source API2026.07 | 17 | 34 | 41.6 | |
| GPT-5.4Category=Closed-Source API2026.07 | 6.4 | 13.2 | 18.6 | |
| InternVL-3.5-8BCategory=Open-Sourced Baselines2026.07 | 4.4 | 9.4 | 13.4 | |
| Claude-Sonnet-4.6Category=Closed-Source API2026.07 | 4.2 | 10.4 | 14.2 | |
| Qwen3VL-8B (SFT+GRPO)Category=Open-Sourced Baselines, Protocol=SFT+GRPO2026.07 | 2 | 2.6 | 3 | |
| Qwen3VL-8B (GRPO)Category=Open-Sourced Baselines, Protocol=GRPO2026.07 | 0.6 | 4 | 6.2 | |
| Qwen3VL-8BCategory=Open-Sourced Baselines2026.07 | 0.2 | 0.8 | 20 |