3D Robotic Manipulation on PrimitiveSkill via ManiSkill
100Place Success RateGPT-5
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-5Model Category=Proprietary2026.05 | 100 | 63 | 0 | 100 | 55 | |
| o3Model Category=Proprietary2026.05 | 100 | 63 | 0 | 100 | 82 | |
| o4-miniModel Category=Proprietary2026.05 | 100 | 50 | 0 | 75 | 33 | |
| Qwen2.5-VL-72BModel Category=Open-Source2026.05 | 100 | 50 | 0 | 100 | 33 | |
| VAGENModel Category=Open-Source2026.05 | 100 | 88 | 88 | 88 | 50 | |
| AtlasVAModel Category=Ours, Parameter Count=3B2026.05 | 100 | 100 | 100 | 100 | 100 | |
| Gemini 2.5 flashModel Category=Proprietary2026.05 | 75 | 50 | 0 | 88 | 82 | |
| Gemini 2.0Model Category=Proprietary2026.05 | 75 | 13 | 0 | 25 | 33 | |
| Gemini 2.5 ProModel Category=Proprietary2026.05 | 63 | 63 | 0 | 75 | 0 | |
| Claude Sonnet 4.5Model Category=Proprietary2026.05 | 63 | 50 | 0 | 100 | 100 | |
| Claude Sonnet 3.7Model Category=Proprietary2026.05 | 63 | 13 | 0 | 100 | 90 | |
| GPT-4oModel Category=Proprietary2026.05 | 50 | 63 | 0 | 88 | 94 | |
| Qwen2.5-VL-7BModel Category=Open-Source2026.05 | 0 | 0 | 0 | 75 | 3 | |
| Qwen2.5-VL-3BModel Category=Open-Source2026.05 | 0 | 0 | 0 | 0 | 0 | |
| VLM-R1-3BModel Category=Open-Source2026.05 | 0 | 0 | 0 | 0 | 0 |