Embodied Decision-making on Game for Peace 1.0 (test)
71.4Parachuting ScoreMAIN-VLA
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MAIN-VLAModel Category=Vision-Language-Action Models, Evaluation Protocol=fine-tuned on 200-hour trajectory dataset with IA and ESA2026.02 | 71.4 | 64.3 | 61.4 | 74.3 | 67.1 | 68.6 | 67.9 | 0.3 | |
| Gemini-2.5-ProModel Category=Proprietary Foundation Models, Evaluation Protocol=zero-shot CoT2026.02 | 55.7 | 42.8 | 25.7 | 58.6 | 47.1 | 51.4 | 46.9 | 1.5 | |
| Qwen2-VL-7B (Vanilla IL)Model Category=Vision-Language-Action Models, Evaluation Protocol=vanilla imitation learning fine-tuned on 200-hour trajectory dataset2026.02 | 54.3 | 48.6 | 44.3 | 68.6 | 51.4 | 52.9 | 53.4 | 1.1 | |
| Claude-3.7-SonnetModel Category=Proprietary Foundation Models, Evaluation Protocol=zero-shot CoT2026.02 | 48.6 | 38.6 | 24.3 | 57.1 | 41.4 | 45.7 | 42.6 | 2.2 | |
| GPT-4oModel Category=Proprietary Foundation Models, Evaluation Protocol=zero-shot CoT2026.02 | 41.4 | 31.4 | 18.6 | 47.1 | 38.6 | 40 | 36.2 | 1.6 | |
| Qwen2-VL-7BModel Category=Open-Source Vision-Language Models, Evaluation Protocol=generalist baseline2026.02 | 22.9 | 15.7 | 8.6 | 27.1 | 18.6 | 21.4 | 19.1 | 1.1 |