Embodied AI reasoning on ERIQ 6K (full)
96.67Action UnderstandingOurs-3B
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ours-3Bsetting=zero-shot2025.12 | 96.67 | 85.25 | 90.5 | 55.36 | 73.86 | 51.6 | 96.44 | 83.26 | 75.45 | 93.1 | 85.71 | 82.72 | |
| Gemini-2.5-prosetting=zero-shot2025.12 | 89.83 | 67.37 | 76.67 | 81.25 | 75.05 | 62.28 | 91.11 | 87.67 | 67.07 | 90.52 | 87.62 | 80.55 | |
| Qwen3-VL-8Bsetting=zero-shot2025.12 | 89.17 | 69.38 | 66.33 | 65.18 | 63.76 | 33.45 | 89.78 | 76.65 | 52.99 | 86.21 | 80 | 75.53 | |
| Cosmos-Reason1-7Bsetting=zero-shot2025.12 | 86.33 | 54.75 | 65 | 53.57 | 63.37 | 22.78 | 84 | 81.94 | 58.08 | 69.83 | 72.38 | 67.99 | |
| GPT-4o-minisetting=zero-shot2025.12 | 84.67 | 71.63 | 65.17 | 71.43 | 68.91 | 49.82 | 89.78 | 79.74 | 66.77 | 77.59 | 68.57 | 77.61 | |
| RoboBrain2.0-7Bsetting=zero-shot2025.12 | 82.33 | 56.88 | 61 | 51.79 | 58.22 | 24.56 | 84.89 | 83.26 | 50.9 | 76.72 | 57.14 | 67.38 | |
| InternVL-3.5-8Bsetting=zero-shot2025.12 | 78.33 | 55 | 59.5 | 59.82 | 58.42 | 27.4 | 80.89 | 83.26 | 50.6 | 74.14 | 65.71 | 66.72 | |
| Qwen2.5-VL-7Bsetting=zero-shot2025.12 | 76.83 | 62.62 | 60.67 | 54.46 | 60.4 | 24.2 | 73.78 | 77.53 | 50.3 | 72.41 | 64.76 | 66.69 | |
| Claude-Sonnet-4setting=zero-shot2025.12 | 67.67 | 61.5 | 61.33 | 62.5 | 55.84 | 32.03 | 92.89 | 83.26 | 37.13 | 84.48 | 83.81 | 65.66 | |
| Qwen2.5-VL-3Bsetting=zero-shot2025.12 | 65.5 | 52.75 | 55.67 | 46.43 | 57.62 | 25.98 | 86.22 | 78.85 | 34.43 | 73.28 | 59.05 | 58.64 |