Spatial Reasoning on MMSI-Bench MindJourney Subset (162 questions) (test)
0.358AccuracyViSA
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ViSABeam depth (γ)=1, top-k=12025.12 | 0.358 | 0.2667 | 0.2667 | 0.2667 | 0.3333 | 0.4667 | 0.3333 | 0.6667 | 0.1333 | 0.3333 | 0.5333 | 0.3333 | |
| MindJourneyBeam depth (γ)=2, top-k=22025.12 | 0.3395 | 0.0667 | 0.3333 | 0.2667 | 0.3333 | 0.4 | 0.4 | 0.5333 | 0.2 | 0.3333 | 0.4 | 0.5 | |
| RandomBeam depth (γ)=1, top-k=12025.12 | 0.3333 | 0.2 | 0.2667 | 0.4667 | 0.2667 | 0.4 | 0.3333 | 0.6 | 0.1333 | 0.2667 | 0.5333 | 0.1667 | |
| RandomBeam depth (γ)=1, top-k=32025.12 | 0.3333 | 0.2 | 0.2 | 0.4 | 0.1333 | 0.4 | 0.2 | 0.5333 | 0.2667 | 0.2 | 0.6667 | 0.5 | |
| RandomBeam depth (γ)=2, top-k=22025.12 | 0.3333 | 0.2667 | 0.2667 | 0.4 | 0.4 | 0.4 | 0.4667 | 0.3333 | 0.2 | 0.2 | 0.5333 | 0.1667 | |
| MindJourneyBeam depth (γ)=1, top-k=12025.12 | 0.3272 | 0.1333 | 0.1333 | 0.4 | 0.4667 | 0.4667 | 0.3333 | 0.6667 | 0.1333 | 0.2667 | 0.3333 | 0.25 | |
| ViSABeam depth (γ)=1, top-k=32025.12 | 0.3272 | 0.2 | 0.2 | 0.4 | 0.4 | 0.4667 | 0.2667 | 0.3333 | 0.2 | 0.3333 | 0.4667 | 0.3333 | |
| MindJourneyBeam depth (γ)=2, top-k=12025.12 | 0.3272 | 0.1333 | 0.2 | 0.4 | 0.2667 | 0.4667 | 0.2 | 0.4 | 0.2 | 0.3333 | 0.5333 | 0.5 | |
| MindJourneyBeam depth (γ)=2, top-k=32025.12 | 0.3272 | 0.2 | 0.2667 | 0.2667 | 0.3333 | 0.4 | 0.3333 | 0.3333 | 0.2 | 0.2 | 0.6 | 0.3333 | |
| ViSABeam depth (γ)=2, top-k=12025.12 | 0.321 | 0.1333 | 0.2667 | 0.5333 | 0.2667 | 0.4 | 0.3333 | 0.4667 | 0.2 | 0.2 | 0.4667 | 0.25 | |
| ViSABeam depth (γ)=2, top-k=22025.12 | 0.3148 | 0.0667 | 0.2667 | 0.5333 | 0.3333 | 0.3333 | 0.4 | 0.6 | 0.2 | 0.2 | 0.4 | 0.0833 | |
| RandomBeam depth (γ)=2, top-k=12025.12 | 0.3025 | 0.2 | 0.2 | 0.3333 | 0.3333 | 0.4667 | 0.4 | 0.4667 | 0.1333 | 0.2 | 0.4667 | 0.0833 | |
| RandomBeam depth (γ)=1, top-k=22025.12 | 0.2963 | 0.2 | 0.0667 | 0.4667 | 0.0667 | 0.4 | 0.4 | 0.4667 | 0.2 | 0 | 0.6667 | 0.3333 | |
| MindJourneyBeam depth (γ)=1, top-k=32025.12 | 0.2963 | 0.1333 | 0.2 | 0.3333 | 0.2667 | 0.5333 | 0.2667 | 0.3333 | 0.2 | 0.2 | 0.5333 | 0.25 | |
| InternVL3-14Bmode=baseline2025.12 | 0.2716 | 0.1333 | 0.2 | 0.3333 | 0.3333 | 0.3333 | 0.2 | 0.3333 | 0.1333 | 0.2667 | 0.5333 | 0.1667 | |
| MindJourneyBeam depth (γ)=1, top-k=22025.12 | 0.2716 | 0.2 | 0.1333 | 0.2 | 0.3333 | 0.2 | 0.4667 | 0.4 | 0.1333 | 0.1333 | 0.5333 | 0.25 | |
| RandomBeam depth (γ)=2, top-k=32025.12 | 0.2716 | 0.2667 | 0.2667 | 0.2667 | 0.2667 | 0.4 | 0.2 | 0.3333 | 0.1333 | 0.0667 | 0.4667 | 0.3333 | |
| ViSABeam depth (γ)=2, top-k=32025.12 | 0.2654 | 0.0667 | 0.1333 | 0.3333 | 0.2 | 0.2667 | 0.3333 | 0.5333 | 0.1333 | 0.2 | 0.4667 | 0.25 | |
| ViSABeam depth (γ)=1, top-k=22025.12 | 0.2593 | 0.1333 | 0.2 | 0.3333 | 0.1333 | 0.3333 | 0.2 | 0.2667 | 0.2 | 0.0667 | 0.6667 | 0.3333 |