# Spatial Reasoning on VSI-Bench 59 (test)

Source: [Spatial Reasoning on VSI-Bench 59 (test)](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-19T10:12:47.115Z

Evaluation of model performance on the VSI-Bench 59 test split, focusing on Spatial Reasoning capabilities. Metrics include object count, absolute/relative distance and direction errors, room size analysis, route planning success, and approach order accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Spatial Reasoning |
| Dataset (including any stated split) | VSI-Bench 59 (test) |
| Metric | Object Count Score |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Object Count Score](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test.md?metric=Object%20Count%20Score>): Higher is better; unit: SCALAR.

- [Absolute Distance Error](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test.md?metric=Absolute%20Distance%20Error>): Lower is better; unit: SCALAR.

- [Object Size Score](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test.md?metric=Object%20Size%20Score>): Higher is better; unit: SCALAR.

- [Room Size Score](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test.md?metric=Room%20Size%20Score>): Higher is better; unit: SCALAR.

- [Relative Distance Error](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test.md?metric=Relative%20Distance%20Error>): Lower is better; unit: SCALAR.

- [Relative Direction Error](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test.md?metric=Relative%20Direction%20Error>): Lower is better; unit: SCALAR.

- [Route Planning Success Rate](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test.md?metric=Route%20Planning%20Success%20Rate>): Higher is better; unit: PERCENT.

- [Approach Order Accuracy](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test.md?metric=Approach%20Order%20Accuracy>): Higher is better; unit: PERCENT.

- [Average Score](<https://www.sota2.com/research/sota/spatial-reasoning-on-vsi-bench-59-test.md?metric=Average%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Object Count Score | Result date | Report paper |
| --- | --- | --- | --- |
| Motion-MLLM-4B | 72.5 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| InternVL3-78B | 71.2 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| VG LLM-8B | 67.9 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| VG LLM-4B | 66 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| Spatial-MLLM-4B | 65.3 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| Spacer | 57.8 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| Gemini-1.5 Pro | 56.2 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| LLaVA-Video-72B | 48.9 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| GPT-4o | 46.2 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| LLaVA-OneVision-72B | 43.5 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| Qwen2.5-VL-7B | 40.9 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| Qwen2.5-VL-72B | 25.1 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| Qwen2.5-VL-3B | 24.3 | Not recorded | 69bb830e0636e5bf4afc1a90 |
| VILA-1.5-40B | 22.4 | Not recorded | 69bb830e0636e5bf4afc1a90 |

### Motion-MLLM-4B

Object Count Score: 72.5. Result ID: 69bbcc1547f34ce924a64f73.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

Recorded evaluation configuration: {"Parameters":"4B","Keyframes":"selected by motion-visual keyframe filtering module"}. This record does not assert independent reproduction.

### InternVL3-78B

Object Count Score: 71.2. Result ID: 69bbcc1547f34ce924a64f6e.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78241e209131cea76fa5.md>)

Recorded evaluation configuration: {"Model Category":"Open-source Models","Parameters":"78B"}. This record does not assert independent reproduction.

### VG LLM-8B

Object Count Score: 67.9. Result ID: 69bbcc1547f34ce924a64f71.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c701e209131cea8e12b.md>)

Recorded evaluation configuration: {"Model Category":"Spatial Reasoning Models","Parameters":"8B"}. This record does not assert independent reproduction.

### VG LLM-4B

Object Count Score: 66. Result ID: 69bbcc1547f34ce924a64f70.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c701e209131cea8e12b.md>)

Recorded evaluation configuration: {"Model Category":"Spatial Reasoning Models","Parameters":"4B"}. This record does not assert independent reproduction.

### Spatial-MLLM-4B

Object Count Score: 65.3. Result ID: 69bbcc1547f34ce924a64f72.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78f11e209131cea7b58a.md>)

Recorded evaluation configuration: {"Model Category":"Spatial Reasoning Models","Parameters":"4B"}. This record does not assert independent reproduction.

### Spacer

Object Count Score: 57.8. Result ID: 69bbcc1547f34ce924a64f6f.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a41e209131cea798a1.md>)

Recorded evaluation configuration: {"Model Category":"Spatial Reasoning Models"}. This record does not assert independent reproduction.

### Gemini-1.5 Pro

Object Count Score: 56.2. Result ID: 69bbcc1547f34ce924a64f67.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76361e209131cea6be11.md>)

Recorded evaluation configuration: {"Model Category":"Proprietary Models (API)"}. This record does not assert independent reproduction.

### LLaVA-Video-72B

Object Count Score: 48.9. Result ID: 69bbcc1547f34ce924a64f6a.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a561e209131cea82934.md>)

[Implementation](<https://llava-vl.github.io/blog/llava-video>)

Recorded evaluation configuration: {"Model Category":"Open-source Models","Parameters":"72B"}. This record does not assert independent reproduction.

### GPT-4o

Object Count Score: 46.2. Result ID: 69bbcc1547f34ce924a64f66.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Model Category":"Proprietary Models (API)"}. This record does not assert independent reproduction.

### LLaVA-OneVision-72B

Object Count Score: 43.5. Result ID: 69bbcc1547f34ce924a64f69.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756f1e209131cea67e58.md>)

[Implementation](<https://llava-vl.github.io/blog/llava-onevision>)

Recorded evaluation configuration: {"Model Category":"Open-source Models","Parameters":"72B"}. This record does not assert independent reproduction.

### Qwen2.5-VL-7B

Object Count Score: 40.9. Result ID: 69bbcc1547f34ce924a64f6c.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa773a1e209131cea71d3c.md>)

Recorded evaluation configuration: {"Model Category":"Open-source Models","Parameters":"7B"}. This record does not assert independent reproduction.

### Qwen2.5-VL-72B

Object Count Score: 25.1. Result ID: 69bbcc1547f34ce924a64f6d.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa773a1e209131cea71d3c.md>)

Recorded evaluation configuration: {"Model Category":"Open-source Models","Parameters":"72B"}. This record does not assert independent reproduction.

### Qwen2.5-VL-3B

Object Count Score: 24.3. Result ID: 69bbcc1547f34ce924a64f6b.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa773a1e209131cea71d3c.md>)

Recorded evaluation configuration: {"Model Category":"Open-source Models","Parameters":"3B"}. This record does not assert independent reproduction.

### VILA-1.5-40B

Object Count Score: 22.4. Result ID: 69bbcc1547f34ce924a64f68.

[Reporting paper](<https://www.sota2.com/research/paper/69bb830e0636e5bf4afc1a90.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73311e209131cea5b06b.md>)

[Implementation](<https://github.com/NVlabs/VILA>)

Recorded evaluation configuration: {"Model Category":"Open-source Models","Parameters":"40B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
