# Spatial Visual Question Answering on Human-annotated quantitative spatial relationship dataset (test)

Source: [Spatial Visual Question Answering on Human-annotated quantitative spatial relationship dataset (test)](<https://www.sota2.com/research/sota/spatial-visual-question-answering-on-human-annotated-quantitative-spatial-relationship-dataset-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:54:21.468Z

Evaluation of Spatial Visual Question Answering performance on the test split of the Human-annotated quantitative spatial relationship dataset, focusing on the accuracy of outputting quantitative spatial metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Spatial Visual Question Answering |
| Dataset (including any stated split) | Human-annotated quantitative spatial relationship dataset (test) |
| Metric | Output Accuracy (%) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Output Accuracy (%)](<https://www.sota2.com/research/sota/spatial-visual-question-answering-on-human-annotated-quantitative-spatial-relationship-dataset-test.md?metric=Output%20Accuracy%20(%25)>): Higher is better; unit: PERCENT.

- [Range Compliance Rate \[50-200\]%](<https://www.sota2.com/research/sota/spatial-visual-question-answering-on-human-annotated-quantitative-spatial-relationship-dataset-test.md?metric=Range%20Compliance%20Rate%20%5B50-200%5D%25>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Output Accuracy (%) | Result date | Report paper |
| --- | --- | --- | --- |
| SpatialVLM | 99 | Not recorded | 68fa71f91e209131cea53d86 |
| PaLM 2-E | 88.8 | Not recorded | 68fa71f91e209131cea53d86 |
| PaLM-E | 83.2 | Not recorded | 68fa71f91e209131cea53d86 |
| PaLI | 52 | Not recorded | 68fa71f91e209131cea53d86 |
| InstructBLIP | 26 | Not recorded | 68fa71f91e209131cea53d86 |
| LLaVA-1.5 | 20.9 | Not recorded | 68fa71f91e209131cea53d86 |
| GPT-4V | 1 | Not recorded | 68fa71f91e209131cea53d86 |

### SpatialVLM

Output Accuracy (%): 99. Result ID: 6992132d3a6c3148490c68b9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Implementation](<https://spatial-vlm.github.io/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### PaLM 2-E

Output Accuracy (%): 88.8. Result ID: 6992132d3a6c3148490c68b8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa708e1e209131cea4bc0a.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### PaLM-E

Output Accuracy (%): 83.2. Result ID: 6992132d3a6c3148490c68b7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6eb61e209131cea41b08.md>)

[Implementation](<https://palm-e.github.io/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### PaLI

Output Accuracy (%): 52. Result ID: 6992132d3a6c3148490c68b6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f921e209131cea46550.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### InstructBLIP

Output Accuracy (%): 26. Result ID: 6992132d3a6c3148490c68b5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6fb71e209131cea47021.md>)

[Implementation](<https://github.com/salesforce/LAVIS/tree/main/projects/instructblip>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### LLaVA-1.5

Output Accuracy (%): 20.9. Result ID: 6992132d3a6c3148490c68b4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### GPT-4V

Output Accuracy (%): 1. Result ID: 6992132d3a6c3148490c68b3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.