# Binary Predicate Prediction on WebLI Qualitative Spatial VQA human-annotated (test)

Source: [Binary Predicate Prediction on WebLI Qualitative Spatial VQA human-annotated (test)](<https://www.sota2.com/research/sota/binary-predicate-prediction-on-webli-qualitative-spatial-vqa-human-annotated-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:29:25.470Z

Evaluation of Binary Predicate Prediction performance on the human-annotated test split of the WebLI Qualitative Spatial VQA dataset, measured using Accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Binary Predicate Prediction |
| Dataset (including any stated split) | WebLI Qualitative Spatial VQA human-annotated (test) |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/binary-predicate-prediction-on-webli-qualitative-spatial-vqa-human-annotated-test.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| SpatialVLM | 75.2 | Not recorded | 68fa71f91e209131cea53d86 |
| LLaVA-1.5 | 71.3 | Not recorded | 68fa71f91e209131cea53d86 |
| GPT-4V | 68 | Not recorded | 68fa71f91e209131cea53d86 |
| PaLI | 60.7 | Not recorded | 68fa71f91e209131cea53d86 |
| InstructBLIP | 60.4 | Not recorded | 68fa71f91e209131cea53d86 |
| PaLM 2-E | 50.4 | Not recorded | 68fa71f91e209131cea53d86 |
| PaLM-E | 50.2 | Not recorded | 68fa71f91e209131cea53d86 |

### SpatialVLM

Accuracy: 75.2. Result ID: 6992132d3a6c3148490c68c0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Implementation](<https://spatial-vlm.github.io/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### LLaVA-1.5

Accuracy: 71.3. Result ID: 6992132d3a6c3148490c68bb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### GPT-4V

Accuracy: 68. Result ID: 6992132d3a6c3148490c68ba.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### PaLI

Accuracy: 60.7. Result ID: 6992132d3a6c3148490c68bd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f921e209131cea46550.md>)

Recorded evaluation configuration: {"variant":"PaLI-X 55B"}. This record does not assert independent reproduction.

### InstructBLIP

Accuracy: 60.4. Result ID: 6992132d3a6c3148490c68bc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6fb71e209131cea47021.md>)

[Implementation](<https://github.com/salesforce/LAVIS/tree/main/projects/instructblip>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### PaLM 2-E

Accuracy: 50.4. Result ID: 6992132d3a6c3148490c68bf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa708e1e209131cea4bc0a.md>)

Recorded evaluation configuration: {"parameter count":"PaLM 2-S equivalent"}. This record does not assert independent reproduction.

### PaLM-E

Accuracy: 50.2. Result ID: 6992132d3a6c3148490c68be.

[Reporting paper](<https://www.sota2.com/research/paper/68fa71f91e209131cea53d86.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6eb61e209131cea41b08.md>)

[Implementation](<https://palm-e.github.io/>)

Recorded evaluation configuration: {"variant":"PaLM-E 12B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.