Binary Predicate Prediction on WebLI Qualitative Spatial VQA human-annotated (test)
75.2AccuracySpatialVLM
Evaluation results
| Method | Links | |
|---|---|---|
| SpatialVLM2024.01 | 75.2 | |
| LLaVA-1.52024.01 | 71.3 | |
| GPT-4V2024.01 | 68 | |
| PaLIvariant=PaLI-X 55B2024.01 | 60.7 | |
| InstructBLIP2024.01 | 60.4 | |
| PaLM 2-Eparameter count=PaLM 2-S equivalent2024.01 | 50.4 | |
| PaLM-Evariant=PaLM-E 12B2024.01 | 50.2 |