Robust Visual Question Answering on VizWiz (L-inf Perturbations)
41.5VQA Accuracy (Clean)Sim-CLIP²
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Sim-CLIP²VLM=LLaVA 1.5, Vision encoder=Sim-CLIP²2024.07 | 41.5 | 30.3 | 19.8 | 14.6 | |
| Sim-CLIP⁴VLM=LLaVA 1.5, Vision encoder=Sim-CLIP⁴2024.07 | 40 | 29.3 | 22.3 | 16.7 | |
| CLIPVLM=LLaVA 1.5, Vision encoder=CLIP2024.07 | 39.3 | 13.3 | 3.2 | 0 | |
| TeCoA²VLM=LLaVA 1.5, Vision encoder=TeCoA²2024.07 | 38.5 | 25.4 | 15.4 | 8.3 | |
| FARE⁴VLM=LLaVA 1.5, Vision encoder=FARE⁴2024.07 | 38.5 | 31.3 | 21 | 10.1 | |
| FARE²VLM=LLaVA 1.5, Vision encoder=FARE²2024.07 | 37.3 | 29.3 | 17.6 | 10.4 | |
| TeCoA⁴VLM=LLaVA 1.5, Vision encoder=TeCoA⁴2024.07 | 34.5 | 30.5 | 18.3 | 9.3 | |
| CLIPVLM=Open Flamingo, Vision encoder=CLIP2024.07 | 23.8 | 2.4 | 1.8 | 0 | |
| TeCoA²VLM=Open Flamingo, Vision encoder=TeCoA²2024.07 | 22.3 | 15.5 | 10.6 | 3.5 | |
| FARE²VLM=Open Flamingo, Vision encoder=FARE²2024.07 | 22.1 | 15.9 | 12.3 | 6.7 | |
| Sim-CLIP²VLM=Open Flamingo, Vision encoder=Sim-CLIP²2024.07 | 21.8 | 17.3 | 13.6 | 8.5 | |
| Sim-CLIP⁴VLM=Open Flamingo, Vision encoder=Sim-CLIP⁴2024.07 | 20 | 15.6 | 15.7 | 12.4 | |
| TeCoA⁴VLM=Open Flamingo, Vision encoder=TeCoA⁴2024.07 | 19.3 | 15.1 | 14.7 | 8.4 | |
| FARE⁴VLM=Open Flamingo, Vision encoder=FARE⁴2024.07 | 16.4 | 15.7 | 13.7 | 10.2 |