Visual Question Answering on OKVQA (L-inf Robustness Evaluation)
60.3VQA Accuracy (Clean)Sim-CLIP²
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Sim-CLIP²VLM=LLaVA 1.5, Vision encoder=Sim-CLIP²2024.07 | 60.3 | 47.5 | 31.5 | 17.5 | |
| Sim-CLIP⁴VLM=LLaVA 1.5, Vision encoder=Sim-CLIP⁴2024.07 | 58.6 | 46.5 | 38.5 | 22.3 | |
| CLIPVLM=LLaVA 1.5, Vision encoder=CLIP2024.07 | 57.3 | 8.3 | 3 | 0 | |
| TeCoA²VLM=LLaVA 1.5, Vision encoder=TeCoA²2024.07 | 55.6 | 40.3 | 30.5 | 14.2 | |
| FARE²VLM=LLaVA 1.5, Vision encoder=FARE²2024.07 | 54.3 | 43.5 | 30.1 | 15.3 | |
| FARE⁴VLM=LLaVA 1.5, Vision encoder=FARE⁴2024.07 | 53.5 | 45 | 34.8 | 15.3 | |
| TeCoA⁴VLM=LLaVA 1.5, Vision encoder=TeCoA⁴2024.07 | 50.3 | 39 | 32.3 | 12.3 | |
| CLIPVLM=Open Flamingo, Vision encoder=CLIP2024.07 | 48.5 | 1.8 | 0 | 0 | |
| Sim-CLIP²VLM=Open Flamingo, Vision encoder=Sim-CLIP²2024.07 | 35.1 | 29.3 | 19.7 | 11.6 | |
| FARE²VLM=Open Flamingo, Vision encoder=FARE²2024.07 | 34.5 | 30.6 | 17.1 | 9.8 | |
| TeCoA²VLM=Open Flamingo, Vision encoder=TeCoA²2024.07 | 33.6 | 23.4 | 15.3 | 6.7 | |
| Sim-CLIP⁴VLM=Open Flamingo, Vision encoder=Sim-CLIP⁴2024.07 | 32 | 27.4 | 22 | 15.7 | |
| FARE⁴VLM=Open Flamingo, Vision encoder=FARE⁴2024.07 | 31.8 | 28 | 19.2 | 13.5 | |
| TeCoA⁴VLM=Open Flamingo, Vision encoder=TeCoA⁴2024.07 | 31 | 22.4 | 20.5 | 10.1 |