Image Captioning on COCO (L_inf Perturbed CIDEr)
125.6CIDEr (Clean)Sim-CLIP²
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Sim-CLIP²VLM=LLaVA 1.5, Vision encoder=Sim-CLIP²2024.07 | 125.6 | 109.4 | 93.5 | 45.6 | |
| FARE²VLM=LLaVA 1.5, Vision encoder=FARE²2024.07 | 123.5 | 105.2 | 86.4 | 39.4 | |
| Sim-CLIP⁴VLM=LLaVA 1.5, Vision encoder=Sim-CLIP⁴2024.07 | 122.3 | 108.1 | 90.3 | 44.3 | |
| CLIPVLM=LLaVA 1.5, Vision encoder=CLIP2024.07 | 121.9 | 21.8 | 13.5 | 2.4 | |
| FARE⁴VLM=LLaVA 1.5, Vision encoder=FARE⁴2024.07 | 119.4 | 100.5 | 83.5 | 41.6 | |
| TeCoA²VLM=LLaVA 1.5, Vision encoder=TeCoA²2024.07 | 115.6 | 98.3 | 73.5 | 38.1 | |
| TeCoA⁴VLM=LLaVA 1.5, Vision encoder=TeCoA⁴2024.07 | 110.3 | 95.5 | 75.6 | 35.3 | |
| Sim-CLIP²VLM=Open Flamingo, Vision encoder=Sim-CLIP²2024.07 | 85.6 | 72.8 | 58.4 | 19.3 | |
| FARE²VLM=Open Flamingo, Vision encoder=FARE²2024.07 | 84.3 | 68.2 | 53.5 | 18.4 | |
| Sim-CLIP⁴VLM=Open Flamingo, Vision encoder=Sim-CLIP⁴2024.07 | 81.6 | 71.5 | 60.5 | 26 | |
| FARE⁴VLM=Open Flamingo, Vision encoder=FARE⁴2024.07 | 81.4 | 67.9 | 56.1 | 23.3 | |
| CLIPVLM=Open Flamingo, Vision encoder=CLIP2024.07 | 80.5 | 7.82 | 5.6 | 2.4 | |
| TeCoA²VLM=Open Flamingo, Vision encoder=TeCoA²2024.07 | 74.5 | 59.7 | 40.3 | 10.3 | |
| TeCoA⁴VLM=Open Flamingo, Vision encoder=TeCoA⁴2024.07 | 71 | 58.3 | 50.3 | 15.8 |