Image Captioning on Flickr30k (Adversarial CIDEr Robustness)
80.5CIDEr (Clean)Sim-CLIP²
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Sim-CLIP²VLM=LLaVA 1.5, Vision encoder=Sim-CLIP²2024.07 | 80.5 | 73.1 | 63.8 | 29.8 | |
| CLIPVLM=LLaVA 1.5, Vision encoder=CLIP2024.07 | 79 | 15.3 | 10 | 3.4 | |
| Sim-CLIP⁴VLM=LLaVA 1.5, Vision encoder=Sim-CLIP⁴2024.07 | 79 | 72.3 | 61.3 | 32.5 | |
| FARE²VLM=LLaVA 1.5, Vision encoder=FARE²2024.07 | 78.9 | 70.3 | 60.5 | 25.1 | |
| FARE⁴VLM=LLaVA 1.5, Vision encoder=FARE⁴2024.07 | 76.3 | 70.3 | 56.5 | 29.5 | |
| TeCoA²VLM=LLaVA 1.5, Vision encoder=TeCoA²2024.07 | 75.6 | 65.3 | 50.5 | 29.4 | |
| TeCoA⁴VLM=LLaVA 1.5, Vision encoder=TeCoA⁴2024.07 | 71.8 | 62.5 | 51 | 27 | |
| CLIPVLM=Open Flamingo, Vision encoder=CLIP2024.07 | 61 | 6.4 | 3.8 | 1.4 | |
| Sim-CLIP²VLM=Open Flamingo, Vision encoder=Sim-CLIP²2024.07 | 56.3 | 50.5 | 35.1 | 16.4 | |
| Sim-CLIP⁴VLM=Open Flamingo, Vision encoder=Sim-CLIP⁴2024.07 | 54.5 | 48 | 39.2 | 20.4 | |
| FARE²VLM=Open Flamingo, Vision encoder=FARE²2024.07 | 53.1 | 48.6 | 34.3 | 12.3 | |
| FARE⁴VLM=Open Flamingo, Vision encoder=FARE⁴2024.07 | 51.8 | 47.3 | 37.6 | 20.1 | |
| TeCoA²VLM=Open Flamingo, Vision encoder=TeCoA²2024.07 | 48.2 | 37.3 | 27.4 | 10.3 | |
| TeCoA⁴VLM=Open Flamingo, Vision encoder=TeCoA⁴2024.07 | 45.6 | 36.2 | 32.9 | 18 |