Image Captioning on Flickr30k (BLEU-4, METEOR, CIDEr, SPICE)
84.5CIDErInstructBLIP
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| InstructBLIPzero-shot=true, backbone=FlanT5-XL2023.05 | 84.5 | — | — | — | |
| InstructBLIPzero-shot=true, backbone=FlanT5-XXL2023.05 | 83.5 | — | — | — | |
| InstructBLIPzero-shot=true, backbone=Vicuna-13B2023.05 | 82.8 | — | — | — | |
| InstructBLIPzero-shot=true, backbone=Vicuna-7B2023.05 | 82.4 | — | — | — | |
| CleanModel=LLaVa1.5-7B2026.02 | 77.5 | — | — | — | |
| CleanModel=LLaVa1.5-13B2026.02 | 77.1 | — | — | — | |
| BLIP-2zero-shot=true, backbone=FlanT5-XL2023.05 | 76.1 | — | — | — | |
| BLIP-2zero-shot=true, backbone=Vicuna-7B2023.05 | 74.9 | — | — | — | |
| BLIP-2zero-shot=true, backbone=FlanT5-XXL2023.05 | 73.7 | — | — | — | |
| Uni-PerceiverTuning protocol=fine-tuning, Percentage of data used in tuning=100%2021.12 | 72.7 | 30.1 | 24.5 | 18.2 | |
| BLIP-2zero-shot=true, backbone=Vicuna-13B2023.05 | 71.6 | — | — | — | |
| Unified VLP2021.12 | 67.4 | 30.1 | 23 | 17 | |
| Flamingozero-shot=true, backbone=80B2023.05 | 67.2 | — | — | — | |
| Uni-PerceiverTuning protocol=prompt-tuning, Percentage of data used in tuning=10%2021.12 | 61.7 | 28.8 | 22.1 | 16.8 | |
| Flamingozero-shot=true, backbone=9B2023.05 | 61.5 | — | — | — | |
| Flamingozero-shot=true, backbone=3B2023.05 | 60.6 | — | — | — | |
| CleanModel=OF-9B2026.02 | 60.1 | — | — | — | |
| Uni-PerceiverTuning protocol=prompt-tuning, Percentage of data used in tuning=1%2021.12 | 59.1 | 28.1 | 21.6 | 15.6 | |
| Uni-PerceiverTuning protocol=fine-tuning, Percentage of data used in tuning=10%2021.12 | 52.1 | 26.9 | 21.6 | 14.5 | |
| Uni-PerceiverTuning protocol=fine-tuning, Percentage of data used in tuning=1%2021.12 | 45.3 | 18.9 | 19.7 | 14.3 | |
| MIX.AttackModel=LLaVa1.5-7B, Perturbation (ε)=2/2552026.02 | 42.1 | — | — | — | |
| Uni-PerceiverTuning protocol=none2021.12 | 41.2 | 17 | 16.2 | 11.2 | |
| MIX.AttackModel=LLaVa1.5-13B, Perturbation (ε)=2/2552026.02 | 41 | — | — | — | |
| VT-AttackModel=LLaVa1.5-13B, Perturbation (ε)=2/2552026.02 | 40.7 | — | — | — | |
| VT-AttackModel=LLaVa1.5-7B, Perturbation (ε)=2/2552026.02 | 40 | — | — | — | |
| VT-AttackModel=OF-9B, Perturbation (ε)=2/2552026.02 | 38.6 | — | — | — | |
| FEWVLMlargeModel size=740M, zero-shot=true2021.10 | 36.5 | — | — | 10.7 | |
| MIX.AttackModel=LLaVa1.5-7B, Perturbation (ε)=4/2552026.02 | 35.9 | — | — | — | |
| MIX.AttackModel=OF-9B, Perturbation (ε)=2/2552026.02 | 33.7 | — | — | — | |
| MIX.AttackModel=LLaVa1.5-13B, Perturbation (ε)=4/2552026.02 | 32.2 | — | — | — | |
| FEWVLMbaseModel size=224M, zero-shot=true2021.10 | 31 | — | — | 10 | |
| AttackVLM-iiModel=LLaVa1.5-7B, Perturbation (ε)=2/2552026.02 | 30.2 | — | — | — | |
| AttackVLM-iiModel=LLaVa1.5-13B, Perturbation (ε)=2/2552026.02 | 27.5 | — | — | — | |
| VT-AttackModel=LLaVa1.5-13B, Perturbation (ε)=4/2552026.02 | 25.2 | — | — | — | |
| Unified VLPModel size=122M, zero-shot=true2021.10 | 24.9 | — | — | 7.2 | |
| VT-AttackModel=OF-9B, Perturbation (ε)=4/2552026.02 | 23 | — | — | — | |
| AttackVLM-iiModel=LLaVa1.5-7B, Perturbation (ε)=4/2552026.02 | 20.3 | — | — | — | |
| AttackVLM-iiModel=OF-9B, Perturbation (ε)=2/2552026.02 | 20.1 | — | — | — | |
| VT-AttackModel=LLaVa1.5-7B, Perturbation (ε)=4/2552026.02 | 19.9 | — | — | — | |
| MIX.AttackModel=OF-9B, Perturbation (ε)=4/2552026.02 | 18 | — | — | — | |
| AttackVLM-iiModel=LLaVa1.5-13B, Perturbation (ε)=4/2552026.02 | 17.9 | — | — | — | |
| VEAttackModel=LLaVa1.5-7B, Perturbation (ε)=2/2552026.02 | 10.7 | — | — | — | |
| AttackVLM-iiModel=OF-9B, Perturbation (ε)=4/2552026.02 | 9.3 | — | — | — | |
| VEAttackModel=LLaVa1.5-13B, Perturbation (ε)=2/2552026.02 | 9.1 | — | — | — | |
| VEAttackModel=OF-9B, Perturbation (ε)=2/2552026.02 | 8.7 | — | — | — | |
| VEAttackModel=LLaVa1.5-7B, Perturbation (ε)=4/2552026.02 | 6.3 | — | — | — | |
| VEAttackModel=LLaVa1.5-13B, Perturbation (ε)=4/2552026.02 | 5.7 | — | — | — | |
| PA-AttackModel=OF-9B, Perturbation (ε)=2/2552026.02 | 5.4 | — | — | — | |
| PA-AttackModel=LLaVa1.5-7B, Perturbation (ε)=2/2552026.02 | 4.7 | — | — | — | |
| PA-AttackModel=LLaVa1.5-13B, Perturbation (ε)=2/2552026.02 | 4.1 | — | — | — | |
| PA-AttackModel=OF-9B, Perturbation (ε)=4/2552026.02 | 3.7 | — | — | — | |
| PA-AttackModel=LLaVa1.5-7B, Perturbation (ε)=4/2552026.02 | 3.3 | — | — | — | |
| VEAttackModel=OF-9B, Perturbation (ε)=4/2552026.02 | 3.2 | — | — | — | |
| VL-T5no-vqaModel size=224M, zero-shot=true2021.10 | 2.6 | — | — | 2 | |
| PA-AttackModel=LLaVa1.5-13B, Perturbation (ε)=4/2552026.02 | 2 | — | — | — |