Image Captioning on NoCaps (Domain Breakdown Evaluation)
111.3CIDEr (in-domain)BLIP
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| BLIP# Img-Text Pairs=14M2022.09 | 111.3 | 15.1 | 104.5 | 14.4 | 102.4 | 13.7 | 105.1 | 14.4 | |
| OmniVL# Img-Text Pairs=14M*2022.09 | 104.6 | 15 | 108.3 | 14.9 | 106.3 | 14.2 | 107.5 | 14.7 | |
| LEMON# Img-Text Pairs=12M2022.09 | 104.5 | 14.6 | 100.7 | 14 | 96.7 | 12.4 | 100.4 | 13.8 | |
| VinVL# Img-Text Pairs=5.7M2022.09 | 103.1 | 14.2 | 96.1 | 13.8 | 88.3 | 12.1 | 95.5 | 13.5 | |
| sample-kk=4, Datastore=+Web2024.06 | 94.8 | — | 89.5 | — | 93.1 | — | — | — | |
| c-sample-kk=4, Datastore=+Web2024.06 | 94.5 | — | 90 | — | 93.3 | — | — | — | |
| top-kk=4, Datastore=+Web2024.06 | 94.2 | — | 89.4 | — | 91.2 | — | — | — | |
| sample-kk=3, Datastore=+Web2024.06 | 92.6 | — | 88 | — | 90 | — | — | — | |
| Enc-Dec# Img-Text Pairs=15M2022.09 | 92.6 | 12.5 | 88.3 | 12.1 | 94.5 | 11.9 | 90.2 | 12.1 | |
| top-kk=3, Datastore=+Web2024.06 | 91.7 | — | 88.3 | — | 89.9 | — | — | — | |
| sample-kk=2, Datastore=+Web2024.06 | 90.6 | — | 85.3 | — | 86.7 | — | — | — | |
| top-kk=2, Datastore=+Web2024.06 | 90.5 | — | 86.2 | — | 89.5 | — | — | — | |
| c-sample-kk=4, Datastore=Standard2024.06 | 90.1 | — | 81.3 | — | 71.5 | — | — | — | |
| sample-kk=4, Datastore=Standard2024.06 | 89.7 | — | 80.9 | — | 71.1 | — | — | — | |
| sample-kk=3, Datastore=Standard2024.06 | 88.7 | — | 80.3 | — | 69.4 | — | — | — | |
| sample-kk=2, Datastore=Standard2024.06 | 87.8 | — | 77.4 | — | 67.6 | — | — | — | |
| sample-kk=1, Datastore=+Web2024.06 | 87.8 | — | 81.2 | — | 77.5 | — | — | — | |
| top-kk=3, Datastore=Standard2024.06 | 87.4 | — | 79.6 | — | 68.3 | — | — | — | |
| top-kk=4, Datastore=Standard2024.06 | 87.4 | — | 79.6 | — | 68.3 | — | — | — | |
| sample-kk=1, Datastore=Standard2024.06 | 87 | — | 75.7 | — | 63.6 | — | — | — | |
| top-kk=1, Datastore=Standard2024.06 | 85 | — | 74.3 | — | 62.3 | — | — | — | |
| top-kk=2, Datastore=Standard2024.06 | 85 | — | 74.3 | — | 62.3 | — | — | — | |
| top-kk=1, Datastore=+Web2024.06 | 84.1 | — | 80.7 | — | 81.5 | — | — | — | |
| TipCapEncoder=ViT-L/14, Decoder=GPT2-large2025.12 | 80.2 | — | 62.3 | — | 39.6 | — | 60.3 | — | |
| TOMCap (large)Encoder=ViT-L/16, Decoder=GPT2-large2025.12 | 77.1 | — | 76.6 | — | 75.9 | — | 76.4 | — | |
| TOMCapEncoder=ViT-L/16, Decoder=GPT2-base2025.12 | 76.3 | — | 75.5 | — | 75.7 | — | 76.2 | — | |
| IFCapEncoder=ViT-B/32, Decoder=GPT2-base2025.12 | 70.1 | — | 72.5 | — | 72.1 | — | 74 | — | |
| CapDec+RLCF-SEncoder=ViT-B/16, Decoder=OPT-125M2025.12 | 68.3 | — | 58.5 | — | 35.3 | — | — | — | |
| ViECap+ToCaEncoder=ViT-B/32, Decoder=GPT2-base2025.12 | 64.6 | — | 69.1 | — | 70.5 | — | 70.9 | — | |
| EntroCapEncoder=ViT-B/32, Decoder=GPT2-base2025.12 | 62.5 | — | 64.5 | — | 67.5 | — | 67 | — | |
| ViECapEncoder=ViT-B/32, Decoder=GPT2-base2025.12 | 61.1 | — | 64.3 | — | 65 | — | 66.2 | — | |
| CapDecEncoder=ResNet50, Decoder=GPT2-large2025.12 | 60.1 | — | 50.2 | — | 28.7 | — | 45.9 | — | |
| ICSDEncoder=ViT-B/32, Decoder=BERT-base2025.12 | 42.9 | — | 44.3 | — | 35.6 | — | 42.7 | — | |
| DeCapEncoder=ViT-B/32, Decoder=Transformer2025.12 | 41.9 | — | 41.7 | — | 46.2 | — | 42.7 | — | |
| MeaCapToTEncoder=ViT-B/32, Decoder=CBART2025.12 | 38.5 | — | 43.6 | — | 50 | — | 45.1 | — | |
| MeaCapTFEncoder=ViT-B/32, Decoder=CBART2025.12 | 35.3 | — | 39 | — | 45.1 | — | 40.2 | — | |
| SIMVLM# Img-Text Pairs=1.8B2022.09 | — | — | — | — | — | — | 94.8 | 13.1 |