Image Captioning Evaluation on Flickr8k-CF
56.6Kendall-b Correlation (tau_b)ITIScore
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ITIScoreZero-shot=true2026.04 | 56.6 | 57.2 | |
| PAC-SZero-shot=true2026.04 | 53.9 | 54.3 | |
| MIDZero-shot=true2026.04 | 53.5 | 54.9 | |
| ChatGPT-5Zero-shot=true2026.04 | 53 | 55.8 | |
| SPICEZero-shot=true2026.04 | 51.7 | 44.9 | |
| CLIP-SZero-shot=true2026.04 | 51.1 | 51.2 | |
| Gemini-3.0-FlashZero-shot=true2026.04 | 50.8 | 52.5 | |
| ChatGPT-4oZero-shot=true2026.04 | 48 | 49.5 | |
| UMICZero-shot=true2026.04 | 47.3 | 46.8 | |
| Gemini-2.5-FlashZero-shot=true2026.04 | 45.5 | 46.8 | |
| CIDErZero-shot=true2026.04 | 43.6 | 43.9 | |
| InternVL3.5 (7B)Zero-shot=true2026.04 | 42.8 | 44.1 | |
| METEORZero-shot=true2026.04 | 41.5 | 41.8 | |
| RefDISCODE-INFF=true2025.12 | 40.5 | 20.9 | |
| RefDISCODE-LVFF=true2025.12 | 40.4 | 20.9 | |
| RIGELReference Requirement=Reference-based2026.06 | 40.4 | 20.9 | |
| RIGELReference Requirement=Reference-free2026.06 | 40.4 | 20.9 | |
| DISCODEBackbone=LLaVA, Reference Requirement=Reference-free2026.06 | 40.2 | 20.8 | |
| DISCODEBackbone=InternVL, Reference Requirement=Reference-free2026.06 | 40.1 | 20.8 | |
| BEiTScore-L/14Size (M)=674.69, GFLOPs=296.732026.05 | 39.6 | — | |
| EXPERTSize (M)=13350.84, GFLOPs=9317.472026.05 | 39.3 | — | |
| EXPERTReference Requirement=Reference-free2026.06 | 39.3 | — | |
| HarmonicEvalReference protocol=Reference-free2024.12 | 39.2 | — | |
| Qwen3-VL (7B)Zero-shot=true2026.04 | 39.1 | 40 | |
| GPT-FLEURReference protocol=Reference-free2024.12 | 39 | — | |
| RefPAC-S++ViT-LFF=false2025.12 | 38.8 | — | |
| RefFLEURFF=true2025.12 | 38.8 | — | |
| RefFLEURReference protocol=Reference-based2024.12 | 38.8 | — | |
| Ref-FLEURReference Requirement=Reference-based2026.06 | 38.8 | — | |
| G-VEvalReference protocol=Reference-free2024.12 | 38.7 | — | |
| G-VEvalReference Requirement=Reference-free2026.06 | 38.7 | 20.2 | |
| PearlReference-based=true, Backbone=ViT-B/32, Test time [hour]=0.0432025.12 | 38.6 | 20 | |
| FLEURReference protocol=Reference-free2024.12 | 38.6 | — | |
| FLEURSize (M)=13350.84, GFLOPs=9317.472026.05 | 38.6 | — | |
| PearlReference Requirement=Reference-based2026.06 | 38.6 | 20 | |
| FLEURReference Requirement=Reference-free2026.06 | 38.6 | — | |
| BERT-SZero-shot=true2026.04 | 38.5 | 39.2 | |
| PAC-S++/ViT-LSize (M)=430.58, GFLOPs=56.262026.05 | 38.5 | — | |
| BEiTScore-L/14 w/o Stage 1Size (M)=674.69, GFLOPs=296.732026.05 | 38.5 | — | |
| BEiTScore-B/32Size (M)=222.28, GFLOPs=83.852026.05 | 38.5 | — | |
| DENEB (ViT-L)FF=false2025.12 | 38.3 | — | |
| CLAIRFF=true2025.12 | 38.2 | — | |
| G-VEval-C (GPT-4o)FF=true2025.12 | 38.2 | 19.9 | |
| Ref-HICEScoreReference Requirement=Reference-based2026.06 | 38.2 | 19.8 | |
| CLAIRReference Requirement=Reference-based2026.06 | 38.2 | 17 | |
| G-VEvalReference Requirement=Reference-based2026.06 | 38.2 | 19.9 | |
| DENEBReference Requirement=Reference-based2026.06 | 38 | 19.6 | |
| RefPAC-S++ViT-B2025.12 | 37.9 | 19.6 | |
| RefPAC-S++Reference Requirement=Reference-based2026.06 | 37.9 | 19.6 | |
| PearlReference-based=false, Backbone=ViT-B/32, Test time [hour]=0.0432025.12 | 37.8 | 19.5 | |
| PolosFF=false2025.12 | 37.8 | — | |
| PolosReference protocol=Reference-based2024.12 | 37.8 | — | |
| BEiTScore-B/32 w/o Stage 1Size (M)=222.28, GFLOPs=83.852026.05 | 37.8 | — | |
| PolosReference Requirement=Reference-based2026.06 | 37.8 | 19.5 | |
| PearlReference Requirement=Reference-free2026.06 | 37.8 | 19.5 | |
| RefPAC-S2025.12 | 37.6 | 19.5 | |
| RefPAC-SReference Requirement=Reference-based2026.06 | 37.6 | 19.5 | |
| MIDFF=true2025.12 | 37.3 | — | |
| HICEScoreReference Requirement=Reference-free2026.06 | 37.2 | 19.2 | |
| PAC-S/ViT-LSize (M)=427.62, GFLOPs=56.262026.05 | 37 | — | |
| PAC-S++/ViT-BSize (M)=155.42, GFLOPs=4.882026.05 | 37 | — | |
| PAC-S++Reference Requirement=Reference-free2026.06 | 37 | 19.1 | |
| BLIP2-ScoreReference Requirement=Reference-free2026.06 | 36.7 | 19 | |
| CLIP-StuneType=w/o ref2023.05 | 36.6 | — | |
| InfoMetIC+Type=w/o ref2023.05 | 36.6 | — | |
| InfoMetIC+Reference protocol=Reference-free2024.12 | 36.6 | — | |
| VQAScore (CLIP-FlanT5-XXL)Size (M)=11459.59, GFLOPs=3742.12026.05 | 36.5 | — | |
| CLIP-S/ViT-LSize (M)=427.62, GFLOPs=84.362026.05 | 36.5 | — | |
| RefCLIP-SType=w/ ref2023.05 | 36.4 | — | |
| RefCLIP-SFF=true2025.12 | 36.4 | 18.8 | |
| RefCLIP-ScoreReference protocol=Reference-based2024.12 | 36.4 | — | |
| RefCLIP-SReference Requirement=Reference-based2026.06 | 36.4 | 18.8 | |
| InfoMetICType=w/o ref2023.05 | 36.3 | — | |
| BRIDGEReference-based=false2025.12 | 36.3 | 19 | |
| InfoMetICReference protocol=Reference-free2024.12 | 36.3 | — | |
| BRIDGEReference Requirement=Reference-free2026.06 | 36.3 | 19 | |
| PAC-S/ViT-BSize (M)=151.28, GFLOPs=4.882026.05 | 36 | — | |
| PAC-SReference Requirement=Reference-free2026.06 | 36 | 18.6 | |
| LongCLIP-S/ViT-LSize (M)=427.75, GFLOPs=98.92026.05 | 35.9 | — | |
| FAIEr-4Type=w/ ref2023.05 | 35.4 | — | |
| FAIEr-4Reference protocol=Reference-based2024.12 | 35.4 | — | |
| CLIP-S/ViT-BSize (M)=151.28, GFLOPs=7.282026.05 | 35.3 | — | |
| LongCLIP-S/ViT-BSize (M)=149.84, GFLOPs=26.242026.05 | 35.2 | — | |
| CLIP-SType=w/o ref2023.05 | 34.4 | — | |
| CLIP-SReference protocol=Reference-free2024.12 | 34.4 | — | |
| CLIP-SReference Requirement=Reference-free2026.06 | 34.4 | 17.7 | |
| FAIEr-rType=w/o ref2023.05 | 32.4 | — | |
| FAIEr-rReference protocol=Reference-free2024.12 | 32.4 | — | |
| BLEUZero-shot=true2026.04 | 32.2 | 32.3 | |
| ROUGEZero-shot=true2026.04 | 32.1 | 32.3 | |
| UMICReference protocol=Reference-free2024.12 | 30.1 | — | |
| LEIC2025.12 | 29.5 | — | |
| InfoCLIPtuneType=w/o ref2023.05 | 27.7 | — | |
| SPARCSReference-based=true, Test time [hour]=< 0.012025.12 | 25.2 | 13 | |
| CIDErType=w/ ref2023.05 | 24.6 | — | |
| CIDErFF=false2025.12 | 24.6 | 12.7 | |
| CIDErReference protocol=Reference-based2024.12 | 24.6 | — | |
| CIDErReference Requirement=Reference-based2026.06 | 24.6 | 12.7 | |
| SPICEType=w/ ref2023.05 | 24.4 | — | |
| SPICEFF=false2025.12 | 24.4 | 12 |