Image Captioning Evaluation on Flickr8K Expert (test)
56.4Kendall tau_cPolos
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PolosType=reference+image, Explainable=false2024.06 | 56.4 | — | — | |
| RefPAC-SType=Reference-based2023.03 | 55.9 | — | 55.5 | |
| RefPAC-SType=reference+image, Explainable=false2024.06 | 55.9 | — | — | |
| InfoMetIC+Type=reference-free, Explainable=false2024.06 | 55.5 | — | — | |
| CLIP + DN-refSetting=Reference-based2023.02 | 55.3 | — | — | |
| MID2022.05 | 54.9 | — | — | |
| MIDType=Hybrid2023.03 | 54.9 | — | — | |
| PAC-SType=Reference-free2023.03 | 54.3 | — | 53.9 | |
| CLIP + DNSetting=Reference-free2023.02 | 54.3 | — | — | |
| CLIP + DN*-refSetting=Reference-based2023.02 | 54.3 | — | — | |
| PAC-SType=reference-free, Explainable=false2024.06 | 54.3 | — | — | |
| CLIP + TTA + DNSetting=Reference-free2023.02 | 53.6 | — | — | |
| CLIP + TTA + DN*Setting=Reference-free2023.02 | 53.5 | — | — | |
| CLIP + DN*Setting=Reference-free2023.02 | 53.2 | — | — | |
| RefCLIP-S2022.05 | 53 | — | — | |
| RefCLIP-SType=Reference-based2023.03 | 53 | — | 52.6 | |
| CLIP-refSetting=Reference-based2023.02 | 53 | — | — | |
| RefCLIPScoreType=reference+image, Explainable=false2024.06 | 53 | — | — | |
| FLEURType=reference-free, Explainable=true2024.06 | 53 | — | — | |
| CLIP + TTASetting=Reference-free2023.02 | 51.9 | — | — | |
| RefFLEURType=reference+image, Explainable=true2024.06 | 51.9 | — | — | |
| CLIPSetting=Reference-free2023.02 | 51.4 | — | — | |
| CLIP-S2022.05 | 51.2 | — | — | |
| CLIP-SType=Reference-free2023.03 | 51.2 | — | 51.1 | |
| CLIPScoreType=reference-free, Explainable=false2024.06 | 51.2 | — | — | |
| ViLBERTScore-F2022.05 | 50.1 | — | — | |
| ViLBERTScoreType=Reference-based2023.03 | 50.1 | — | — | |
| ViLBERTScore-FSetting=Reference-based2023.02 | 50.1 | — | — | |
| VILBERTScore-FType=reference+image, Explainable=false2024.06 | 50.1 | — | — | |
| NUBIA2022.05 | 49.5 | — | — | |
| NUBIA*Setting=Reference-based2023.02 | 49.5 | — | — | |
| TIGEr2022.05 | 49.3 | — | — | |
| TIGErType=Reference-based2023.03 | 49.3 | — | — | |
| TIGErType=reference+image, Explainable=false2024.06 | 49.3 | — | — | |
| CLAIRType=reference-only, Explainable=true2024.06 | 48.3 | — | — | |
| UMICType=Reference-based2023.03 | 46.8 | — | — | |
| BERT-S++2022.05 | 46.7 | — | — | |
| BERT-S++Type=Reference-based2023.03 | 46.7 | — | — | |
| LEICcoefficient_type=tau_b2022.05 | 46.6 | — | — | |
| SPICE2022.05 | 44.9 | — | — | |
| SPICEType=Reference-based2023.03 | 44.9 | — | 51.7 | |
| SPICEType=reference-only, Explainable=false2024.06 | 44.9 | — | — | |
| CIDEr2022.05 | 43.9 | — | — | |
| CIDErType=Reference-based2023.03 | 43.9 | — | 43.6 | |
| CIDErSetting=Reference-based2023.02 | 43.9 | — | — | |
| CIDErType=reference-only, Explainable=false2024.06 | 43.9 | — | — | |
| TCL + DNSetting=Reference-free2023.02 | 42 | — | — | |
| METEOR2022.05 | 41.8 | — | — | |
| METEORType=Reference-based2023.03 | 41.8 | — | 41.5 | |
| METEORType=reference-only, Explainable=false2024.06 | 41.8 | — | — | |
| BERT-Sbackbone=RoBERTa-F2022.05 | 39.2 | — | — | |
| BERT-SType=Reference-based2023.03 | 39.2 | — | — | |
| BERTScoreType=reference-only, Explainable=false2024.06 | 39.2 | — | — | |
| TCL + DN*Setting=Reference-free2023.02 | 36 | — | — | |
| ALBEF + DNSetting=Reference-free2023.02 | 34.8 | — | — | |
| BLEU-12022.05 | 32.3 | — | — | |
| ROUGE-L2022.05 | 32.3 | — | — | |
| BLEU-1Type=Reference-based2023.03 | 32.3 | — | 32.2 | |
| ROUGEType=Reference-based2023.03 | 32.3 | — | 32.1 | |
| BLEU-1Setting=Reference-based2023.02 | 32.3 | — | — | |
| ROUGE-LType=reference-only, Explainable=false2024.06 | 32.3 | — | — | |
| TCLSetting=Reference-free2023.02 | 31 | — | — | |
| BLEU-42022.05 | 30.8 | — | — | |
| BLEU-4Type=Reference-based2023.03 | 30.8 | — | 30.6 | |
| BLEU-4Setting=Reference-based2023.02 | 30.8 | — | — | |
| BLEU-4Type=reference-only, Explainable=false2024.06 | 30.8 | — | — | |
| ALBEF + DN*Setting=Reference-free2023.02 | 29.2 | — | — | |
| ALBEFSetting=Reference-free2023.02 | 24.9 | — | — | |
| FACTUAL-T5(pre)Metric=SoftSPICE2023.05 | 0.5335 | 0.6952 | — | |
| SPICE-ParserMetric=SoftSPICE2023.05 | 0.5189 | 0.6811 | — | |
| CDP-T5Metric=SoftSPICE2023.05 | 0.4554 | 0.5964 | — | |
| FACTUAL-T5(pre)Metric=SPICE2023.05 | 0.4512 | 0.6078 | — | |
| SPICE-ParserMetric=SPICE2023.05 | 0.4477 | 0.6011 | — | |
| VG-T5Metric=SoftSPICE2023.05 | 0.3966 | 0.5305 | — | |
| VG-T5Metric=SPICE2023.05 | 0.3718 | 0.5194 | — | |
| CDP-T5Metric=SPICE2023.05 | 0.335 | 0.495 | — | |
| LEICType=Reference-based2023.03 | — | — | 46.6 |