Image Captioning Evaluation on Nebula
57.5Kendall tau_cRIGEL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RIGELReference Requirement=Reference-based2026.06 | 57.5 | 60.6 | |
| BEiTScore-L/14Size (M)=674.69, GFLOPs=296.732026.05 | 56.7 | — | |
| BEiTScore-L/14 w/o Stage 1Size (M)=674.69, GFLOPs=296.732026.05 | 56.3 | — | |
| PearlReference-based=true, Backbone=ViT-B/32, Test time [hour]=0.0432025.12 | 55.4 | 58.4 | |
| PearlReference-based=true, Backbone=ViT-B/322025.12 | 55.4 | 58.4 | |
| PearlReference Requirement=Reference-based2026.06 | 55.4 | 58.4 | |
| BEiTScore-B/32Size (M)=222.28, GFLOPs=83.852026.05 | 55.3 | — | |
| DENEBReference-based=true, Backbone=ViT-B/322025.12 | 55.1 | 58.1 | |
| DENEBReference Requirement=Reference-based2026.06 | 55.1 | 58.1 | |
| RIGELReference Requirement=Reference-free2026.06 | 55.1 | 58.1 | |
| PolosReference-based=true2025.12 | 55 | 58 | |
| PolosReference Requirement=Reference-based2026.06 | 55 | 58 | |
| EXPERTReference-based=false2025.12 | 54.9 | — | |
| EXPERTSize (M)=13350.84, GFLOPs=9317.472026.05 | 54.9 | — | |
| BEiTScore-B/32 w/o Stage 1Size (M)=222.28, GFLOPs=83.852026.05 | 54.6 | — | |
| PommsEvaluation Framework=Pomms, Ensemble Composition=(v) + Qwen2.5-VL-7B2026.04 | 54.1 | 57 | |
| PommsEvaluation Framework=Pomms, Ensemble Composition=(vi) + Phi-3.5-Vision2026.04 | 54.1 | 57 | |
| PommsEvaluation Framework=Pomms, Ensemble Composition=(iii) + LLaVA-OneVision-7B2026.04 | 53.7 | 56.6 | |
| PommsEvaluation Framework=Pomms, Ensemble Composition=(ii) + Eagle2-9B2026.04 | 53.6 | 56.6 | |
| PommsEvaluation Framework=Pomms, Ensemble Composition=(i) GPT-4o and InternVL2.5-8B2026.04 | 53.5 | 56.4 | |
| PommsEvaluation Framework=Pomms, Ensemble Composition=(iv) + DeepSeek-VL22026.04 | 53.5 | 56.4 | |
| GPT-4oEvaluation Framework=G-VEval2026.04 | 53.2 | 56.1 | |
| PearlReference-based=false, Backbone=ViT-B/32, Test time [hour]=0.0432025.12 | 53 | 55.9 | |
| PearlReference-based=false, Backbone=ViT-B/322025.12 | 53 | 55.9 | |
| PearlReference Requirement=Reference-free2026.06 | 53 | 55.9 | |
| FLEURSize (M)=13350.84, GFLOPs=9317.472026.05 | 52.7 | — | |
| Qwen2.5-VL-7BEvaluation Framework=G-VEval2026.04 | 52.4 | 55.3 | |
| RefPAC-SReference-based=true, Backbone=ViT-B/322025.12 | 51.9 | 54.7 | |
| RefPAC-SReference Requirement=Reference-based2026.06 | 51.9 | 54.7 | |
| InternVL2.5-8BEvaluation Framework=G-VEval2026.04 | 51.3 | 54.1 | |
| RefCLIP-SReference-based=true, Backbone=ViT-B/322025.12 | 50.8 | 53.6 | |
| RefCLIP-SReference Requirement=Reference-based2026.06 | 50.8 | 53.6 | |
| BLIP2ScoreReference-based=false2025.12 | 50.7 | 53 | |
| BLIP2-ScoreReference Requirement=Reference-free2026.06 | 50.7 | 53 | |
| RefPAC-S++Reference-based=true, Backbone=ViT-B/322025.12 | 50.6 | 53.3 | |
| RefPAC-S++Reference Requirement=Reference-based2026.06 | 50.6 | 53.3 | |
| SPARCSReference-based=true, Test time [hour]=< 0.012025.12 | 50 | 53.9 | |
| VQAScore (CLIP-FlanT5-XXL)Size (M)=11459.59, GFLOPs=3742.12026.05 | 49.4 | — | |
| LongCLIP-S/ViT-BSize (M)=149.84, GFLOPs=26.242026.05 | 48.9 | — | |
| PAC-S++/ViT-LSize (M)=430.58, GFLOPs=56.262026.05 | 48.9 | — | |
| CIDErReference-based=true2025.12 | 48.8 | 51.5 | |
| CLIP-S/ViT-LSize (M)=427.62, GFLOPs=84.362026.05 | 48.8 | — | |
| CIDErReference Requirement=Reference-based2026.06 | 48.8 | 51.5 | |
| LongCLIP-S/ViT-LSize (M)=427.75, GFLOPs=98.92026.05 | 48.6 | — | |
| PAC-SReference-based=false, Backbone=ViT-B/322025.12 | 48.3 | 51 | |
| PAC-SReference Requirement=Reference-free2026.06 | 48.3 | 51 | |
| CLIP-SReference-based=false2025.12 | 47.9 | 50.5 | |
| PAC-S++Reference-based=false, Backbone=ViT-B/322025.12 | 47.9 | 50.5 | |
| PAC-S/ViT-LSize (M)=427.62, GFLOPs=56.262026.05 | 47.9 | — | |
| CLIP-SReference Requirement=Reference-free2026.06 | 47.9 | 50.5 | |
| PAC-S++Reference Requirement=Reference-free2026.06 | 47.9 | 50.5 | |
| CLIP-S/ViT-BSize (M)=151.28, GFLOPs=7.282026.05 | 47.8 | — | |
| METEORReference-based=true2025.12 | 47.6 | 50.2 | |
| METEORReference Requirement=Reference-based2026.06 | 47.6 | 50.2 | |
| SPICEReference-based=true2025.12 | 47.4 | 51.5 | |
| SPICEReference Requirement=Reference-based2026.06 | 47.4 | 51.5 | |
| PAC-S/ViT-BSize (M)=151.28, GFLOPs=4.882026.05 | 47.2 | — | |
| BERTScoreReference-based=true, Test time [hour]=0.0402025.12 | 47.1 | 47.5 | |
| BERTScoreReference Requirement=Reference-based2026.06 | 47.1 | 47.5 | |
| PAC-S++/ViT-BSize (M)=155.42, GFLOPs=4.882026.05 | 46.9 | — | |
| BARTScoreReference-based=true, Test time [hour]=0.682025.12 | 45 | 47.4 | |
| BLEUReference-based=true2025.12 | 44.1 | 46.5 | |
| BLEUReference Requirement=Reference-based2026.06 | 44.1 | 46.5 | |
| ROUGEReference-based=true2025.12 | 43.4 | 45.8 | |
| ROUGEReference Requirement=Reference-based2026.06 | 43.4 | 45.8 | |
| SPECSSize (M)=151.49, GFLOPs=9.22026.05 | 34.4 | — |