Image Captioning on TextCaps (test)
164.3CIDErPaLI-3
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PaLI-3parameters=5B, pretrained dataset=10B-WebLI2024.11 | 164.3 | — | — | — | — | |
| PaLINumber of parameters=17B2022.09 | 160.4 | — | — | — | — | |
| PaLItype=State of the art w/ pipelines2022.10 | 160.4 | — | — | — | — | |
| PaLItype=specialized pipeline2023.05 | 160.4 | — | — | — | — | |
| PaLI-X-55BProtocol=fine-tuned2023.11 | 147 | — | — | — | — | |
| GIT2Number of parameters=5.1B2022.09 | 145 | — | — | — | — | |
| GIT2Pretraining=Image captioning, Pixel only=true2022.10 | 145 | — | — | — | — | |
| GIT2Train Data=12.9B, Protocol=fine-tuned2023.11 | 145 | — | — | — | — | |
| GIT22022.05 | 145 | 33.8 | 27 | 53 | 20.2 | |
| GIT22022.05 | 145 | — | — | — | — | |
| CogVLMTrain Data=1.5B, Protocol=fine-tuned2023.11 | 144.9 | — | — | — | — | |
| GITNumber of parameters=0.7B2022.09 | 138.2 | — | — | — | — | |
| GITTrain Data=0.8B, Protocol=fine-tuned2023.11 | 138.2 | — | — | — | — | |
| GIT2022.05 | 138.2 | — | — | — | — | |
| GIT2022.05 | 138.2 | 33.1 | 26.2 | 52.2 | 19.6 | |
| GIT2022.05 | 138.2 | — | — | — | — | |
| PaLI-17BTrain Data=1.6B, Protocol=fine-tuned2023.11 | 135.4 | — | — | — | — | |
| BLIP-2-OPT-2.7B + OursModel Params=3.8B, Trainable Params=14M, Pre-training Data=129M, Fine-tuning Data=650K2024.11 | 135.2 | — | — | — | — | |
| BLIP-2-OPT-2.7B + UReaderModel Params=3.8B, Trainable Params=8M, Pre-training Data=129M, Fine-tuning Data=650K2024.11 | 126.3 | — | — | — | — | |
| Human2022.05 | 125.5 | — | — | — | — | |
| Human2022.05 | 125.5 | 24.4 | 26.1 | 47 | 18.8 | |
| HumanProtocol=fine-tuned2023.11 | 125.1 | — | — | — | — | |
| HVFAbackbone=mPLUG-Owl, fine-tuning dataset=650K2024.11 | 123.1 | — | — | — | — | |
| mPLUG-Owl-7B + OursModel Params=7.2B, Trainable Params=96M, Pre-training Data=1.1B, Fine-tuning Data=650K2024.11 | 123.1 | — | — | — | — | |
| mPLUG-Owl-7B + UReaderModel Params=7.2B, Trainable Params=86M, Pre-training Data=1.1B, Fine-tuning Data=650K2024.11 | 118.4 | — | — | — | — | |
| TAPNote=Winner entry of the CVPR 2021 workshop challenge2022.05 | 109.7 | — | — | — | — | |
| TAPchallenge_winner=CVPR 2021 workshop challenge2022.05 | 109.7 | 22.9 | 22 | 46.5 | 14.6 | |
| Prior SOTA2022.05 | 109.7 | — | — | — | — | |
| TAP2022.05 | 103.2 | — | — | — | — | |
| TAP2022.05 | 103.2 | 21.9 | 21.8 | 45.6 | 14.6 | |
| Pix2Struct LargePretraining=Screenshot parsing, Pixel only=true2022.10 | 95.5 | — | — | — | — | |
| Pix2Structsize=large2023.05 | 95.5 | — | — | — | — | |
| Pix2Struct LargeModel Params=1.3B, Trainable Params=1.3B, Pre-training Data=80M, Fine-tuning Data=Task-specific2024.11 | 95.5 | — | — | — | — | |
| DUBLINresolution=fixed2023.05 | 92.8 | — | — | — | — | |
| DUBLINresolution=variable2023.05 | 92.8 | — | — | — | — | |
| Pix2Struct BasePretraining=Screenshot parsing, Pixel only=true2022.10 | 88 | — | — | — | — | |
| Pix2Struct BaseModel Params=282M, Trainable Params=282M, Pre-training Data=80M, Fine-tuning Data=Task-specific2024.11 | 88 | — | — | — | — | |
| Anc.-Cap.2022.05 | 87.4 | — | — | — | — | |
| Anc.-Cap.2022.05 | 87.4 | 20.7 | 20.7 | 44.6 | 13.4 | |
| M4C-Cap.Source=Sidorov et al. (2020)2022.05 | 81 | — | — | — | — | |
| M4C-Cap.source=Sidorov et al. (2020)2022.05 | 81 | 18.9 | 19.8 | 43.2 | 12.8 | |
| DonutPretraining=OCR, Pixel only=true2022.10 | 74.4 | — | — | — | — | |
| Donut2023.05 | 74.4 | — | — | — | — | |
| DonutModel Params=143M, Trainable Params=143M, Pre-training Data=13M, Fine-tuning Data=Task-specific2024.11 | 74.4 | — | — | — | — | |
| MonkeyModel Params=9.8B, Trainable Params=207M, Pre-training Data=1.4B+76.8M, Fine-tuning Data=1.44M2024.11 | 67.6 | — | — | — | — | |
| Qwen-VLModel Params=9.6B, Trainable Params=207M, Pre-training Data=1.4B+76.8M, Fine-tuning Data=Task-specific2024.11 | 63.8 | — | — | — | — | |
| VISEModel Scale=Qwen3-VL-32B-Instruct2026.06 | 43.06 | — | 30.34 | 46.54 | — | |
| VisionZero-RWModel Scale=Qwen3-VL-32B-Instruct2026.06 | 42.84 | — | 30.27 | 46.41 | — | |
| VisionZero (CLEVR)Model Scale=Qwen3-VL-32B-Instruct2026.06 | 42.31 | — | 29.92 | 46.21 | — | |
| iReasonerModel Scale=Qwen3-VL-32B-Instruct2026.06 | 42.19 | — | 29.94 | 46.19 | — | |
| VISEModel Scale=Qwen3-VL-2B-Instruct2026.06 | 41.86 | — | 26.2 | 41.81 | — | |
| VisPlayModel Scale=Qwen3-VL-32B-Instruct2026.06 | 41.63 | — | 29.61 | 45.96 | — | |
| BaseModel Scale=Qwen3-VL-32B-Instruct2026.06 | 41.25 | — | 29.68 | 45.85 | — | |
| VisionZero (Chart)Model Scale=Qwen3-VL-32B-Instruct2026.06 | 41.11 | — | 29.88 | 46.12 | — | |
| EvoLMMModel Scale=Qwen3-VL-32B-Instruct2026.06 | 41.02 | — | 29.71 | 45.93 | — | |
| VisionZero-RWModel Scale=Qwen3-VL-4B-Instruct2026.06 | 38.89 | — | 25.45 | 39.89 | — | |
| VISEModel Scale=Qwen3-VL-4B-Instruct2026.06 | 38.59 | — | 26.43 | 42.05 | — | |
| VISEModel Scale=Qwen3-VL-8B-Instruct2026.06 | 38.42 | — | 27.11 | 42.78 | — | |
| iReasonerModel Scale=Qwen3-VL-4B-Instruct2026.06 | 38.41 | — | 25.47 | 39.97 | — | |
| EvoLMMModel Scale=Qwen3-VL-4B-Instruct2026.06 | 38.29 | — | 25.34 | 39.82 | — | |
| VisionZero (CLEVR)Model Scale=Qwen3-VL-4B-Instruct2026.06 | 38.08 | — | 25.54 | 39.91 | — | |
| VisionZero-RWModel Scale=Qwen3-VL-8B-Instruct2026.06 | 38.03 | — | 26.96 | 42.63 | — | |
| iReasonerModel Scale=Qwen3-VL-8B-Instruct2026.06 | 37.48 | — | 26.63 | 42.21 | — | |
| VisionZero (CLEVR)Model Scale=Qwen3-VL-8B-Instruct2026.06 | 37.21 | — | 26.53 | 41.97 | — | |
| VisPlayModel Scale=Qwen3-VL-8B-Instruct2026.06 | 36.73 | — | 26.24 | 41.89 | — | |
| BaseModel Scale=Qwen3-VL-8B-Instruct2026.06 | 36.21 | — | 26.28 | 41.62 | — | |
| EvoLMMModel Scale=Qwen3-VL-8B-Instruct2026.06 | 36.05 | — | 26.31 | 41.73 | — | |
| VisionZero (Chart)Model Scale=Qwen3-VL-8B-Instruct2026.06 | 35.98 | — | 26.39 | 41.88 | — | |
| AoANetSource=Sidorov et al. (2020)2022.05 | 34.6 | — | — | — | — | |
| AoANetsource=Sidorov et al. (2020)2022.05 | 34.6 | 15.9 | 16.6 | 40.4 | 10.5 | |
| BaseModel Scale=Qwen3-VL-4B-Instruct2026.06 | 34.54 | — | 25.26 | 39.7 | — | |
| BUTDSource=Sidorov et al. (2020)2022.05 | 33.8 | — | — | — | — | |
| BUTDsource=Sidorov et al. (2020)2022.05 | 33.8 | 14.9 | 15.2 | 39.9 | 8.8 | |
| VisionZero (Chart)Model Scale=Qwen3-VL-4B-Instruct2026.06 | 32.95 | — | 25.47 | 39.91 | — | |
| VisPlayModel Scale=Qwen3-VL-4B-Instruct2026.06 | 31.34 | — | 25.03 | 39.42 | — | |
| VisionZero-RWModel Scale=Qwen3-VL-2B-Instruct2026.06 | 25.28 | — | 26.21 | 39.79 | — | |
| iReasonerModel Scale=Qwen3-VL-2B-Instruct2026.06 | 23.14 | — | 24.79 | 39.47 | — | |
| EvoLMMModel Scale=Qwen3-VL-2B-Instruct2026.06 | 23.04 | — | 24.68 | 39.39 | — | |
| VisionZero (CLEVR)Model Scale=Qwen3-VL-2B-Instruct2026.06 | 22.82 | — | 24.49 | 39.31 | — | |
| BaseModel Scale=Qwen3-VL-2B-Instruct2026.06 | 22.2 | — | 25.31 | 38.66 | — | |
| VisPlayModel Scale=Qwen3-VL-2B-Instruct2026.06 | 22.11 | — | 25.46 | 39.13 | — | |
| VisionZero (Chart)Model Scale=Qwen3-VL-2B-Instruct2026.06 | 21.41 | — | 25.96 | 39.47 | — |