Video Captioning on MSVD (test)
189.4CIDErAdaCM²
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| AdaCM²2024.11 | 189.4 | — | 51.4 | — | — | |
| GiT2Pre-training Data Scale (M)=129002023.05 | 185.4 | 82.2 | 52.3 | 88.7 | — | |
| GIT2Parameters=12.9B2023.06 | 185.4 | 82.2 | — | — | — | |
| HierarQLLM Finetuning=true2025.03 | 183.1 | — | — | — | — | |
| HierarQLLM Finetuning=false2025.03 | 182.6 | — | — | — | — | |
| GiTPre-training Data Scale (M)=8002023.05 | 180.2 | 79.5 | 51.1 | 87.3 | — | |
| GITParameters=1.7B2023.06 | 180.2 | 79.5 | — | — | — | |
| GIT2024.11 | 180.2 | — | 51.1 | — | — | |
| GIT2025.03 | 180.2 | — | — | — | — | |
| GIT2022.05 | 180.2 | 79.5 | — | — | — | |
| VLAB_GPre-training Data Scale (M)=262023.05 | 179.8 | 79.3 | 51.2 | 87.9 | — | |
| MA-LMM2024.11 | 179.1 | — | 51 | — | — | |
| MA-LMM2025.03 | 179.1 | — | — | — | — | |
| VALOR-LParameters=433.5M, Additional modalities=true2023.06 | 178.5 | 80.7 | — | — | — | |
| COSA (1.2B)Parameters=415M2023.06 | 178.5 | 76.5 | — | — | — | |
| VideoLLaMA2024.11 | 175.3 | — | 49.8 | — | — | |
| VLAB_LPre-training Data Scale (M)=262023.05 | 174.1 | 78.7 | 50.3 | 86.9 | — | |
| COSA-LParameters=417M2023.06 | 169.1 | 74 | — | — | — | |
| mPLUG-22024.11 | 165.8 | — | 48.4 | — | — | |
| mPLUG-22025.03 | 165.8 | — | — | — | — | |
| EVLGen-videomode=video2023.10 | 158.2 | — | 46.8 | 83.1 | 68.4 | |
| HowToCaption2025.03 | 154.2 | — | — | — | — | |
| COSA-BParameters=17M2023.06 | 150.9 | 68.7 | — | — | — | |
| VideoCoCa (open)adaptation=Open-CoCa framework2023.10 | 150.9 | — | 45.3 | 81.9 | 67.7 | |
| LAVENDERPre-training Data Scale (M)=302023.05 | 150.7 | — | — | — | — | |
| LAVENDERParameters=30M2023.06 | 150.7 | — | — | — | — | |
| SwinBERT2025.03 | 149.4 | — | — | — | — | |
| HiTeAPre-training Data Scale (M)=172023.05 | 146.9 | 71 | 45.3 | 81.4 | — | |
| Vid2SeqTrained Parameters=314M, Pretraining Data=YT-Temporal-1B2023.02 | 146.2 | — | 45.3 | — | — | |
| COSA-BParameters=5M2023.06 | 146 | 66.9 | — | — | — | |
| HiTeAParameters=5M2023.06 | 145.1 | — | — | — | — | |
| Vid2SeqTrained Parameters=314M, Pretraining Data=HowTo100M2023.02 | 140.6 | — | 44.5 | — | — | |
| VIOLETV2Parameters=5M2023.06 | 139.2 | — | — | — | — | |
| CoCap (ViT-L/14)Decoding=false, E2E=true, 3D Action=CLIP2023.09 | 121.5 | 60.1 | 41.4 | 78.2 | — | |
| Video-LLaMA2023.10 | 121.2 | — | 40.3 | 77.8 | 61.6 | |
| SWINBERTframes=642021.11 | 120.6 | — | — | — | — | |
| SWINBERT2D Appearance=VidSwin2021.11 | 120.6 | 58.2 | 41.3 | 77.5 | — | |
| SwinBERTTrained Parameters=229M, Pretraining Data=Ø2023.02 | 120.6 | — | 41.3 | — | — | |
| SwinBERT2023.05 | 120.6 | 58.2 | 41.3 | 77.5 | — | |
| SwinBERTDecoding=true, E2E=true, 3D Action=VidSwin2023.09 | 120.6 | 58.2 | 41.3 | 77.5 | — | |
| SwinBERT2024.11 | 120.6 | — | 41.3 | — | — | |
| SwinBERT2022.05 | 120.6 | 58.2 | — | — | — | |
| Vid2SeqTrained Parameters=314M, Pretraining Data=Ø2023.02 | 120.3 | — | 41.4 | — | — | |
| VideoChat2023.10 | 118.4 | — | 41 | 78.7 | 64.1 | |
| CoCap (ViT-B/16)Decoding=false, E2E=true, 3D Action=CLIP2023.09 | 113 | 55.9 | 39.9 | 76.8 | — | |
| TextKGEvaluation Mode=micro-level2023.03 | 105.2 | 60.8 | 38.5 | 75.1 | — | |
| HMNEvaluation Mode=micro-level2023.03 | 104 | 59.2 | 37.7 | 75.1 | — | |
| HMNDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D, Object Detection=FasterRCNN2023.09 | 104 | 59.2 | 37.7 | 75.1 | — | |
| MGRMPEvaluation Mode=micro-level2023.03 | 98.5 | 55.8 | 36.9 | 74.5 | — | |
| MGRMPDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D2023.09 | 98.5 | 55.8 | 36.9 | 74.5 | — | |
| JCRREvaluation Mode=micro-level2023.03 | 96.8 | 57 | 36.8 | — | — | |
| ORG-TRLAppearance=InceptionResnetV2, Motion=C3D, Object=FasterRCNN2020.02 | 95.2 | 54.3 | 36.4 | 73.9 | — | |
| SOTA2021.11 | 95.2 | — | — | — | — | |
| ORG-TRL2D Appearance=IncepResnetV2, 3D Motion=C3D, Object Detection=FasterRCNN2021.11 | 95.2 | 54.3 | 36.4 | 73.9 | — | |
| ORG-TRLPretraining Data=02023.02 | 95.2 | — | 36.4 | — | — | |
| ORG-TRLEvaluation Mode=micro-level2023.03 | 95.2 | 54.3 | 36.4 | 73.9 | — | |
| ORG-TRLDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D, Object Detection=FasterRCNN2023.09 | 95.2 | 54.3 | 36.4 | 73.9 | — | |
| ORG-TRL2022.05 | 95.2 | 54.3 | — | — | — | |
| PMI-CAP2D Appearance=IncepResnetV2, 3D Motion=C3D2021.11 | 95.1 | 54.6 | 36.4 | — | — | |
| PMI-CAPDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D2023.09 | 95.1 | 54.6 | 36.4 | — | — | |
| PMI-CAP2022.05 | 95.1 | 54.6 | — | — | — | |
| SGNEvaluation Mode=micro-level2023.03 | 94.3 | 52.8 | 35.5 | 72.9 | — | |
| SGNDecoding=false, E2E=false, 2D Appearance=ResNet101, 3D Action=C3D2023.09 | 94.3 | 52.8 | 35.5 | 72.9 | — | |
| STG-KD2D Appearance=ResNet101, 3D Motion=I3D, Object Detection=FasterRCNN2021.11 | 93 | 52.2 | 36.9 | 73.9 | — | |
| STG-KDEvaluation Mode=micro-level2023.03 | 93 | 52.2 | 36.9 | 73.9 | — | |
| STG-KDDecoding=false, E2E=false, 2D Appearance=ResNet101, 3D Action=I3D, Object Detection=FasterRCNN2023.09 | 93 | 52.2 | 36.9 | 73.9 | — | |
| STG-KD2022.05 | 93 | 52.2 | — | — | — | |
| MARNYear=2019, Appearance=ResNet-101, Motion=C3D2020.02 | 92.2 | 48.6 | 35.1 | 71.9 | — | |
| OA-BTGYear=2019, Appearance=ResNet-200, Object=Mask-RCNN2020.02 | 90.6 | 56.9 | 36.2 | — | — | |
| OA-BTG2D Appearance=ResNet200, Object Detection=MaskRCNN2021.11 | 90.6 | 56.9 | 36.2 | — | — | |
| OA-BTGEvaluation Mode=micro-level2023.03 | 90.6 | 56.9 | 36.2 | — | — | |
| OA-BTG2022.05 | 90.6 | 56.9 | — | — | — | |
| POS+CGYear=2019, Appearance=InceptionResnetV2, Motion=OpticalFlow2020.02 | 88.7 | 52.5 | 34.1 | 71.3 | — | |
| POS+CG2D Appearance=IncepResnetV2, 3D Motion=OpticalFlow2021.11 | 88.7 | 52.5 | 34.1 | 71.3 | — | |
| POS-CGEvaluation Mode=micro-level2023.03 | 88.7 | 52.5 | 34.1 | 71.3 | — | |
| POS+CG2022.05 | 88.7 | 52.5 | — | — | — | |
| SibNetYear=2019, Appearance=GoogleNet2020.02 | 88.2 | 54.2 | 34.8 | 71.7 | — | |
| SibNet2D Appearance=GoogleNet2021.11 | 88.2 | 54.2 | 34.8 | 71.7 | — | |
| SibNet2022.05 | 88.2 | 54.2 | — | — | — | |
| POS+VCTYear=2019, Appearance=InceptionResnetV2, Motion=C3D2020.02 | 87.8 | 52.8 | 36.1 | 71.8 | — | |
| POS+VCT2D Appearance=IncepResnetV2, 3D Motion=C3D2021.11 | 87.8 | 52.8 | 36.1 | 71.8 | — | |
| POS+VCT2022.05 | 87.8 | 52.8 | — | — | — | |
| MGSAYear=2019, Appearance=InceptionResnetV2, Motion=C3D2020.02 | 86.7 | 53.4 | 35 | — | — | |
| MGSA2D Appearance=IncepResnetV2, 3D Motion=C3D2021.11 | 86.7 | 53.4 | 35 | — | — | |
| MGSAEvaluation Mode=micro-level2023.03 | 86.7 | 53.4 | 35 | — | — | |
| MGSA2022.05 | 86.7 | 53.4 | — | — | — | |
| SAAT2D Appearance=IncepResnetV2, 3D Motion=C3D2021.11 | 81 | 46.5 | 33.5 | 69.4 | — | |
| SAATDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D2023.09 | 81 | 46.5 | 33.5 | 69.4 | — | |
| SAAT2022.05 | 81 | 46.5 | — | — | — | |
| RecNetYear=2018, Appearance=Inception-V42020.02 | 80.3 | 52.3 | 34.1 | 69.8 | — | |
| GRU-EVEYear=2019, Appearance=InceptionResnetV2, Motion=C3D, Object=YOLO2020.02 | 78.1 | 47.9 | 35 | 71.5 | — | |
| GRU-EVE2D Appearance=IncepResnetV2, 3D Motion=C3D, Object Detection=YOLO2021.11 | 78.1 | 47.9 | 35 | 71.5 | — | |
| GRU-EVE2022.05 | 78.1 | 47.9 | — | — | — | |
| PickNet*Year=2018, Appearance=ResNet-1522020.02 | 76.5 | 52.3 | 33.3 | 69.6 | — | |
| PickNet2D Appearance=ResNet1522021.11 | 76.5 | 52.3 | 33.3 | 69.6 | — | |
| PickNet2022.05 | 76.5 | 52.3 | — | — | — | |
| SA-LSTMYear=2018, Appearance=Inception-V42020.02 | 76.2 | 45.3 | 31.9 | 64.2 | — | |
| ComHeat2024.09 | 66.3 | 51.3 | 34.7 | 69.9 | — | |
| GL-RG+IT2024.09 | 60.6 | 46.9 | 31.2 | 65.7 | — | |
| POSRL2024.09 | 53.4 | 41.3 | 28.7 | 62.1 | — |