Video Captioning on YouCook2 (test)
190CIDErUniVL + MELTR
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| UniVL + MELTRModality=V+T2023.03 | 190 | 17.92 | 47.04 | 22.56 | — | 24.12 | |
| UniVLModality=V+T2023.03 | 181 | 17.35 | 46.52 | 22.35 | — | 23.87 | |
| UniVL#VideoPT=1.2M, ASR=Yes, Evaluation Protocol=Fine-tuning2022.05 | 176.8 | 16.6 | 45.7 | 21.6 | 106.2 | — | |
| Pretrained FT SOTAFT=true, Shot=10K2022.04 | 138.7 | — | — | — | — | — | |
| UniVL + MELTRModality=V2023.03 | 138 | 11.98 | 41.28 | 18.19 | — | 17.35 | |
| COSA (1.2B)Parameters=415M2023.06 | 131.3 | 10.1 | — | — | — | — | |
| GIT2Parameters=12.9B2023.06 | 131.2 | 9.4 | — | — | — | — | |
| VALUEParameters=136M, Additional modalities=true2023.06 | 130.3 | 12.4 | — | — | — | — | |
| GITParameters=1.7B2023.06 | 129.8 | 10.3 | — | — | — | — | |
| UniVLModality=V2023.03 | 127 | 11.17 | 40.09 | 17.57 | — | 16.46 | |
| UniVL#VideoPT=1.2M, ASR=No, Evaluation Protocol=Fine-tuning2022.05 | 127 | 11.2 | 40.1 | 17.6 | 70.2 | — | |
| COSA-LParameters=417M2023.06 | 125.8 | 10.2 | — | — | — | — | |
| Flamingo(80B)Parameters=2.3B2023.06 | 118.6 | — | — | — | — | — | |
| AT+VideoModality=V+T2023.03 | 112 | — | 36.65 | 17.77 | — | 9.01 | |
| VidIL#VideoPT=0, ASR=Yes, Evaluation Protocol=Few-shot, Num Shots=102022.05 | 111.6 | 10.7 | 35.9 | 19.4 | 75.3 | — | |
| SWINBERTframes=642021.11 | 109 | — | — | — | — | — | |
| COSA-BParameters=17M2023.06 | 106.2 | 8.6 | — | — | — | — | |
| COSA-BParameters=5M2023.06 | 100.9 | 7.8 | — | — | — | — | |
| FlamingoFT=false, Shot=322022.04 | 86.8 | — | — | — | — | — | |
| Flamingo-80B#VideoPT=27M, ASR=No, Evaluation Protocol=Few-shot, Num Shots=162022.05 | 84.2 | — | — | — | — | — | |
| Flamingo-9BFT=false, Shot=322022.04 | 77.3 | — | — | — | — | — | |
| Flamingo-3BFT=false, Shot=322022.04 | 76.7 | — | — | — | — | — | |
| FlamingoFT=false, Shot=42022.04 | 74.5 | — | — | — | — | — | |
| Flamingo-3B#VideoPT=27M, ASR=No, Evaluation Protocol=Few-shot, Num Shots=162022.05 | 73.2 | — | — | — | — | — | |
| Flamingo-9BFT=false, Shot=42022.04 | 70.8 | — | — | — | — | — | |
| ActBERTModality=V2023.03 | 65 | 5.41 | 30.56 | 13.3 | — | 8.66 | |
| Flamingo-3BFT=false, Shot=42022.04 | 64.6 | — | — | — | — | — | |
| CBTModality=V2023.03 | 64 | 5.12 | 30.44 | 12.97 | — | — | |
| FlamingoFT=false, Shot=02022.04 | 60.1 | — | — | — | — | — | |
| COOTModality=V2023.03 | 57 | 11.3 | 37.94 | 19.85 | — | 17.97 | |
| Flamingo-3BFT=false, Shot=02022.04 | 55.8 | — | — | — | — | — | |
| Flamingo-9BFT=false, Shot=02022.04 | 55 | — | — | — | — | — | |
| VideoBERTModality=V+T2023.03 | 55 | 4.33 | 28.8 | 11.94 | — | 7.59 | |
| SOTA2021.11 | 53.6 | — | — | — | — | — | |
| UniVL#VideoPT=1.2M, ASR=Yes, Evaluation Protocol=Few-shot, Num Shots=102022.05 | 48.6 | 4.3 | 26.4 | 12.2 | 26 | — | |
| VideoBERTModality=V2023.03 | 47 | 3.81 | 27.14 | 10.81 | — | 6.33 | |
| EMTModality=V2023.03 | 38 | 4.38 | 27.44 | 11.55 | — | 7.53 | |
| UniVL#VideoPT=1.2M, ASR=No, Evaluation Protocol=Few-shot, Num Shots=102022.05 | 34.1 | 3.3 | 25.3 | 11.6 | 13.3 | — | |
| VidIL#VideoPT=0, ASR=No, Evaluation Protocol=Few-shot, Num Shots=102022.05 | 27 | 2.6 | 22.9 | 9.5 | 33.3 | — | |
| Zero/Few shot SOTAFT=false, Shot=02022.04 | 11.6 | — | — | — | — | — | |
| BLIP#VideoPT=0, ASR=No, Evaluation Protocol=Few-shot, Num Shots=102022.05 | 11.5 | 0.7 | 9 | 3.4 | 23.9 | — | |
| BLIP_cap#VideoPT=0, ASR=No, Evaluation Protocol=Few-shot, Num Shots=102022.05 | 9.4 | 3.7 | 8.6 | 3.8 | 22.8 | — | |
| DPCModality=V+T2023.03 | — | 2.76 | — | 18.08 | — | 7.6 |