Segment-level Video Captioning on YouCook2
15.2BLEU-4Human
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| HumanInput=Video + ASR, Pretraining=-2020.11 | 15.2 | 25.9 | 45.1 | 3.8 | |
| E2vidD6-MASSvid-BiDInput=Video + ASR, Pretraining=YT8M-cook + Recipe1M2020.11 | 12.04 | 18.32 | 39.03 | 1.23 | |
| E2vidD6-MASSalign-BiDInput=Video + ASR, Pretraining=YT8M-cook + Recipe1M2020.11 | 11.53 | 17.62 | 39.03 | 1.22 | |
| E2vidD6-MASS-BiDInput=Video + ASR, Pretraining=YT8M-cook + Recipe1M2020.11 | 11.47 | 17.7 | 38.8 | 1.25 | |
| E2D6-MASS-BiDInput=ASR, Pretraining=YT8M-cook + Recipe1M2020.11 | 10.6 | 17.42 | 38.08 | 1.2 | |
| E2vidD6-MASSdrop-BiDInput=Video + ASR, Pretraining=YT8M-cook + Recipe1M2020.11 | 10.45 | 17.74 | 38.82 | 1.22 | |
| UniViLM #5Input=Video + ASR, Pretraining=HowTo100M2020.11 | 10.42 | 16.93 | 38.02 | 1.2 | |
| AT+VideoInput=Video + ASR, Pretraining=-2020.11 | 9.01 | 17.77 | 36.65 | 1.12 | |
| UniViLM #2Input=Video + ASR, Pretraining=-2020.11 | 8.67 | 15.38 | 35.02 | 1 | |
| ATInput=ASR, Pretraining=-2020.11 | 8.55 | 16.93 | 35.54 | 1.06 | |
| E2vidD6-BiDInput=Video + ASR, Pretraining=-2020.11 | 8.01 | 16.19 | 34.66 | 0.91 | |
| MARTInput=Video, Pretraining=-2020.11 | 8 | 15.9 | — | 0.36 | |
| E2D6-BiDInput=ASR, Pretraining=-2020.11 | 7.9 | 15.7 | 34.86 | 0.93 | |
| UniViLM #1Input=Video, Pretraining=-2020.11 | 6.06 | 12.47 | 31.48 | 0.64 | |
| CBTInput=Video, Pretraining=Kinetics + HowTo100M2020.11 | 5.12 | 12.97 | 30.44 | 0.64 | |
| EMTInput=Video, Pretraining=-2020.11 | 4.38 | 11.55 | 27.44 | 0.38 | |
| Constant PredInput=-, Pretraining=-2020.11 | 2.7 | 10.3 | 21.7 | 0.15 |