Video Summary on Ego4D-HCap v1 (test)
31.06CIDErVideo ReCap-U
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Video ReCap-UVideo Encoder=TSF-B, Text Decoder=GPT2, Train Params=113M, Pseudo Ann.=true, Protocol=Finetuned2024.02 | 31.06 | 33.32 | 14.16 | |
| Video ReCapVideo Encoder=TSF-B, Text Decoder=GPT2, Train Params=339M, Pseudo Ann.=true, Protocol=Finetuned2024.02 | 29.34 | 32.64 | 14.45 | |
| Video ReCapVideo Encoder=TSF-B, Text Decoder=GPT2, Train Params=339M, Pseudo Ann.=false, Protocol=Finetuned2024.02 | 28.06 | 32.27 | 14.26 | |
| LaViLa + FLANT5Video Encoder=TSF-B, Text Decoder=FT5-XL, Train Params=586M, Pseudo Ann.=false, Protocol=Finetuned2024.02 | 20.12 | 30.06 | 13.17 | |
| LaViLa + GPT2Video Encoder=TSF-B, Text Decoder=GPT2, Train Params=336M, Pseudo Ann.=false, Protocol=Finetuned2024.02 | 17.98 | 29.48 | 12.81 | |
| LaViLa + GPT3.5Video Encoder=TSF-B, Text Decoder=GPT2, Train Params=0, Pseudo Ann.=false, Protocol=Zero-Shot2024.02 | 12.16 | 24.49 | 12.48 | |
| BLIP2 + GPT3.5Video Encoder=VIT-G, Text Decoder=FT5-XL, Train Params=0, Pseudo Ann.=false, Protocol=Zero-Shot2024.02 | 11.13 | 22.41 | 12.1 | |
| LaViLaVideo Encoder=TSF-B, Text Decoder=GPT2, Train Params=258M, Pseudo Ann.=false, Protocol=Finetuned2024.02 | 6.54 | 23.97 | 10.95 |