Video Captioning on Vript-HAL (full)
58.3F1 ScoreVriptor-S
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Vriptor-Svideo input mode=scene-by-scene, voice-over transcription utilization=true2024.06 | 58.3 | 76.1 | 47.2 | |
| Vriptor-Svideo input mode=scene-by-scene, voice-over transcription utilization=false2024.06 | 56.6 | 73.4 | 46.1 | |
| GPT-4Oframe rate=1 fps, frames=1002024.06 | 49.6 | — | — | |
| GPT-4V2024.06 | 49.3 | 77.3 | 36.2 | |
| GPT-4O2024.06 | 49.3 | — | — | |
| Claude 3.5-Sonnet2024.06 | 44.6 | — | — | |
| Claude 3-Opus2024.06 | 41.7 | 73.4 | 29.1 | |
| Vriptor-Wvideo input mode=whole-video, voice-over transcription utilization=true2024.06 | 41.1 | 80.3 | 27.7 | |
| Vriptor-Wvideo input mode=whole-video, voice-over transcription utilization=false2024.06 | 40 | 79.1 | 26.8 | |
| LLaVA 1.6 34B2024.06 | 37.8 | 67.6 | 26.3 | |
| Claude 3-Sonnet2024.06 | 37.2 | 71.7 | 25.1 | |
| PLLaVA 7B2024.06 | 32.8 | — | — | |
| VILA-1.5 8B2024.06 | 31.8 | — | — | |
| VideoChat2024.06 | 30.9 | 59.1 | 20.9 | |
| ST-LLM2024.06 | 27.3 | 67.1 | 17.2 | |
| Gemini-1.5-Pro2024.06 | 27 | — | — | |
| Video-LLaMA2024.06 | 24.5 | 65.1 | 15.1 | |
| InstructBLIP2024.06 | 21.8 | 71 | 13.1 | |
| VideoChatGPT2024.06 | 16.7 | 69.4 | 9.5 | |
| VideoChat22024.06 | 15.4 | 73.2 | 8.6 | |
| Qwen-VL2024.06 | 12.4 | 68.8 | 7.2 | |
| BLIP22024.06 | 4.8 | 77.9 | 2.5 |