Video Generative Performance on Video-based Generative Performance Benchmark
3.49Correctness ScoreFMVR-LLaVA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FMVR-LLaVA#Vision Tokens=28802026.03 | 3.49 | 3.71 | 3.85 | 3.14 | 3.44 | 3.53 | |
| FMVR-LLaVA#Vision Tokens=7202026.03 | 3.45 | 3.62 | 3.77 | 3.05 | 3.21 | 3.42 | |
| FMVR-LLaVA#Vision Tokens=1802026.03 | 3.3 | 3.55 | 3.72 | 2.84 | 3.1 | 3.3 | |
| LLaVA-Mini#Frames=1fps, #Vision Tokens per Frame=12025.01 | 2.97 | 2.99 | 3.61 | 2.48 | 2.67 | 2.94 | |
| LLaMA-VID#Frames=1fps, #Vision Tokens per Frame=22025.01 | 2.96 | 3 | 3.53 | 2.46 | 2.51 | 2.88 | |
| LLaMA-VID#Vision Tokens=1fps×22026.03 | 2.96 | 3 | 3.53 | 2.46 | 2.51 | 2.89 | |
| FMVR-LLaVA#Vision Tokens=452026.03 | 2.91 | 2.88 | 3.23 | 2.41 | 2.46 | 2.78 | |
| Video-LLaVA#Frames=8, #Vision Tokens per Frame=2562025.01 | 2.87 | 2.94 | 3.44 | 2.45 | 2.51 | 2.84 | |
| Video-LLAVA#Vision Tokens=20482026.03 | 2.87 | 2.94 | 3.44 | 2.45 | 2.51 | 2.84 | |
| MovieChat#Frames=2048, #Vision Tokens per Frame=322025.01 | 2.76 | 2.93 | 3.01 | 2.24 | 2.42 | 2.65 | |
| MovieChat#Vision Tokens=655362026.03 | 2.76 | 2.93 | 3.01 | 2.24 | 2.42 | 2.67 | |
| BT-Adapter#Frames=100, #Vision Tokens per Frame=~2.62025.01 | 2.68 | 2.69 | 3.27 | 2.34 | 2.46 | 2.69 | |
| BT-Adapter#Vision Tokens=~2602026.03 | 2.68 | 2.69 | 3.27 | 2.34 | 2.46 | 2.69 | |
| FMVR-LLaVA#Vision Tokens=52026.03 | 2.54 | 2.77 | 2.83 | 2.34 | 2.41 | 2.58 | |
| Video-ChatGPT#Frames=100, #Vision Tokens per Frame=~3.62025.01 | 2.4 | 2.52 | 2.62 | 1.98 | 2.37 | 2.37 | |
| Video-ChatGPT#Vision Tokens=~3602026.03 | 2.4 | 2.52 | 2.62 | 1.98 | 2.37 | 2.38 | |
| VideoChat#Frames=16, #Vision Tokens per Frame=322025.01 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | 2.3 | |
| VideoChat#Vision Tokens=5122026.03 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | 2.29 | |
| LLaMA Adapter#Frames=5, #Vision Tokens per Frame=2562025.01 | 2.03 | 2.32 | 2.3 | 1.98 | 2.15 | 2.19 | |
| LLaMA Adapter#Vision Tokens=12802026.03 | 2.03 | 2.32 | 2.3 | 1.98 | 2.15 | 2.16 | |
| Video-LLaMA#Frames=16, #Vision Tokens per Frame=642025.01 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | 1.99 | |
| Video-LLAMA#Vision Tokens=10242026.03 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | 1.98 |