Video Generation Evaluation on Video-ChatGPT benchmark
61.6Correctness ScoreCAT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CATLLM Size=7B2024.03 | 61.6 | 62 | 69.8 | 56.2 | 57.8 | — | |
| LLAMA-VIDLLM Size=13B2024.03 | 61.4 | 61 | 72 | 51.6 | 52.6 | — | |
| VideoChat2LLM Size=7B2024.03 | 60.4 | 57.8 | 70.2 | 53.2 | 56.2 | — | |
| LLAMA-VIDLLM Size=7B2024.03 | 59.2 | 60 | 70.6 | 49.2 | 50.2 | — | |
| Chat-UniViLLM Size=7B2024.03 | 57.8 | 58.2 | 69.2 | 57.8 | 56.2 | — | |
| VISTA-LLAMALLM Size=7B2024.03 | 48.8 | 52.8 | 63.6 | 45.2 | 46.2 | — | |
| Video-ChatGPTLLM Size=7B2024.03 | 48 | 50.4 | 52.4 | 39.6 | 47.4 | — | |
| VideoChatLLM Size=7B2024.03 | 44.6 | 50 | 50.6 | 38.8 | 44.8 | — | |
| LLaMA-AdapterLLM Size=7B2024.03 | 40.6 | 46.4 | 46 | 39.6 | 43 | — | |
| Video-LLaMALLM Size=7B2024.03 | 39.2 | 43.6 | 43.2 | 36.4 | 35.8 | — | |
| GPT-4V2024.02 | 3.85 | 3.45 | 3.84 | 3.63 | 2.8 | — | |
| LLaVA-NeXT-Video-DPOLLM Size=34B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 3.81 | 3.55 | 4.24 | 3.14 | 4.12 | 3.77 | |
| LLaVA-NeXT-Video-DPOLLM Size=7B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 3.64 | 3.45 | 4.17 | 2.95 | 4.08 | 3.66 | |
| VLM-RLAIFLLM Size=7B2024.02 | 3.63 | 3.25 | 4 | 3.23 | 3.32 | — | |
| TS-LLAVALLM Size=34B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 3.55 | 3.03 | 3.86 | 2.77 | 3.69 | 3.38 | |
| LLaVA-NeXT-VideoLLM Size=34B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 3.48 | 3.37 | 3.95 | 2.64 | 3.28 | 3.34 | |
| SF-LLAVALLM Size=34B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 3.48 | 2.96 | 3.84 | 2.77 | 3.57 | 3.32 | |
| LLaVA-NeXT-VideoLLM Size=7B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 3.39 | 3.29 | 3.92 | 2.6 | 3.12 | 3.26 | |
| LLaVA-NeXT-ImageLLM Size=34B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 3.29 | 3.23 | 3.83 | 2.51 | 3.47 | 3.27 | |
| VideoGPT+LLM Size=3.8B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 3.27 | 3.18 | 3.74 | 2.83 | 3.39 | 3.28 | |
| ST-LLM2024.03 | 3.23 | 3.05 | 3.74 | 2.93 | 2.81 | 3.15 | |
| IG-VLMLLM Size=34B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 3.21 | 2.87 | 3.54 | 2.51 | 3.34 | 3.09 | |
| TS-LLaVALLM Size=7B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 3.18 | 2.82 | 3.58 | 2.53 | 3.34 | 3.09 | |
| Video-LLaMA2LLM Size=7B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 3.16 | 3.08 | 3.69 | 2.56 | 3.14 | 3.13 | |
| IG-VLMLLM Size=7B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 3.11 | 2.78 | 3.51 | 2.44 | 3.29 | 3.03 | |
| SF-LLaVALLM Size=7B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 3.09 | 2.7 | 3.57 | 2.52 | 3.35 | 3.04 | |
| Video-LLaMA2LLM Size=46.7B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 3.08 | 3.11 | 3.64 | 2.67 | 3.26 | 3.15 | |
| LLAMA-VIDLLM=Vicuna-13B, Res.=2242023.11 | 3.07 | 3.05 | 3.6 | 2.58 | 2.63 | — | |
| LLaMA-VID-13BParameters=13B2024.03 | 3.07 | 3.05 | 3.6 | 2.58 | 2.63 | 2.99 | |
| LLaMA-VIDLLM Size=13B2024.02 | 3.07 | 3.05 | 3.6 | 2.58 | 2.63 | — | |
| LLaVA-NeXT-ImageLLM Size=7B, Vision Encoder=CLIP-L, Training-free=true2024.11 | 3.05 | 3.12 | 3.68 | 2.37 | 3.16 | 3.07 | |
| VideoChat22024.03 | 3.02 | 2.88 | 3.51 | 2.66 | 2.81 | 2.98 | |
| VideoChat2LLM Size=7B2024.02 | 3.02 | 2.88 | 3.51 | 2.66 | 2.81 | — | |
| VideoChat2LLM Size=7B, Vision Encoder=UMT-L, Training-free=false2024.11 | 3.02 | 2.88 | 3.51 | 2.66 | 2.81 | 2.98 | |
| LLAMA-VIDLLM=Vicuna-7B, Res.=2242023.11 | 2.96 | 3 | 3.53 | 2.46 | 2.51 | — | |
| LLaMA-VID-7BParameters=7B2024.03 | 2.96 | 3 | 3.53 | 2.46 | 2.51 | 2.89 | |
| LLaMA-VIDLLM Size=7B2024.02 | 2.96 | 3 | 3.53 | 2.46 | 2.51 | — | |
| LLaMA-VIDLLM Size=13B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 2.96 | 3 | 3.53 | 2.46 | 2.51 | 2.89 | |
| LITA2024.03 | 2.94 | 2.98 | 3.43 | 2.68 | 3.19 | 3.04 | |
| Chat-UniVi2024.03 | 2.89 | 2.91 | 3.46 | 2.89 | 2.81 | 2.99 | |
| Video-LLaVALLM Size=7B2024.02 | 2.84 | 2.86 | 3.44 | 2.46 | 2.57 | — | |
| VLM-SFTLLM Size=7B2024.02 | 2.79 | 2.82 | 3.37 | 2.28 | 2.49 | — | |
| VTimeLLM2023.11 | 2.78 | 3.1 | 3.4 | 2.49 | 2.47 | 2.85 | |
| VTimeLLM2024.03 | 2.78 | 3.1 | 3.4 | 2.49 | 2.47 | 2.85 | |
| VTimeLLMLLM Size=7B2024.02 | 2.78 | 3.1 | 3.4 | 2.49 | 2.47 | — | |
| MovieChatLLM Size=7B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 2.76 | 2.93 | 3.01 | 2.24 | 2.42 | 2.67 | |
| BT-AdapterLLM=Vicuna-7B, Res.=2242023.11 | 2.68 | 2.69 | 3.27 | 2.34 | 2.46 | — | |
| BT-Adapter2023.11 | 2.68 | 2.69 | 3.27 | 2.34 | 2.46 | 2.69 | |
| BT-Adapter2024.03 | 2.68 | 2.69 | 3.27 | 2.34 | 2.46 | 2.69 | |
| BT-AdapterLLM Size=7B2024.02 | 2.68 | 2.69 | 3.27 | 2.34 | 2.46 | — | |
| Video-ChatGPTLLM Size=7B, Vision Encoder=CLIP-L, Training-free=false2024.11 | 2.5 | 2.57 | 2.69 | 2.16 | 2.2 | 2.42 | |
| Vista-LLaMALLM Size=7B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 2.44 | 2.64 | 3.18 | 2.26 | 2.31 | 2.57 | |
| ValleyLLM Size=7B2024.02 | 2.43 | 2.13 | 2.86 | 2.04 | 2.45 | — | |
| Video-ChatGPTLLM=Vicuna-7B, Res.=2242023.11 | 2.4 | 2.52 | 2.62 | 1.98 | 2.37 | — | |
| VideoChatGPT2023.11 | 2.4 | 2.52 | 2.62 | 1.98 | 2.37 | 2.38 | |
| Video-ChatGPT2024.03 | 2.4 | 2.52 | 2.62 | 1.98 | 2.37 | 2.38 | |
| VideoChatGPT2024.03 | 2.4 | 2.52 | 2.62 | 1.98 | 2.37 | 2.38 | |
| Video-ChatGPTLLM Size=7B2024.02 | 2.4 | 2.52 | 2.62 | 1.98 | 2.37 | — | |
| Video-LLaMA-v22024.03 | 2.36 | 2.42 | 2.74 | 1.83 | 2.12 | 2.29 | |
| VideoChatLLM=Vicuna-7B, Res.=2242023.11 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | — | |
| VideoChat2023.11 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | 2.29 | |
| VideoChat2024.03 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | 2.29 | |
| VideoChat2024.03 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | 2.29 | |
| VideoChatLLM Size=7B2024.02 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | — | |
| VideoChatLLM Size=7B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | 2.29 | |
| LLaMA-AdapterLLM=LLaMA-7B, Res.=2242023.11 | 2.03 | 2.32 | 2.3 | 1.98 | 2.15 | — | |
| LLaMA-Adapter2023.11 | 2.03 | 2.32 | 2.3 | 1.98 | 2.15 | 2.16 | |
| LLaMA-Adapter2024.03 | 2.03 | 2.32 | 2.3 | 1.98 | 2.15 | 2.16 | |
| LLaMA-Adapter2024.03 | 2.03 | 2.32 | 2.3 | 1.98 | 2.15 | 2.16 | |
| LLaMA-AdapterLLM Size=7B2024.02 | 2.03 | 2.32 | 2.3 | 1.98 | 2.15 | — | |
| VideoLLaMALLM=Vicuna-7B, Res.=2242023.11 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | — | |
| VideoLLaMA2023.11 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | 1.98 | |
| Video-LLaMA2024.03 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | 1.98 | |
| VideoLLaMA2024.03 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | 1.98 | |
| VideoLLaMALLM Size=7B2024.02 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | — | |
| Video-LLaMALLM Size=7B, Vision Encoder=CLIP-G, Training-free=false2024.11 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | 1.98 |