Video Question Answering on VCG Bench
3.85CIVLM-RLAIF
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VLM-RLAIFBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=true2024.11 | 3.85 | 3.45 | 3.84 | 3.63 | 2.8 | 3.49 | |
| PPLLaVABase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=true2024.11 | 3.85 | 3.56 | 4.21 | 3.21 | 3.81 | 3.73 | |
| LLaVA-Next-VideoBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=true2024.11 | 3.64 | 3.45 | 4.17 | 2.95 | 4.08 | 3.66 | |
| LLaVA-Next-VideoBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 3.39 | 3.29 | 3.92 | 2.6 | 3.12 | 3.26 | |
| Grounded-VideoLLM2025.01 | 3.34 | 2.94 | 3.66 | 3.12 | 3.14 | 3.24 | |
| PPLLaVABase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 3.32 | 3.2 | 3.88 | 3 | 3.2 | 3.32 | |
| LLaVA-ST2025.01 | 3.29 | 3.18 | 3.8 | 2.9 | 3.41 | 3.32 | |
| DynFocusModel Size=7B, Protocol=Fine-tuned on all datasets and post-tuned on VideoChatGPT-100K2024.11 | 3.27 | 3.15 | 3.78 | 2.86 | 2.78 | 3.17 | |
| VideoGPT+2025.01 | 3.27 | 3.18 | 3.74 | 2.83 | 3.39 | 3.28 | |
| ST-LLMBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 3.23 | 3.05 | 3.74 | 2.93 | 2.81 | 3.15 | |
| ST-LLMModel Size=7B2024.11 | 3.23 | 3.05 | 3.74 | 2.93 | 2.81 | 3.15 | |
| ST-LLM2025.01 | 3.23 | 3.05 | 3.74 | 2.93 | 2.81 | 3.15 | |
| PLLaVABase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 3.21 | 2.86 | 3.62 | 2.33 | 2.93 | 2.99 | |
| PLLAVAModel Size=7B2024.11 | 3.21 | 2.86 | 3.62 | 2.33 | 2.93 | 3.12 | |
| P-LLaVAParameters=7B2025.01 | 3.21 | 2.86 | 3.62 | 2.33 | 2.93 | 3.12 | |
| DynFocusModel Size=7B, Protocol=Standard2024.11 | 3.12 | 3.11 | 3.68 | 2.57 | 2.74 | 3.05 | |
| CATBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 3.08 | 2.95 | 3.49 | 2.81 | 2.89 | 3.07 | |
| VideoChat2Base LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 3.02 | 2.88 | 3.51 | 2.66 | 2.81 | 2.98 | |
| VideoChat2Model Size=7B2024.11 | 3.02 | 2.88 | 3.51 | 2.66 | 2.81 | 2.98 | |
| VideoChat22025.01 | 3.02 | 2.88 | 3.51 | 2.66 | 2.81 | 2.98 | |
| LLaMA-VIDBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 2.96 | 3 | 3.53 | 2.46 | 2.51 | 2.89 | |
| LLAMA-VIDModel Size=7B2024.11 | 2.96 | 3 | 3.53 | 2.46 | 2.51 | 2.89 | |
| LITA2025.01 | 2.94 | 2.98 | 3.43 | 2.68 | 3.19 | 3.04 | |
| Chat-UniViBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 2.89 | 2.91 | 3.46 | 2.89 | 2.81 | 2.99 | |
| Chat-UniViModel Size=7B2024.11 | 2.89 | 2.91 | 3.46 | 2.89 | 2.81 | 2.99 | |
| Chat-UniVi2025.01 | 2.89 | 2.91 | 3.46 | 2.89 | 2.81 | 2.99 | |
| VTimeLLMModel Size=7B2024.11 | 2.78 | 3.1 | 3.4 | 2.49 | 2.47 | 2.85 | |
| VTimeLLM2025.01 | 2.78 | 3.1 | 3.4 | 2.49 | 2.47 | 2.85 | |
| MovieChatBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 2.76 | 2.93 | 3.01 | 2.24 | 2.42 | 2.67 | |
| MovieChat2025.01 | 2.76 | 2.93 | 3.01 | 2.24 | 2.42 | 2.67 | |
| LongVLM2025.01 | 2.76 | 2.86 | 3.34 | 2.39 | 3.11 | 2.89 | |
| BT-AdapterBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 2.68 | 2.69 | 3.27 | 2.34 | 2.46 | 2.69 | |
| BT-AdapterModel Size=7B2024.11 | 2.68 | 2.69 | 3.27 | 2.34 | 2.46 | 2.69 | |
| Video-ChatGPTBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 2.5 | 2.57 | 2.69 | 2.16 | 2.2 | 2.42 | |
| Video-ChatGPT2025.01 | 2.5 | 2.57 | 2.69 | 2.16 | 2.2 | 2.42 | |
| Vista-LLaMABase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 2.44 | 2.64 | 3.18 | 2.26 | 2.31 | 2.57 | |
| VideoChatGPTModel Size=7B2024.11 | 2.4 | 2.52 | 2.62 | 1.98 | 2.37 | 2.38 | |
| VideoChatBase LLM=Vicuna-7B, Alignment Tuning (DPO/PPO)=false2024.11 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | 2.29 | |
| VideoChatModel Size=7B2024.11 | 2.23 | 2.5 | 2.53 | 1.94 | 2.24 | 2.29 | |
| LLaMA-AdapterModel Size=7B2024.11 | 2.03 | 2.32 | 2.3 | 1.98 | 2.15 | 2.16 | |
| VideoLLaMAModel Size=7B2024.11 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | 1.98 | |
| Video-LLaMA2025.01 | 1.96 | 2.18 | 2.16 | 1.82 | 1.79 | 1.98 |