Video Question Answering on MovieChat long video 1K (test)
3.11CI ScoreMovieChat
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MovieChatEvaluation Mode=Global Mode2023.07 | 3.11 | 2.93 | 3.24 | 3.17 | 3.25 | |
| Video ChatEvaluation Mode=Global Mode2023.07 | 3.04 | 2.75 | 3.09 | 3 | 3.21 | |
| Video LLAMAEvaluation Mode=Global Mode2023.07 | 2.75 | 2.24 | 2.83 | 2.62 | 2.97 | |
| MovieChat+Text Decoder=LLM based2024.04 | 2.65 | 2.62 | 2.87 | 2.51 | 3.07 | |
| MovieChatText Decoder=LLM based2024.04 | 2.64 | 2.6 | 2.87 | 2.49 | 3.08 | |
| Video-ChatGPTText Decoder=LLM based2024.04 | 2.62 | 2.65 | 2.86 | 2.32 | 2.96 | |
| Video ChatText Decoder=LLM based2024.04 | 2.42 | 2.51 | 2.81 | 2.1 | 2.78 | |
| Video-ChatGPTEvaluation Mode=Global Mode2023.07 | 2.37 | 2.3 | 2.58 | 2.49 | 2.69 | |
| Video LLAMAText Decoder=LLM based2024.04 | 2.04 | 2.29 | 2.63 | 2 | 2.87 | |
| mPLUG-2Text Decoder=non-LLM based2024.04 | 1.53 | 1.62 | 1.91 | 1.56 | 2.14 | |
| GITText Decoder=non-LLM based2024.04 | 1.44 | 1.48 | 1.96 | 1.77 | 2.01 |