Video Question-Answering on Countix-QA (test)
3.77MAEPeriod-LLM
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Period-LLMLLM=LLaMA-7B, Evaluation Protocol=Fine-tune, Resolution=224x2242025.05 | 3.77 | 7.14 | 0.397 | 0.81 | 0.415 | |
| Video-ChatGPTLLM=Vicuna-7B, Evaluation Protocol=Fine-tune, Resolution=224x2242025.05 | 4.64 | 9.15 | 0.356 | 0.643 | 0.367 | |
| VideoLLaMALLM=Vicuna-7B, Evaluation Protocol=Fine-tune, Resolution=224x2242025.05 | 4.98 | 8.89 | 0.389 | 0.57 | 0.358 | |
| VideoChatLLM=Vicuna-7B, Evaluation Protocol=Fine-tune, Resolution=224x2242025.05 | 5.03 | 9.53 | 0.275 | 0.679 | 0.388 | |
| FrozenBiLMLLM=DeBERTa-V2, Evaluation Protocol=Fine-tune, Resolution=224x2242025.05 | 5.22 | 9.44 | 0.233 | 0.61 | 0.201 | |
| LLaMA-VIDLLM=Vicuna-7B, Evaluation Protocol=Fine-tune, Resolution=224x2242025.05 | 5.34 | 9.85 | 0.388 | 0.783 | 0.411 | |
| Period-LLMLLM=LLaMA-7B, Evaluation Protocol=Zero-shot, Pre-training=text-QA, Resolution=224x2242025.05 | 5.51 | 8.33 | 0.288 | 0.702 | 0.308 | |
| LLaMA-AdapterLLM=LLaMA-7B, Evaluation Protocol=Fine-tune, Resolution=224x2242025.05 | 5.78 | 10.02 | 0.378 | 0.789 | 0.397 | |
| BT-AdapterLLM=Vicuna-7B, Evaluation Protocol=Fine-tune, Resolution=224x2242025.05 | 6.12 | 9.7 | 0.296 | 0.689 | 0.353 | |
| VideoLLaMALLM=Vicuna-7B, Evaluation Protocol=Zero-shot, Resolution=224x2242025.05 | 6.18 | 10.82 | 0.187 | 0.36 | 0.245 | |
| Video-ChatGPTLLM=Vicuna-7B, Evaluation Protocol=Zero-shot, Resolution=224x2242025.05 | 6.44 | 10.75 | 0.203 | 0.589 | 0.297 | |
| VideoChatLLM=Vicuna-7B, Evaluation Protocol=Zero-shot, Resolution=224x2242025.05 | 6.73 | 11.23 | 0.165 | 0.549 | 0.277 | |
| LLaMA-VIDLLM=Vicuna-7B, Evaluation Protocol=Zero-shot, Resolution=224x2242025.05 | 6.97 | 11.21 | 0.267 | 0.68 | 0.301 | |
| FrozenBiLMLLM=DeBERTa-V2, Evaluation Protocol=Zero-shot, Resolution=224x2242025.05 | 7.22 | 11.53 | 0.145 | 0.55 | 0.112 | |
| LLaMA-AdapterLLM=LLaMA-7B, Evaluation Protocol=Zero-shot, Resolution=224x2242025.05 | 7.44 | 10.51 | 0.244 | 0.683 | 0.312 | |
| BT-AdapterLLM=Vicuna-7B, Evaluation Protocol=Zero-shot, Resolution=224x2242025.05 | 7.73 | 11.7 | 0.276 | 0.674 | 0.341 |