Video Question Answering on MovieChat
34.04Throughput (Tokens/s)LVSPEC
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LVSPECModel=Qwen2.5-VL, Setting=Self-SD2026.04 | 34.04 | 7.99 | 1.33 | — | — | |
| SPECVLMModel=Qwen2.5-VL, Setting=Self-SD2026.04 | 32.2 | 3.66 | 1.26 | — | — | |
| FLY⊖Model=Qwen2.5-VL, Setting=Self-SD2026.04 | 28.23 | 5.98 | 1.11 | — | — | |
| FLYModel=Qwen2.5-VL, Setting=Self-SD2026.04 | 25.82 | 5.44 | 1.01 | — | — | |
| AutoregressiveModel=Qwen2.5-VL, Setting=Self-SD2026.04 | 25.51 | — | 1 | — | — | |
| NAIVE SDModel=Qwen2.5-VL, Setting=Self-SD2026.04 | 21.86 | 4.46 | 0.86 | — | — | |
| LVSPECModel=Qwen2.5-VL, Setting=Std.-SD2026.04 | 21.49 | 7.74 | 2.19 | — | — | |
| SPECVLMModel=LLaVA-OV, Setting=Self-SD2026.04 | 21.19 | 4.17 | 1.23 | — | — | |
| FLY⊖Model=Qwen2.5-VL, Setting=Std.-SD2026.04 | 20.76 | 6.74 | 2.12 | — | — | |
| LVSPECModel=LLaVA-OV, Setting=Self-SD2026.04 | 20.28 | 8.77 | 1.18 | — | — | |
| FLY⊖Model=LLaVA-OV, Setting=Self-SD2026.04 | 17.94 | 8.06 | 1.04 | — | — | |
| FLYModel=LLaVA-OV, Setting=Self-SD2026.04 | 17.29 | 7.65 | 1 | — | — | |
| AutoregressiveModel=LLaVA-OV, Setting=Self-SD2026.04 | 17.23 | — | 1 | — | — | |
| SPECVLMModel=Qwen2.5-VL, Setting=Std.-SD2026.04 | 17.05 | 3.19 | 1.74 | — | — | |
| NAIVE SDModel=Qwen2.5-VL, Setting=Std.-SD2026.04 | 13.38 | 3.19 | 1.36 | — | — | |
| FLYModel=Qwen2.5-VL, Setting=Std.-SD2026.04 | 13.01 | 3.86 | 1.33 | — | — | |
| LVSPECModel=LLaVA-OV, Setting=Std.-SD2026.04 | 12.16 | 8.1 | 2.75 | — | — | |
| NAIVE SDModel=LLaVA-OV, Setting=Self-SD2026.04 | 11.16 | 4.79 | 0.65 | — | — | |
| AutoregressiveModel=Qwen2.5-VL, Setting=Std.-SD2026.04 | 9.81 | — | 1 | — | — | |
| FLY⊖Model=LLaVA-OV, Setting=Std.-SD2026.04 | 9.23 | 5.55 | 2.09 | — | — | |
| SPECVLMModel=LLaVA-OV, Setting=Std.-SD2026.04 | 8.45 | 3.77 | 1.92 | — | — | |
| FLYModel=LLaVA-OV, Setting=Std.-SD2026.04 | 5.72 | 5.01 | 1.3 | — | — | |
| NAIVE SDModel=LLaVA-OV, Setting=Std.-SD2026.04 | 5.36 | 3.88 | 1.22 | — | — | |
| AutoregressiveModel=LLaVA-OV, Setting=Std.-SD2026.04 | 4.41 | — | 1 | — | — | |
| TrajViTLLM=Llama-3, Setup=Linear layer connector, Pretraining Data=8M video + 50M image2025.05 | — | — | — | 36.7 | 3.1 | |
| ViT3DLLM=Llama-3, Setup=Linear layer connector, Pretraining Data=8M video + 50M image2025.05 | — | — | — | 34.7 | 3.1 |