Video Question Answering on MMVU (Accuracy)
58.5AccuracyVanilla
Evaluation Results
| Method | Links | |
|---|---|---|
| VanillaBackbone=Qwen3-VL-8B, Retention Ratio R=100%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 58.5 | |
| VanillaBackbone=Qwen3-VL-8B, Retention Ratio R=100%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 57.5 | |
| Random DropBackbone=Qwen3-VL-8B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 57.1 | |
| Random DropBackbone=Qwen3-VL-8B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 56.8 | |
| VideoAuto-R1Backbone=Qwen2.5-VL-7B, Retention Ratio R=100%, Reasoning (CoT)=true, Frames=128 Frames2026.03 | 56.7 | |
| ResAdaptBackbone=Qwen3-VL-8B, Retention Ratio R=22.9%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 56.3 | |
| ToMeBackbone=Qwen3-VL-8B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 56 | |
| ToMeBackbone=Qwen3-VL-8B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 55.9 | |
| VisionZipBackbone=Qwen3-VL-8B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 55.7 | |
| Random DropBackbone=Qwen3-VL-8B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 55.4 | |
| ResAdaptBackbone=Qwen3-VL-8B, Retention Ratio R=11.1%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 55.4 | |
| ResAdaptBackbone=Qwen3-VL-8B, Retention Ratio R=23.8%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 55.3 | |
| ToMeBackbone=Qwen3-VL-8B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 55.1 | |
| VideoAuto-R1Backbone=Qwen2.5-VL-7B, Retention Ratio R=100%, Reasoning (CoT)=true, Frames=32 Frames2026.03 | 55 | |
| FlashVidBackbone=Qwen3-VL-8B, Retention Ratio R=30.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 54.8 | |
| FlashVidBackbone=Qwen3-VL-8B, Retention Ratio R=12.2%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 54.8 | |
| ResAdaptBackbone=Qwen3-VL-8B, Retention Ratio R=11.4%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 54.6 | |
| VisionZipBackbone=Qwen3-VL-8B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 54.4 | |
| FixedScaleBackbone=Qwen3-VL-8B, Retention Ratio R=12.3%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 54.4 | |
| VisionZipBackbone=Qwen3-VL-8B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 54.2 | |
| FixedScaleBackbone=Qwen3-VL-8B, Retention Ratio R=12.3%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 53.8 | |
| VisionZipBackbone=Qwen3-VL-8B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 53.7 | |
| VideoAuto-R1 + ResAdaptBackbone=Qwen2.5-VL-7B, Retention Ratio R=23.8%, Reasoning (CoT)=true, Frames=128 Frames2026.03 | 53.5 | |
| VideoAuto-R1 + ResAdaptBackbone=Qwen2.5-VL-7B, Retention Ratio R=23.8%, Reasoning (CoT)=true, Frames=32 Frames2026.03 | 53.2 | |
| Random DropBackbone=Qwen3-VL-8B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 53.2 | |
| VanillaBackbone=Qwen2.5-VL-7B, Retention Ratio R=100%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 53.1 | |
| ToMeBackbone=Qwen3-VL-8B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 53.1 | |
| ResAdaptBackbone=Qwen2.5-VL-7B, Retention Ratio R=22.9%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 52.8 | |
| VanillaBackbone=Qwen2.5-VL-7B, Retention Ratio R=100%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 52.7 | |
| VideoAuto-R1 + ResAdaptBackbone=Qwen2.5-VL-7B, Retention Ratio R=11.4%, Reasoning (CoT)=true, Frames=128 Frames2026.03 | 52.4 | |
| ResAdaptBackbone=Qwen2.5-VL-7B, Retention Ratio R=23.8%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 51.9 | |
| ToMeBackbone=Qwen2.5-VL-7B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 51.9 | |
| VideoAuto-R1 + ResAdaptBackbone=Qwen2.5-VL-7B, Retention Ratio R=11.4%, Reasoning (CoT)=true, Frames=32 Frames2026.03 | 51.8 | |
| FixedScaleBackbone=Qwen2.5-VL-7B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 51.2 | |
| FlashVidBackbone=Qwen2.5-VL-7B, Retention Ratio R=29.3%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 51.1 | |
| VisionZipBackbone=Qwen2.5-VL-7B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 51.1 | |
| ToMeBackbone=Qwen2.5-VL-7B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 51 | |
| Random DropBackbone=Qwen2.5-VL-7B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 50.8 | |
| VisionZipBackbone=Qwen2.5-VL-7B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 49.8 | |
| Random DropBackbone=Qwen2.5-VL-7B, Retention Ratio R=25.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 49.6 | |
| FixedScaleBackbone=Qwen2.5-VL-7B, Retention Ratio R=12.3%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 49.6 | |
| ResAdaptBackbone=Qwen2.5-VL-7B, Retention Ratio R=11.4%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 49.2 | |
| ResAdaptBackbone=Qwen2.5-VL-7B, Retention Ratio R=11.1%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 49 | |
| ToMeBackbone=Qwen2.5-VL-7B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 48.9 | |
| FlashVidBackbone=Qwen2.5-VL-7B, Retention Ratio R=10.4%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 47.9 | |
| FixedScaleBackbone=Qwen2.5-VL-7B, Retention Ratio R=12.3%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 47.7 | |
| VisionZipBackbone=Qwen2.5-VL-7B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 47.6 | |
| Random DropBackbone=Qwen2.5-VL-7B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=32 Frames2026.03 | 47.1 | |
| Random DropBackbone=Qwen2.5-VL-7B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 45.8 | |
| VisionZipBackbone=Qwen2.5-VL-7B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 44.8 | |
| ToMeBackbone=Qwen2.5-VL-7B, Retention Ratio R=10.0%, Reasoning (CoT)=false, Frames=128 Frames2026.03 | 44.2 |