Video Question Answering on VideoMME (Length Breakdown, ER, CS Metrics)
76.7Accuracy (Short Length)Qwen2.5-VL-7B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-VL-7BSize=7B, Tokens (#)=13447, Tokens (%)=100, TFLOPs (Value)=124.0, TFLOPs (%)=1002026.03 | 76.7 | 68.2 | 56.9 | 67.3 | 100 | 1 | 66.3 | — | |
| FastVIDSize=7B, Tokens (#)=3240, Tokens (%)=24.1, TFLOPs (Value)=23.2, TFLOPs (%)=18.72026.03 | 74.3 | 61.3 | 52.8 | 62.8 | 93.3 | 5.2 | 33 | — | |
| SemVIDSize=7B, Tokens (#)=3222, Tokens (%)=24.0, TFLOPs (Value)=23.1, TFLOPs (%)=18.72026.03 | 73.8 | 62.8 | 52.4 | 63.1 | 93.7 | 5.4 | 30.7 | — | |
| VanillaRetention=100%2026.04 | 71.22 | 56.22 | 48.61 | — | — | — | — | — | |
| HolitomRetention=15%2026.04 | 67.89 | 56.78 | 48 | — | — | — | — | 1.31 | |
| Holitom+SToPRetention=15%2026.04 | 67.89 | 55.67 | 48.89 | — | — | — | — | 0.95 | |
| FastVid+SToPRetention=15%2026.04 | 67.67 | 54.22 | 49.11 | — | — | — | — | 1.18 | |
| VisionZip+SToPRetention=15%2026.04 | 67.33 | 54.89 | 49.33 | — | — | — | — | 1.6 | |
| PruneVIDSize=7B, Tokens (#)=3295, Tokens (%)=24.5, TFLOPs (Value)=23.7, TFLOPs (%)=19.12026.03 | 67 | 59.4 | 51.6 | 59.3 | 88.1 | — | — | — | |
| PruneVidRetention=15%2026.04 | 66.78 | 54.56 | 47.22 | — | — | — | — | 2.32 | |
| Holitom+SToPRetention=10%2026.04 | 66.67 | 55.22 | 48.89 | — | — | — | — | 2.02 | |
| FastVidRetention=15%2026.04 | 65.89 | 53.56 | 48.11 | — | — | — | — | 3.15 | |
| VisionZip+SToPRetention=10%2026.04 | 65.67 | 53.78 | 47.67 | — | — | — | — | 3.34 | |
| FastVid+SToPRetention=10%2026.04 | 65.22 | 54.22 | 48.22 | — | — | — | — | 2.9 | |
| VisionZipRetention=15%2026.04 | 65.11 | 54.56 | 48.11 | — | — | — | — | 3.23 | |
| PruneVidRetention=10%2026.04 | 64.89 | 54.22 | 47.67 | — | — | — | — | 3.17 | |
| HolitomRetention=10%2026.04 | 64.78 | 54.44 | 47.78 | — | — | — | — | 3.21 | |
| FastVidRetention=10%2026.04 | 62.55 | 52.67 | 46.67 | — | — | — | — | 5.12 | |
| VisionZipRetention=10%2026.04 | 61 | 52.89 | 45.78 | — | — | — | — | 7.36 |