Multiple-choice question answering on MLVU
64.66AccuracyVanilla
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VanillaRetention=100%2026.04 | 64.66 | — | |
| Holitom+SToPRetention=15%2026.04 | 63.75 | 0.95 | |
| FastVid+SToPRetention=15%2026.04 | 63.43 | 1.18 | |
| VisionZip+SToPRetention=15%2026.04 | 62.93 | 1.6 | |
| HolitomRetention=15%2026.04 | 62.74 | 1.31 | |
| PruneVidRetention=15%2026.04 | 62.65 | 2.32 | |
| VisionZipRetention=15%2026.04 | 62.65 | 3.23 | |
| FastVidRetention=15%2026.04 | 62.55 | 3.15 | |
| Holitom+SToPRetention=10%2026.04 | 62.42 | 2.02 | |
| VisionZip+SToPRetention=10%2026.04 | 61.96 | 3.34 | |
| PruneVidRetention=10%2026.04 | 61.78 | 3.17 | |
| HolitomRetention=10%2026.04 | 61.64 | 3.21 | |
| FastVid+SToPRetention=10%2026.04 | 61.59 | 2.9 | |
| FastVidRetention=10%2026.04 | 61.09 | 5.12 | |
| VisionZipRetention=10%2026.04 | 60.76 | 7.36 |