Visual Question Answering on VizWiz (VW, Avg %)
70.4VW ScoreVanilla
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VanillaModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 70.4 | 100 | |
| FastVModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 69.8 | 98.77 | |
| ToFuModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 69.6 | 100.75 | |
| LightKVModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 69.4 | 101.37 | |
| PiToMeModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 69.1 | 100.24 | |
| ToMeModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 68.3 | 100.04 | |
| VanillaBackbone=InternVL2-8B, Retention Rate=100%2026.05 | 61 | 100 | |
| LightKVBackbone=InternVL2-8B, Retention Rate=60%2026.05 | 61 | 100.19 | |
| LightKVBackbone=InternVL2-8B, Retention Rate=55%2026.05 | 61 | 99.58 | |
| ToFuBackbone=InternVL2-8B, Retention Rate=60%2026.05 | 60 | 95.49 | |
| PiToMeBackbone=InternVL2-8B, Retention Rate=60%2026.05 | 60 | 95.99 | |
| ToMe (P)Backbone=InternVL2-8B, Retention Rate=60%2026.05 | 60 | 97.86 | |
| ToMe (P)Backbone=InternVL2-8B, Retention Rate=55%2026.05 | 60 | 95.62 | |
| ToFuBackbone=InternVL2-8B, Retention Rate=55%2026.05 | 60 | 95.82 | |
| LightKVBackbone=LLaVA-NeXT-7B, Compression Timing=During prefill, FLOPs (Tera)=22.3, Memory (GB)=0.67, TTFT (sec)=0.259, Vision Token Retention=55%2026.05 | 58 | 98.85 | |
| PiToMeBackbone=InternVL2-8B, Retention Rate=55%2026.05 | 57 | 95.54 | |
| LightKVBackbone=LLaVA-v1.5-13B, Compression Timing=During prefill, FLOPs (Tera)=12.6, Memory (GB)=0.37, TTFT (sec)=0.098, Vision Token Retention=55%2026.05 | 56 | 99.94 | |
| VanillaBackbone=EVE-7B-v1-HD, Vision token retention rate=55%2026.05 | 56 | — | |
| FastVBackbone=EVE-7B-v1-HD, Vision token retention rate=55%2026.05 | 55 | — | |
| LightKVBackbone=EVE-7B-v1-HD, Vision token retention rate=55%2026.05 | 54 | — | |
| FastVBackbone=InternVL2-8B, Retention Rate=60%2026.05 | 53 | 81.9 | |
| FastVBackbone=InternVL2-8B, Retention Rate=55%2026.05 | 48 | 79.49 | |
| VanillaBackbone=EVE-7B-v1, Vision token retention rate=55%2026.05 | 46 | — | |
| FastVBackbone=EVE-7B-v1, Vision token retention rate=55%2026.05 | 44 | — | |
| LightKVBackbone=EVE-7B-v1, Vision token retention rate=55%2026.05 | 43 | — |