Multi-image understanding on MileBench (test)
57.3Temporal Multi-Image Score (Task T)FlashCache
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| FlashCacheBackbone MLLM=Qwen2.5-VL-32B, KV Cache Retention Ratio=0.22025.11 | 57.3 | 66.21 | 24.69 | 19.5 | |
| SnapKVBackbone MLLM=Qwen2.5-VL-32B, KV Cache Retention Ratio=0.22025.11 | 57.07 | 65.61 | 22.34 | 18.83 | |
| H2OBackbone MLLM=Qwen2.5-VL-32B, KV Cache Retention Ratio=0.22025.11 | 57.01 | 64.73 | 14.32 | 18 | |
| StreamingLLMBackbone MLLM=Qwen2.5-VL-32B, KV Cache Retention Ratio=0.22025.11 | 56.75 | 63.15 | 3.75 | 19.33 | |
| Full CacheBackbone MLLM=Qwen2.5-VL-32B, KV Cache Retention Ratio=1.02025.11 | 56.51 | 65.21 | 27.19 | 18.17 | |
| Full CacheBackbone MLLM=Qwen2.5-VL-7B, KV Cache Retention Ratio=1.02025.11 | 55.59 | 69.17 | 27.35 | 14.17 | |
| StreamingLLMBackbone MLLM=Qwen2.5-VL-7B, KV Cache Retention Ratio=0.22025.11 | 55.59 | 67.51 | 9.69 | 14 | |
| H2OBackbone MLLM=Qwen2.5-VL-7B, KV Cache Retention Ratio=0.22025.11 | 55.59 | 68.6 | 12.66 | 14.67 | |
| SnapKVBackbone MLLM=Qwen2.5-VL-7B, KV Cache Retention Ratio=0.22025.11 | 55.59 | 68.27 | 13.59 | 15.33 | |
| MEDABackbone MLLM=Qwen2.5-VL-7B, KV Cache Retention Ratio=0.22025.11 | 55.59 | 68.13 | 9.07 | 14.5 | |
| FlashCacheBackbone MLLM=Qwen2.5-VL-7B, KV Cache Retention Ratio=0.22025.11 | 55.59 | 68.85 | 26.72 | 15.5 | |
| LOOK-MBackbone MLLM=Qwen2.5-VL-7B, KV Cache Retention Ratio=0.22025.11 | 55.55 | 67.5 | 11.88 | 11.83 | |
| MEDABackbone MLLM=Qwen2.5-VL-32B, KV Cache Retention Ratio=0.22025.11 | 51.86 | 54.01 | 5.16 | 14.5 | |
| LOOK-MBackbone MLLM=Qwen2.5-VL-32B, KV Cache Retention Ratio=0.22025.11 | 51.03 | 53.37 | 5.78 | 13 | |
| Full CacheBackbone MLLM=LLaVA-OneVision-1.5-8B, KV Cache Retention Ratio=1.02025.11 | 46.97 | 65.33 | 27.03 | 11.67 | |
| StreamingLLMBackbone MLLM=LLaVA-OneVision-1.5-8B, KV Cache Retention Ratio=0.22025.11 | 46.97 | 65.04 | 9.22 | 12.17 | |
| H2OBackbone MLLM=LLaVA-OneVision-1.5-8B, KV Cache Retention Ratio=0.22025.11 | 46.97 | 65.29 | 19.69 | 12.33 | |
| SnapKVBackbone MLLM=LLaVA-OneVision-1.5-8B, KV Cache Retention Ratio=0.22025.11 | 46.97 | 65.27 | 19.37 | 12.17 | |
| LOOK-MBackbone MLLM=LLaVA-OneVision-1.5-8B, KV Cache Retention Ratio=0.22025.11 | 46.97 | 65.23 | 18.6 | 11.83 | |
| FlashCacheBackbone MLLM=LLaVA-OneVision-1.5-8B, KV Cache Retention Ratio=0.22025.11 | 46.97 | 65.46 | 25.94 | 12.83 | |
| MEDABackbone MLLM=LLaVA-OneVision-1.5-8B, KV Cache Retention Ratio=0.22025.11 | 46.91 | 65.08 | 19.97 | 12.33 |