Image Question Answering on OCR-VQA (ROUGE-L)
70.5ROUGE-LFull KV Caching
Evaluation results
| Method | Links | |
|---|---|---|
| Full KV CachingModel=QwenVL-Chat-7B2026.03 | 70.5 | |
| AttentionPackModel=QwenVL-Chat-7B, Rkv=64, Rvv=64, Cache Memory Reduction=2.77×, Average Throughput Change=+61%2026.03 | 68.45 | |
| AttentionPackModel=QwenVL-Chat-7B, Rkv=32, Rvv=32, Cache Memory Reduction=4.02×, Average Throughput Change=+74%2026.03 | 68.02 | |
| H2OModel=QwenVL-Chat-7B, e=50%, Cache Memory Reduction=2×, Average Throughput Change=+51%2026.03 | 67.15 | |
| ScissorHandsModel=QwenVL-Chat-7B, e=50%, Cache Memory Reduction=2×, Average Throughput Change=+51%2026.03 | 67.1 | |
| FastVModel=QwenVL-Chat-7B, k=2, e=50%, Cache Memory Reduction=1×, Average Throughput Change=+45%2026.03 | 66.43 | |
| Full KV CachingModel=LLaVA1.5-13B2026.03 | 55.01 | |
| FastVModel=LLaVA1.5-13B, k=5, e=50%, Cache Memory Reduction=1×, Average Throughput Change=+20%2026.03 | 53.5 | |
| AttentionPackModel=LLaVA1.5-13B, Rkv=64, Rvv=64, Cache Memory Reduction=5.17×, Average Throughput Change=+43%2026.03 | 53.22 | |
| MinicacheModel=LLaVA1.5-13B, Cache Memory Reduction=4.35×, Average Throughput Change=+38%2026.03 | 53.18 | |
| FastVModel=LLaVA1.5-7B, k=5, e=50%, Cache Memory Reduction=1×, Average Throughput Change=+29%2026.03 | 52.62 | |
| H2OModel=LLaVA1.5-13B, e=50%, Cache Memory Reduction=2×, Average Throughput Change=+24%2026.03 | 52.49 | |
| AttentionPackModel=LLaVA1.5-7B, Rkv=64, Rvv=64, Cache Memory Reduction=5.09×, Average Throughput Change=+54%2026.03 | 52.44 | |
| H2OModel=LLaVA1.5-7B, e=50%, Cache Memory Reduction=2×, Average Throughput Change=+32%2026.03 | 52.09 | |
| MinicacheModel=LLaVA1.5-7B, Cache Memory Reduction=4.51×, Average Throughput Change=+44%2026.03 | 51.93 | |
| AttentionPackModel=LLaVA1.5-13B, Rkv=32, Rvv=32, Cache Memory Reduction=7.30×, Average Throughput Change=+50%2026.03 | 51.92 | |
| AttentionPackModel=LLaVA1.5-7B, Rkv=32, Rvv=32, Cache Memory Reduction=7.24×, Average Throughput Change=+65%2026.03 | 51.69 | |
| ScissorHandsModel=LLaVA1.5-13B, e=50%, Cache Memory Reduction=2×, Average Throughput Change=+24%2026.03 | 51.32 | |
| Full KV CachingModel=LLaVA1.5-7B2026.03 | 51.05 | |
| ScissorHandsModel=LLaVA1.5-7B, e=50%, Cache Memory Reduction=2×, Average Throughput Change=+32%2026.03 | 50.08 |