Multimodal Question Answering on 9 Multimodal Benchmarks Suite (test val)
80VQAv2 AccuracyLLaVA-1.5-13B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| LLaVA-1.5-13BBackbone=LLaVA-1.5-13B, Retained Tokens=5762026.03 | 80 | 63.3 | 53.6 | 72.8 | 61.2 | 86 | 1,531 | 68.5 | 63.5 | 100 | |
| PDropBackbone=LLaVA-1.5-13B, Retained Tokens=1282026.03 | 78.2 | 61 | 53.8 | 73.3 | 60.2 | 83.6 | 1,489 | 67.5 | 62.8 | 98.4 | |
| SparseVLMBackbone=LLaVA-1.5-13B, Retained Tokens=1282026.03 | 77.6 | 59.6 | 51.4 | 74.3 | 59.3 | 85 | 1,488 | 68.4 | 62.6 | 97.8 | |
| AgilePrunerBackbone=LLaVA-1.5-13B, Retained Tokens=1282026.03 | 77.5 | 59.1 | 52.5 | 72.8 | 58.9 | 86.9 | 1,481 | 67.6 | 61.9 | 97.6 | |
| DivPruneBackbone=LLaVA-1.5-13B, Retained Tokens=1282026.03 | 77.1 | 59.2 | 53.5 | 72.8 | 58 | 86.8 | 1,458 | 66.3 | 60.7 | 97 | |
| VisionZipBackbone=LLaVA-1.5-13B, Retained Tokens=1282026.03 | 76.8 | 57.9 | 52.3 | 73.8 | 58.9 | 82.7 | 1,450 | 67.4 | 62.5 | 96.7 | |
| PruMerge+Backbone=LLaVA-1.5-13B, Retained Tokens=1282026.03 | 76.2 | 58.3 | 52.8 | 73.3 | 56.1 | 82.7 | 1,446 | 66.3 | 61.2 | 95.8 | |
| AgilePrunerBackbone=LLaVA-1.5-13B, Retained Tokens=642026.03 | 75.7 | 57.5 | 54.2 | 72 | 58.6 | 82 | 1,437 | 66.2 | 61.6 | 96 | |
| FastVBackbone=LLaVA-1.5-13B, Retained Tokens=1282026.03 | 75.3 | 58.3 | 54.6 | 74.2 | 58.6 | 75.5 | 1,460 | 66.1 | 62.3 | 96 | |
| DivPruneBackbone=LLaVA-1.5-13B, Retained Tokens=642026.03 | 75.2 | 57.9 | 54.4 | 71.7 | 57.4 | 84.5 | 1,454 | 64.1 | 59.8 | 95.6 | |
| VisionZipBackbone=LLaVA-1.5-13B, Retained Tokens=642026.03 | 73.7 | 56.2 | 53.2 | 74.2 | 57.4 | 75.7 | 1,380 | 64.9 | 61.3 | 93.9 | |
| SparseVLMBackbone=LLaVA-1.5-13B, Retained Tokens=642026.03 | 73.2 | 55.9 | 52.1 | 73 | 57.1 | 77.9 | 1,374 | 65.2 | 60.3 | 93.5 | |
| PruMerge+Backbone=LLaVA-1.5-13B, Retained Tokens=642026.03 | 72.6 | 56.3 | 52.4 | 73.5 | 54.4 | 75.7 | 1,338 | 65 | 59.3 | 92.3 | |
| PDropBackbone=LLaVA-1.5-13B, Retained Tokens=642026.03 | 70.8 | 54.1 | 50.5 | 73.1 | 55.3 | 66.1 | 1,247 | 63.1 | 56.6 | 88.7 | |
| FastVBackbone=LLaVA-1.5-13B, Retained Tokens=642026.03 | 65.3 | 51.9 | 53.8 | 73.1 | 53.4 | 56.9 | 1,246 | 59.2 | 55.1 | 85.8 |