Multimodal Evaluation on LLaVA Evaluation Suite 7B v1.5 (test)
61.9GQAVanilla
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| VanillaRetained Tokens=576, Token Percentage=100%2026.05 | 61.9 | 69.5 | 58.2 | 85.9 | 1,862 | 78.5 | 64.7 | 58.3 | 100 | — | — | — | — | — | — | — | |
| VanillaRetain Tokens=576, Pruning Rate=0%2026.06 | 61.9 | 69.5 | 58.2 | 85.9 | 1,862 | 78.5 | 64.7 | 58.1 | — | 100 | — | — | — | — | — | — | |
| Pre-trained ModelBackbone=LLaVA-1.5-7B, Target Task=Flickr30k2026.06 | 61.9 | 70.4 | — | 87.3 | — | 78.5 | — | 58.3 | — | 60.08 | 50 | 58.2 | 64.3 | 66.1 | 11.73 | 19.93 | |
| OccamTokenType=IMG.+QRY., Retained Tokens=1282026.05 | 60.9 | 69.1 | 58 | 86.3 | 1,825 | 77.9 | 63.9 | 57.5 | 99.1 | — | — | — | — | — | — | — | |
| TwigVLMType=LEARNED, Retained Tokens=1282026.05 | 60.6 | 69.5 | 57.8 | 86.6 | 1,818 | 77.9 | 63.5 | — | 99 | — | — | — | — | — | — | — | |
| LearnPrunerType=LEARNED, Retained Tokens=1282026.05 | 60.3 | 68.5 | 57.3 | 86.7 | 1,820 | 77.3 | 63.8 | 56.8 | 98.5 | — | — | — | — | — | — | — | |
| StepPruneRetain Tokens=192, Pruning Rate=66.7%2026.06 | 59.5 | 69.9 | 56.5 | 85.7 | 1,773 | 76.6 | 63.5 | 57.4 | — | 97.9 | — | — | — | — | — | — | |
| CGMBackbone=LLaVA-1.5-7B, Target Task=Flickr30k2026.06 | 59.5 | 69.3 | — | 86.5 | — | 77.1 | — | 52.7 | — | 62.01 | 53.3 | 49.5 | 62.7 | 63.8 | 47.9 | 54.71 | |
| DivPruneType=FIXED, Retained Tokens=1282026.05 | 59.3 | 69 | 56.1 | 86.7 | 1,718 | 76 | 62 | 54.8 | 96.4 | — | — | — | — | — | — | — | |
| OccamTokenType=IMG.+QRY., Retained Tokens=642026.05 | 59.3 | 69 | 57.4 | 86.2 | 1,801 | 77.2 | 63.2 | 56.9 | 98.1 | — | — | — | — | — | — | — | |
| VisionZipRetain Tokens=192, Pruning Rate=66.7%2026.06 | 59.3 | 68.9 | 57.3 | 85.3 | 1,783 | 76.8 | 63 | 57.3 | — | 97.8 | — | — | — | — | — | — | |
| OccamTokenType=IMG.+QRY., Retained Tokens=322026.05 | 59.1 | 69.1 | 56.3 | 86.2 | 1,780 | 75.5 | 62.9 | 56.2 | 97.2 | — | — | — | — | — | — | — | |
| VisionZipType=LEARNED, Retained Tokens=1282026.05 | 58.9 | 68.3 | 57 | 83.7 | 1,823 | 76.6 | 62.6 | — | 97.3 | — | — | — | — | — | — | — | |
| LearnPrunerType=LEARNED, Retained Tokens=642026.05 | 58.9 | 68.3 | 56.6 | 86.8 | 1,750 | 76 | 62.6 | 55.7 | 96.9 | — | — | — | — | — | — | — | |
| PruneSIDType=IMG., Retained Tokens=1282026.05 | 58.8 | 68.5 | 54.7 | 86.5 | 1,749 | 75.3 | 62.1 | — | 96.3 | — | — | — | — | — | — | — | |
| TwigVLMType=LEARNED, Retained Tokens=642026.05 | 58.8 | 70 | 55.8 | 82.7 | 1,760 | 75.6 | 60.4 | — | 96 | — | — | — | — | — | — | — | |
| VisPrunerType=FIXED, Retained Tokens=1282026.05 | 58.2 | 69.1 | 57 | 84.6 | 1,794 | 75.8 | 62.7 | 57.3 | 97.2 | — | — | — | — | — | — | — | |
| PruMerge+Type=IMG., Retained Tokens=1282026.05 | 58.2 | 69.1 | 54 | 83.1 | — | 75 | 61.8 | 55.8 | 95.7 | — | — | — | — | — | — | — | |
| DARTType=FIXED, Retained Tokens=1282026.05 | 57.9 | 69.1 | 56.3 | 80.4 | 1,721 | 74.7 | 60.7 | 57.3 | 95.4 | — | — | — | — | — | — | — | |
| DivPruneType=FIXED, Retained Tokens=642026.05 | 57.8 | 68.2 | 54.7 | 85.6 | 1,674 | 74.1 | 59.3 | 52.3 | 93.9 | — | — | — | — | — | — | — | |
| VisionZipType=FIXED, Retained Tokens=1282026.05 | 57.6 | 68.9 | 56.8 | 83.2 | 1,762 | 75.6 | 62 | 56.7 | 96.3 | — | — | — | — | — | — | — | |
| SparseVLMRetain Tokens=192, Pruning Rate=66.7%2026.06 | 57.6 | 69.1 | 56.1 | 83.6 | 1,721 | 75.6 | 62.5 | 58.6 | — | 96.5 | — | — | — | — | — | — | |
| VisionZipRetain Tokens=128, Pruning Rate=77.8%2026.06 | 57.6 | 68.9 | 56.8 | 83.2 | 1,762 | 75.6 | 62 | 56.7 | — | 96.4 | — | — | — | — | — | — | |
| PDropRetain Tokens=192, Pruning Rate=66.7%2026.06 | 57.3 | 69.2 | 56.5 | 82.3 | 1,797 | 75.1 | 63.6 | 56.8 | — | 96 | — | — | — | — | — | — | |
| StepPruneRetain Tokens=128, Pruning Rate=77.8%2026.06 | 57.3 | 69.8 | 55.3 | 85.2 | 1,749 | 75.7 | 63.2 | 57.1 | — | 96.7 | — | — | — | — | — | — | |
| LearnPrunerType=LEARNED, Retained Tokens=322026.05 | 57.2 | 68.2 | 56.1 | 84.5 | 1,672 | 74 | 60.8 | 55.5 | 94.8 | — | — | — | — | — | — | — | |
| PruneSIDType=IMG., Retained Tokens=642026.05 | 57.1 | 67.8 | 54.2 | 83.8 | 1,733 | 73.7 | 58.8 | — | 94 | — | — | — | — | — | — | — | |
| PDropRetain Tokens=128, Pruning Rate=77.8%2026.06 | 57.1 | 68.4 | 56.6 | 82.3 | 1,761 | 72.9 | 61.6 | 56.6 | — | 94.7 | — | — | — | — | — | — | |
| VisionZipType=LEARNED, Retained Tokens=642026.05 | 57 | 68.8 | 56 | 80.9 | 1,756 | 74.2 | 61.5 | — | 95.1 | — | — | — | — | — | — | — | |
| StepPruneRetain Tokens=64, Pruning Rate=88.9%2026.06 | 56.4 | 69.3 | 54.7 | 83.1 | 1,698 | 72.9 | 61.8 | 55.4 | — | 94.6 | — | — | — | — | — | — | |
| SparseVLMType=FIXED, Retained Tokens=1282026.05 | 56 | 67.1 | 54.9 | 80.5 | 1,696 | 73.8 | 60 | 51.1 | 92.6 | — | — | — | — | — | — | — | |
| SparseVLMRetain Tokens=128, Pruning Rate=77.8%2026.06 | 56 | 67.1 | 54.9 | 80.5 | 1,696 | 73.8 | 60 | 51.1 | — | 92.4 | — | — | — | — | — | — | |
| VisPrunerType=FIXED, Retained Tokens=642026.05 | 55.4 | 69.1 | 55.8 | 80.4 | 1,689 | 72.7 | 61.3 | 55.1 | 93.9 | — | — | — | — | — | — | — | |
| PruMerge+Type=IMG., Retained Tokens=642026.05 | 55.4 | 69.5 | 52 | 75.7 | — | 71.3 | 59.6 | 52.1 | 91.3 | — | — | — | — | — | — | — | |
| VisionZipType=FIXED, Retained Tokens=642026.05 | 55.1 | 69 | 55.5 | 77 | 1,690 | 72.4 | 60.1 | 55.4 | 93 | — | — | — | — | — | — | — | |
| VisionZipRetain Tokens=64, Pruning Rate=88.9%2026.06 | 55.1 | 69 | 55.5 | 77 | 1,690 | 72.4 | 60.1 | 50.4 | — | 92 | — | — | — | — | — | — | |
| DivPruneType=FIXED, Retained Tokens=322026.05 | 54.9 | 68.6 | 52.9 | 81.5 | 1,611 | 71.2 | 57.6 | 49.1 | 90.5 | — | — | — | — | — | — | — | |
| DARTType=FIXED, Retained Tokens=642026.05 | 54.7 | 69.3 | 54.7 | 73.9 | 1,705 | 71.3 | 59.5 | 54 | 91.9 | — | — | — | — | — | — | — | |
| CGM†Backbone=LLaVA-1.5-7B, Target Task=Flickr30k2026.06 | 54 | 69.7 | — | 80.1 | — | 72.4 | — | 55.7 | — | 60.26 | 50.6 | 45 | 63 | 61.3 | 51.91 | 56.22 | |
| DARTType=FIXED, Retained Tokens=322026.05 | 52.9 | 69.3 | 52.2 | 69.1 | 1,615 | 67.1 | 58.5 | 50 | 88 | — | — | — | — | — | — | — | |
| PruMerge+Type=IMG., Retained Tokens=322026.05 | 52.9 | 67.9 | 49.2 | 66.7 | — | 65.6 | 55.1 | 45.9 | 84.7 | — | — | — | — | — | — | — | |
| SparseVLMType=FIXED, Retained Tokens=642026.05 | 52.7 | 62.2 | 51.8 | 75.1 | 1,505 | 68.2 | 56.2 | 46.1 | 85.6 | — | — | — | — | — | — | — | |
| FastVRetain Tokens=192, Pruning Rate=66.7%2026.06 | 52.7 | 67.3 | 52.5 | 64.8 | 1,612 | 67.1 | 61.2 | 53.5 | — | 88.3 | — | — | — | — | — | — | |
| SparseVLMRetain Tokens=64, Pruning Rate=88.9%2026.06 | 52.7 | 62.2 | 51.8 | 75.1 | 1,505 | 68.2 | 56.2 | 46.1 | — | 86.7 | — | — | — | — | — | — | |
| VisPrunerType=FIXED, Retained Tokens=322026.05 | 52.2 | 69.2 | 53.9 | 72.7 | 1,567 | 67.7 | 58.4 | 52.7 | 89 | — | — | — | — | — | — | — | |
| VisionZipType=FIXED, Retained Tokens=322026.05 | 51.8 | 68.8 | 53.1 | 68.7 | 1,536 | 67.1 | 57.7 | 50.3 | 87.2 | — | — | — | — | — | — | — | |
| TailorBackbone=LLaVA-1.5-7B, Target Task=Flickr30k2026.06 | 51.7 | 69.3 | — | 77.6 | — | 70.8 | — | 56.9 | — | 53.92 | 13.2 | 42.9 | 61.1 | 55.4 | 41.85 | 47.69 | |
| FastVType=FIXED, Retained Tokens=1282026.05 | 49.6 | 60.2 | 50.6 | 59.6 | 1,490 | 61.8 | 56.1 | 51.4 | 82.1 | — | — | — | — | — | — | — | |
| FastVRetain Tokens=128, Pruning Rate=77.8%2026.06 | 49.6 | 60.2 | 50.6 | 59.6 | 1,490 | 61.8 | 56.1 | 55.9 | — | 85.2 | — | — | — | — | — | — | |
| WandaBackbone=LLaVA-1.5-7B, Target Task=Flickr30k2026.06 | 49.2 | 67.8 | — | 79.6 | — | 69.8 | — | 52.8 | — | 55.97 | 33.5 | 40 | 60.1 | 56.6 | 50.81 | 53.56 | |
| GraftingBackbone=LLaVA-1.5-7B, Target Task=Flickr30k2026.06 | 49 | 69.9 | — | 67.9 | — | 68.7 | — | 55.5 | — | 51.37 | 14.5 | 41.8 | 64.5 | 54 | 30.56 | 39.02 | |
| MagnitudeBackbone=LLaVA-1.5-7B, Target Task=Flickr30k2026.06 | 48.6 | 69.6 | — | 74.5 | — | 69.3 | — | 53 | — | 54.57 | 22.1 | 40.2 | 61.2 | 54.8 | 52.78 | 53.77 | |
| SparseVLMType=FIXED, Retained Tokens=322026.05 | 48.3 | 57.3 | 46.1 | 67.9 | 1,290 | 58.6 | 51.4 | 40.6 | 76.5 | — | — | — | — | — | — | — | |
| PDropRetain Tokens=64, Pruning Rate=88.9%2026.06 | 47.5 | 69 | 50.6 | 55.9 | 1,561 | 69.2 | 58.8 | 50.5 | — | 82.7 | — | — | — | — | — | — | |
| Fine-tuned ModelBackbone=LLaVA-1.5-7B, Target Task=Flickr30k2026.06 | 46.7 | 67.5 | — | 76.1 | — | 67.8 | — | 50.8 | — | 53.6 | 22.4 | 37.9 | 59.7 | 53.6 | 53.57 | 53.59 | |
| FastVType=FIXED, Retained Tokens=642026.05 | 46.1 | 51.1 | 47.8 | 48 | 1,256 | 55 | 48 | 42.7 | 71.4 | — | — | — | — | — | — | — | |
| FastVRetain Tokens=64, Pruning Rate=88.9%2026.06 | 46.1 | 51.1 | 47.8 | 48 | 1,256 | 55 | 48 | 52.7 | — | 76.8 | — | — | — | — | — | — | |
| DAREBackbone=LLaVA-1.5-7B, Target Task=Flickr30k2026.06 | 43.7 | 67.8 | — | 85.3 | — | 62.4 | — | 45.9 | — | 49.36 | 6.2 | 20.8 | 59.6 | 49 | 52.48 | 50.66 | |
| FastVType=FIXED, Retained Tokens=322026.05 | 41.5 | 42.6 | 42.5 | 32.5 | 1,090 | 43.4 | 37.8 | 33.2 | 58.5 | — | — | — | — | — | — | — |