Multimodal Large Language Model Inference Efficiency on LLaVA 13B 1.5 (test)
60.2TTFT (ms)VisionZip
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VisionZipReduction=95%, # Tokens=282026.02 | 60.2 | 2.6 | 2.49 | 61 | 26.2 | 702.2 | |
| VisionZipReduction=90%, # Tokens=572026.02 | 62.4 | 2.51 | 3.23 | 84 | 26.2 | 706.6 | |
| ConsensusDropReduction=95%, # Tokens=282026.02 | 64.3 | 2.43 | 2.49 | 61 | 26.4 | 706.1 | |
| VisPrunerReduction=90%, # Tokens=572026.02 | 64.4 | 2.43 | 3.6 | 84 | 25.37 | 601.7 | |
| VisPrunerReduction=95%, # Tokens=282026.02 | 64.5 | 2.43 | 2.86 | 61 | 25.37 | 589.2 | |
| ConsensusDropReduction=90%, # Tokens=572026.02 | 66.5 | 2.35 | 3.23 | 84 | 26.4 | 710.6 | |
| VisPrunerReduction=75%, # Tokens=1442026.02 | 70.5 | 2.22 | 5.83 | 152 | 25.41 | 639 | |
| SparseVLMReduction=95%, # Tokens=282026.02 | 75.7 | 2.07 | 2.61 | 93 | 32.31 | 944.4 | |
| SparseVLMReduction=90%, # Tokens=572026.02 | 77.3 | 2.02 | 2.99 | 109 | 32.28 | 948.3 | |
| VisionZipReduction=75%, # Tokens=1442026.02 | 80.6 | 1.94 | 5.45 | 152 | 26.2 | 724.4 | |
| ConsensusDropReduction=75%, # Tokens=1442026.02 | 82.5 | 1.9 | 5.45 | 152 | 26.4 | 729.1 | |
| SparseVLMReduction=75%, # Tokens=1442026.02 | 83.9 | 1.87 | 4.19 | 155 | 32.3 | 952.8 | |
| VisPrunerReduction=50%, # Tokens=2882026.02 | 84.7 | 1.85 | 9.52 | 264 | 25.64 | 700.9 | |
| SparseVLMReduction=25%, # Tokens=4322026.02 | 92 | 1.7 | 10.61 | 334 | 32.4 | 960.7 | |
| SparseVLMReduction=50%, # Tokens=2882026.02 | 93.1 | 1.68 | 6.75 | 234 | 32.39 | 962.7 | |
| VisionZipReduction=50%, # Tokens=2882026.02 | 103.5 | 1.51 | 9.13 | 264 | 26.22 | 752.6 | |
| ConsensusDropReduction=50%, # Tokens=2882026.02 | 106.7 | 1.47 | 9.13 | 264 | 26.42 | 758.9 | |
| VisPrunerReduction=25%, # Tokens=4322026.02 | 112.9 | 1.39 | 13.24 | 377 | 25.87 | 762.7 | |
| VisionZipReduction=25%, # Tokens=4322026.02 | 135.1 | 1.16 | 12.84 | 377 | 26.47 | 792.5 | |
| ConsensusDropReduction=25%, # Tokens=4322026.02 | 138.8 | 1.13 | 12.84 | 377 | 26.66 | 800.4 | |
| BaselineReduction=0%, # Tokens=5762026.02 | 156.5 | 1 | 16.56 | 489 | 26.7 | 824.6 |