Visual Question Answering on GQA (Accuracy, Relative Performance)
65.4AccuracyVanilla 13B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Vanilla 13BModel Scale=13B, Visual Tokens=2880, Multimodal Projector Fine-tuning=false2024.12 | 65.4 | 100 | |
| VanillaRetained Tokens=2880, Fine-tuning=false2024.12 | 64.2 | 100 | |
| Vanilla 7BModel Scale=7B, Visual Tokens=2880, Multimodal Projector Fine-tuning=false2024.12 | 64.2 | 98.2 | |
| VisionZip ‡Model Scale=13B, Visual Tokens=640, Multimodal Projector Fine-tuning=true2024.12 | 63.7 | 97.4 | |
| LLaVA-1.5LLM=Vicuna-13B2026.03 | 63.3 | — | |
| iGVLMLLM=Vicuna-13B2026.03 | 63.3 | — | |
| QA-ViTLLM=Vicuna-13B2026.03 | 63.2 | — | |
| VisionZipModel Scale=13B, Visual Tokens=640, Multimodal Projector Fine-tuning=false2024.12 | 63 | 96.3 | |
| iGVLMLLM=Vicuna-7B2026.03 | 62.8 | — | |
| VisionZip ‡Model Scale=13B, Visual Tokens=320, Multimodal Projector Fine-tuning=true2024.12 | 62.5 | 95.6 | |
| VisionZip ‡Retained Tokens=640, Fine-tuning=true2024.12 | 62.4 | 97.2 | |
| Vanilla 7BBase Model=mini-Gemini 7B, Number of Tokens=576, Protocol=Upper Bound2024.12 | 62.4 | 100 | |
| LLaVA-1.5LLM=Qwen2.5-3B2026.03 | 62.4 | — | |
| QA-ViTLLM=Vicuna-7B2026.03 | 62.2 | — | |
| LLaVA-1.5LLM=Vicuna-7B2026.03 | 62 | — | |
| iGVLMLLM=Qwen2.5-3B2026.03 | 61.7 | — | |
| VisionZipBase Model=mini-Gemini 7B, Number of Tokens=192, Fine-tuning Protocol=Fine-tuned multimodal projector2024.12 | 61.6 | 98.7 | |
| VisionZipRetained Tokens=640, Fine-tuning=false2024.12 | 61.3 | 95.5 | |
| VisionZip ‡Retained Tokens=320, Fine-tuning=true2024.12 | 61 | 95 | |
| VisionZipModel Scale=13B, Visual Tokens=320, Multimodal Projector Fine-tuning=false2024.12 | 60.7 | 92.8 | |
| VisionZipBase Model=mini-Gemini 7B, Number of Tokens=1922024.12 | 60.3 | 96.6 | |
| VisionZipBase Model=mini-Gemini 7B, Number of Tokens=128, Fine-tuning Protocol=Fine-tuned multimodal projector2024.12 | 60 | 96.2 | |
| VisionZip ‡Model Scale=13B, Visual Tokens=160, Multimodal Projector Fine-tuning=true2024.12 | 59.7 | 91.3 | |
| VisionZipRetained Tokens=320, Fine-tuning=false2024.12 | 59.3 | 92.3 | |
| VisionZipBase Model=mini-Gemini 7B, Number of Tokens=1282024.12 | 58.7 | 94.1 | |
| VisionZip ‡Retained Tokens=160, Fine-tuning=true2024.12 | 58.2 | 90.7 | |
| VisionZipModel Scale=13B, Visual Tokens=160, Multimodal Projector Fine-tuning=false2024.12 | 57.8 | 88.4 | |
| VisionZipBase Model=mini-Gemini 7B, Number of Tokens=64, Fine-tuning Protocol=Fine-tuned multimodal projector2024.12 | 57.7 | 92.5 | |
| VisionZipBase Model=mini-Gemini 7B, Number of Tokens=642024.12 | 55.8 | 89.4 | |
| VisionZipRetained Tokens=160, Fine-tuning=false2024.12 | 55.5 | 86.4 |