List your company
ResearchFormerly WizWand

Multimodal Understanding on MMMU (Accuracy, Relative Performance)

38AccuracyVisionZip ‡
34.56835.45936.3537.241Dec 5, 2024Jan 17, 2025Mar 2, 2025Apr 15, 2025May 29, 2025Jul 12, 2025Aug 25, 2025
Updated 8mo ago

Evaluation results

MethodLinks
VisionZip ‡Retained Tokens=320, Fine-tuning=true2024.1238108.3
VisionZip ‡Retained Tokens=160, Fine-tuning=true2024.1237.7107.4
VisionZip ‡Retained Tokens=640, Fine-tuning=true2024.1237.2106
MMTokTokens Retained=192, Compression Ratio=↓ 66.7%2025.0836.78101
DivPruneTokens Retained=192, Compression Ratio=↓ 66.7%2025.0836.56100.4
LLaVA-1.5 Vanilla 13BTokens Retained=576, Compression Ratio=100%2025.0836.4100
VisionZipTokens Retained=192, Compression Ratio=↓ 66.7%2025.0836.4100
VisionZip 🔥Tokens Retained=192, Compression Ratio=↓ 66.7%2025.0836.4100
VisionZipTokens Retained=64, Compression Ratio=↓ 88.9%2025.0836.4100
VisionZipRetained Tokens=160, Fine-tuning=false2024.1236.1102.8
VisionZipTokens Retained=128, Compression Ratio=↓ 77.8%2025.0836.199.2
DivPruneTokens Retained=128, Compression Ratio=↓ 77.8%2025.0835.5697.7
MMTokTokens Retained=128, Compression Ratio=↓ 77.8%2025.0835.4497.4
VisionZip 🔥Tokens Retained=128, Compression Ratio=↓ 77.8%2025.0835.497.3
VisionZipRetained Tokens=320, Fine-tuning=false2024.1235.3100.5
VisionZip 🔥Tokens Retained=64, Compression Ratio=↓ 88.9%2025.0835.397
DivPruneTokens Retained=64, Compression Ratio=↓ 88.9%2025.0835.2296.8
MMTokTokens Retained=64, Compression Ratio=↓ 88.9%2025.0835.2296.8
VanillaRetained Tokens=2880, Fine-tuning=false2024.1235.1100
VisionZipRetained Tokens=640, Fine-tuning=false2024.1234.798.9