Vision Understanding on VizWiz (test)
54.7VizWiz ScoreLLaVA-FastV (k=3, r=0.75)
Evaluation Results
| Method | Links | |
|---|---|---|
| LLaVA-FastV (k=3, r=0.75)LLM=Vicuna-13B, Res.=336, Token=144 (-75%), TFLOPS=6.0 (-69%)2024.12 | 54.7 | |
| LLaVA-1.5-13BLLM=Vicuna-13B, Res.=336, Token=576, TFLOPS=19.62024.12 | 53.6 | |
| Dynamic-LLaVA-13BILLM=Vicuna-13B, Res.=336, Token=115 (-80%), TFLOPS=4.7+0.01 (-76%)2024.12 | 53.3 | |
| Dynamic-LLaVA-13BITLLM=Vicuna-13B, Res.=336, Token=115 (-80%), TFLOPS=4.7+0.01 (-76%)2024.12 | 53 | |
| LLaVA-FastV (k=3, r=0.75)LLM=Vicuna-7B, Res.=336, Token=144 (-75%), TFLOPS=3.2 (-68%)2024.12 | 51.9 | |
| Dynamic-LLaVA-7BITLLM=Vicuna-7B, Res.=336, Token=115 (-80%), TFLOPS=2.5+0.01 (-75%)2024.12 | 51.2 | |
| Dynamic-LLaVA-7BILLM=Vicuna-7B, Res.=336, Token=115 (-80%), TFLOPS=2.5+0.01 (-75%)2024.12 | 50.2 | |
| LLaVA-1.5-7BLLM=Vicuna-7B, Res.=336, Token=576, TFLOPS=10.12024.12 | 50 | |
| Qwen2-VL-InstructMode=SFT / Mixture Training2025.05 | 44.09 | |
| TIES w/ DAREMerging Method=TIES + DARE2025.05 | 41.63 | |
| OptMergeMerging Method=Optimization-based Merging2025.05 | 41.61 | |
| Weight AverageMerging Method=Linear Interpolation2025.05 | 41.47 | |
| TSV MergingMerging Method=Orthogonalization2025.05 | 41.43 | |
| Individual VQAFine-tuning=LoRA2025.05 | 41.38 | |
| TIES MergingMerging Method=Sparsification-based2025.05 | 41.38 | |
| TA w/ DAREMerging Method=Task Arithmetic + DARE2025.05 | 40.64 | |
| Task ArithmeticMerging Method=Task Vector Arithmetic2025.05 | 40.52 | |
| Individual GroundingFine-tuning=LoRA2025.05 | 38.6 | |
| Individual ChartFine-tuning=LoRA2025.05 | 38.58 | |
| WUDI MergingMerging Method=WUDI2025.05 | 37.19 | |
| Individual GeometryFine-tuning=LoRA2025.05 | 35.57 | |
| Individual OCRFine-tuning=LoRA2025.05 | 28.38 | |
| Iso-CMerging Method=SVD-based2025.05 | 12.31 | |
| Qwen2-VL-BaseMode=Base Model2025.05 | 5.52 |