List your company
ResearchFormerly WizWand

Vision Understanding on VizWiz (test)

54.7VizWiz ScoreLLaVA-FastV (k=3, r=0.75)
3.552816.831430.1143.3886Dec 1, 2024Dec 30, 2024Jan 28, 2025Feb 27, 2025Mar 28, 2025Apr 26, 2025May 26, 2025
Updated 8mo ago

Evaluation results

MethodLinks
LLaVA-FastV (k=3, r=0.75)LLM=Vicuna-13B, Res.=336, Token=144 (-75%), TFLOPS=6.0 (-69%)2024.1254.7
LLaVA-1.5-13BLLM=Vicuna-13B, Res.=336, Token=576, TFLOPS=19.62024.1253.6
Dynamic-LLaVA-13BILLM=Vicuna-13B, Res.=336, Token=115 (-80%), TFLOPS=4.7+0.01 (-76%)2024.1253.3
Dynamic-LLaVA-13BITLLM=Vicuna-13B, Res.=336, Token=115 (-80%), TFLOPS=4.7+0.01 (-76%)2024.1253
LLaVA-FastV (k=3, r=0.75)LLM=Vicuna-7B, Res.=336, Token=144 (-75%), TFLOPS=3.2 (-68%)2024.1251.9
Dynamic-LLaVA-7BITLLM=Vicuna-7B, Res.=336, Token=115 (-80%), TFLOPS=2.5+0.01 (-75%)2024.1251.2
Dynamic-LLaVA-7BILLM=Vicuna-7B, Res.=336, Token=115 (-80%), TFLOPS=2.5+0.01 (-75%)2024.1250.2
LLaVA-1.5-7BLLM=Vicuna-7B, Res.=336, Token=576, TFLOPS=10.12024.1250
Qwen2-VL-InstructMode=SFT / Mixture Training2025.0544.09
TIES w/ DAREMerging Method=TIES + DARE2025.0541.63
OptMergeMerging Method=Optimization-based Merging2025.0541.61
Weight AverageMerging Method=Linear Interpolation2025.0541.47
TSV MergingMerging Method=Orthogonalization2025.0541.43
Individual VQAFine-tuning=LoRA2025.0541.38
TIES MergingMerging Method=Sparsification-based2025.0541.38
TA w/ DAREMerging Method=Task Arithmetic + DARE2025.0540.64
Task ArithmeticMerging Method=Task Vector Arithmetic2025.0540.52
Individual GroundingFine-tuning=LoRA2025.0538.6
Individual ChartFine-tuning=LoRA2025.0538.58
WUDI MergingMerging Method=WUDI2025.0537.19
Individual GeometryFine-tuning=LoRA2025.0535.57
Individual OCRFine-tuning=LoRA2025.0528.38
Iso-CMerging Method=SVD-based2025.0512.31
Qwen2-VL-BaseMode=Base Model2025.055.52