List your company
ResearchFormerly WizWand

Text-based Visual Question Answering on TextVQA 52

63.8AccuracyQwen-VL
24.38434.61744.8555.083Jul 5, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Qwen-VLLM=Qwen (7B), Res.=448, PT=1.4B, IT=50M2024.0763.8
Qwen-VL-ChatLM=Qwen (7B), Res.=448, PT=1.4B, IT=50M2024.0761.5
Bunny-3BLM=Phi-2 (2.7B), Res.=384, PT=2M, IT=695K2024.0760.6
LAF-7BLM=Vicuna (7B), Res.=336, PT=558K, IT=665K2024.0760.1
LLaVA-1.5+ (Ours)LM=Vicuna(7B), Res.=336, PT=558K, IT=665K2024.0759.8
Imp-v1LM=Phi-2 (2.7B), Res.=384, PT=558K, IT=665K2024.0759.4
TinyLLaVALM=Phi-2 (2.7B), Res.=384, PT=1.2M, IT=665k2024.0759.1
mPLUG-Owl2LM=LLAMA (7B), Res.=448, PT=400M, IT=1.23M2024.0758.2
LLaVA-1.5LM=Vicuna (7B), Res.=336, PT=558K, IT=665K2024.0758.2
Mipha-3B+ (Ours)LM=Phi-2 (2.7B), Res.=384, PT=558K, IT=665K2024.0757.8
Mobile VLM-v2-3BLM=M-LLaMA (2.7B), Res.=336, PT=1.2M, IT=2.4M2024.0757.5
MoE-LLaVA-3.6BLM=Phi-2 (2.7B), Res.=384, PT=558k, IT=1.59M2024.0757
Mipha-3BLM=Phi-2 (2.7B), Res.=384, PT=558K, IT=665K2024.0756.6
Mobile VLM-v2-1.7BLM=M-LLaMA (1.4B), Res.=336, PT=1.2M, IT=2.4M2024.0752.1
InstructBLIPLM=Vicuna (13B), Res.=224, PT=129M, IT=1.2M2024.0750.7
InstructBLIPLM=Vicuna (7B), Res.=224, PT=129M, IT=1.2M2024.0750.1
LLaVA-PhiLM=Phi-2 (2.7B), Res.=336, PT=558k, IT=665K2024.0748.6
Mobile VLM-3BLM=M-LLaMA (2.7B), Res.=336, PT=558K, IT=665K2024.0747.5
BLIP-2LM=Vicuna (13B), Res.=224, PT=129M, IT=-2024.0742.5
Mobile VLM-1.7BLM=M-LLaMA (1.4B), Res.=336, PT=558K, IT=665K2024.0741.5
MC-LLaVALM=Phi-2 (2.7B), Res.=384, PT=558k, IT=665K2024.0738.6
IDEFICS-80BLM=LLaMA (65B), Res.=224, PT=353M, IT=1M2024.0730.9
IDEFICS-9BLM=LLaMA (7B), Res.=224, PT=353M, IT=1M2024.0725.9