List your company
ResearchFormerly WizWand

Visual Question Answering on ScienceQA Image (test)

96.2AccuracyMutimodal-T-SciQLarge
49.461.5573.785.85Mar 3, 2023May 30, 2023Aug 26, 2023Nov 22, 2023Feb 18, 2024May 16, 2024Aug 13, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Mutimodal-T-SciQLargeEvaluation Protocol=Supervised Fine-tuning, Trainable params=738M2024.0896.2
MC-CoT-F-LargeEvaluation Protocol=Supervised Fine-tuning, Trainable params=738M2024.0894.9
CROME-Vicuna-7BEvaluation Protocol=Supervised Fine-tuning, Trainable params=5.24M2024.0893.2
PILL-7BEvaluation Protocol=Supervised Fine-tuning, Trainable params=45 M2024.0891.2
LaVIN-13BEvaluation Protocol=Supervised Fine-tuning, Trainable params=5.4M2024.0890.8
Prophet++Backbone=mPLUG2023.0390.5
LaVIN-7BEvaluation Protocol=Supervised Fine-tuning, Trainable params=3.8M2024.0889.4
ProphetBackbone=mPLUG2023.0388.2
LLaVA2023.0388
Human Average2023.0387.5
LLaMA-AdapterEvaluation Protocol=Supervised Fine-tuning, Trainable params=1.8M2024.0885.2
MM-COT2023.0382.9
Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Llama3-70B LORA2024.0582.4
Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Yi-34B LORA2024.0580.5
LLaMA-Adapter2023.0380.3
InstructBLIP2023.0379.5
Dense ConnectorPT+IT=1.2M+1.5M, Res.=384 AnyRes, LLM=Yi-34B2024.0578
Dense ConnectorPT+IT=1.2M+1.5M, Res.=384, LLM=Vicuna-13B2024.0577.1
mPLUG2023.0377
Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Llama3-8B2024.0575.2
Dense ConnectorPT+IT=1.2M+1.5M, Res.=384 AnyRes, LLM=Vicuna-13B2024.0575.2
MobileVLM V2PT+IT=1.2M+3.6M, Res.=336, LLM=Vicuna-7B2024.0574.8
SPHINX-PlusPT+IT=16M, Res.=448, LLM=Llama2-13B2024.0574.2
VILAPT+IT=50M+1M, Res.=336, LLM=Llama-2-13B2024.0573.7
LLaVA-NeXTPT+IT=0.5M+0.7M, Res.=336 AnyRes, LLM=Vicuna-13B2024.0573.6
LLaVA-LLaMA3PT+IT=0.5M+0.6M, Res.=336, LLM=Llama3-8B2024.0573.3
Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Vicuna-13B2024.0573
MM1PT+IT=3B+1.4M, Res.=1344, LLM=MM1-7B2024.0572.6
Mini-GeminiPT+IT=1.2M+1.5M, Res.=336+768, LLM=Vicuna-13B2024.0572.6
Dense ConnectorPT+IT=1.2M+1.5M, Res.=384 AnyRes, LLM=Vicuna-7B2024.0572
CuMoPT+IT=0.5M+0.6M, Res.=336, LLM=Mistral-7B2024.0571.7
LLaVA-v1.5PT+IT=0.5M+0.6M, Res.=336, LLM=Vicuna-13B2024.0571.6
ShareGPT4VPT+IT=1.2M+0.7M, Res.=336, LLM=Vicuna-13B2024.0571.2
Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Vicuna-7B2024.0570.5
Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Phi2-2.7B2024.0570.3
LLaVA 1.6-7BEvaluation Protocol=Zero-Shot Performance, Trainable params=7B2024.0870.1
MobileVLM V2PT+IT=1.2M+3.6M, Res.=336, LLM=ML-2.7B2024.0570
LLaMA-VIDPT+IT=0.8M+0.7M, Res.=336, LLM=Vicuna-7B2024.0570
TinyLLaVAPT+IT=0.5M+0.6M, Res.=384, LLM=Phi2-2.7B2024.0569.9
mPLUG-Owl2PT+IT=348M+1.2M, Res.=448, LLM=Llama2-7B2024.0568.7
Qwen-VL-ChatPT+IT=1.4B+50M, Res.=448, LLM=Qwen-7B2024.0568.2
CROME-Vicuna-7BEvaluation Protocol=Zero-Shot Performance, Trainable params=199.85M2024.0861.2
InstructBLIP-Vicuna-7BEvaluation Protocol=Zero-Shot Performance, Trainable params=188M2024.0860.5
BLIVA-Vicuna-7BEvaluation Protocol=Zero-Shot Performance, Trainable params=194.61M2024.0857.3
MCAN2023.0351.2