List your company
ResearchFormerly WizWand

Multi-modal Understanding on SEED-Bench all (val)

65.6AccuracyLLaVA-Next
41.57647.81354.0560.287Jun 6, 2024
Updated 8mo ago

Evaluation results

MethodLinks
LLaVA-NextLLM=Vicuna-13B, Eff. Res.=672, Vis. Tok.=2880, Cxt. Len.=2880, PT=558K, SFT=765K2024.0665.6
DeepStack-L-HDLLM=Vicuna-13B, Eff. Res.=1344, Vis. Tok.=14400, Cxt. Len.=2880, PT=558K, SFT=748K2024.0665.1
LLaVA-NextLLM=Vicuna-7B, Eff. Res.=672, Vis. Tok.=2880, Cxt. Len.=2880, PT=558K, SFT=765K2024.0664.7
DeepStack-L-HDLLM=Vicuna-7B, Eff. Res.=1344, Vis. Tok.=14400, Cxt. Len.=2880, PT=558K, SFT=748K2024.0663.6
DeepStack-LLLM=Vicuna-13B, Eff. Res.=672, Vis. Tok.=2880, Cxt. Len.=576, PT=558K, SFT=665K2024.0663.5
DeepStack-VLLM=Vicuna-13B, Eff. Res.=672, Vis. Tok.=2880, Cxt. Len.=576, PT=558K, SFT=665K2024.0663
DeepStack-VLLM=Vicuna-7B, Eff. Res.=672, Vis. Tok.=2880, Cxt. Len.=576, PT=558K, SFT=665K2024.0662.3
LLaVA-1.5LLM=Vicuna-13B, Eff. Res.=336, Vis. Tok.=576, Cxt. Len.=576, PT=558K, SFT=665K2024.0661.6
DeepStack-LLLM=Vicuna-7B, Eff. Res.=672, Vis. Tok.=2880, Cxt. Len.=576, PT=558K, SFT=665K2024.0660.6
Qwen-VL-ChatLLM=Qwen-7B, Eff. Res.=448, Vis. Tok.=256, Cxt. Len.=256, PT=1.4B, SFT=50M2024.0658.2
Qwen-VLLLM=Qwen-7B, Eff. Res.=448, Vis. Tok.=256, Cxt. Len.=256, PT=1.4B, SFT=50M2024.0656.3
IDEFICS-9BLLM=LLaMA-7B, Eff. Res.=224, PT=353M, SFT=1M2024.0653.4
InstructBLIPLLM=Vicuna-13B, Eff. Res.=224, Vis. Tok.=32, Cxt. Len.=32, PT=129M, SFT=1.2M2024.0646.4
BLIP-2LLM=Vicuna-13B, Eff. Res.=224, Vis. Tok.=32, Cxt. Len.=32, PT=129M, SFT=-2024.0642.5