List your company
ResearchFormerly WizWand

Visual Question Answering on VizWiz (test-dev)

76.4AccuracyCogVLM (Specialist)
17.32832.6644863.336Apr 29, 2022Dec 14, 2022Aug 1, 2023Mar 17, 2024Nov 2, 2024Jun 19, 2025Feb 4, 2026
Updated 8mo ago

Evaluation results

MethodLinks
CogVLM (Specialist)OCR pipeline input=false, Model Category=Specialist, Reference Citation=[58]2023.1276.4
CogVLM (Specialist)OCR pipeline input=true, Model Category=Specialist, Reference Citation=[58]2023.1276.4
SOTA [58]2023.1276.4
SMoLA-PaLI-X_FT (Specialist)Base model=PaLI-X, Per-task LoRA tuning=true, LoRA rank=42023.1275.9
PaLI-XBase model=PaLI-X2023.1274.6
PaLI-17BVocabulary setting=Open-vocabulary generation2022.0974.4
PaLI-17Btrained_on_QA_datasets=true2024.0374.4
SMoLA-PaLI-XFTOCR pipeline input=true, Model Category=Generalist, Backbone=PaLI-X2023.1273.8
SMoLA-PaLI-XFTOCR pipeline input=false, Model Category=Generalist, Backbone=PaLI-X2023.1271.7
PaLI-15BVocabulary setting=Open-vocabulary generation2022.0971.1
SMoLA-PaLI-3FTOCR pipeline input=true, Model Category=Generalist, Backbone=PaLI-32023.1271
GIT2Parameters=5.1B, Vocabulary setting=Closed-vocabulary classification2022.0970.97
GIT2Parameters=5.1B2022.0570.97
SMoLA-PaLI-3FTOCR pipeline input=false, Model Category=Generalist, Backbone=PaLI-32023.1270.4
GITParameters=0.7B, Vocabulary setting=Closed-vocabulary classification2022.0968
GITParameters=0.7B2022.0568
PaLI-3BVocabulary setting=Open-vocabulary generation2022.0967.5
PaLI-3Btrained_on_QA_datasets=true2024.0367.5
Flamingomode=Fine-tuned2022.0465.7
FlamingoEvaluation Protocol=Fine-tuned2022.0465.7
FlamingoParameters=80B, Vocabulary setting=Open-vocabulary generation2022.0965.7
Flamingo FTfine-tuned=true, trained_on_QA_datasets=true2024.0365.7
Flamingo2022.0565.7
GIT_LParameters=0.3B2022.0562.5
Liu et al. (2021)##2022.0561.8
InternVL-ChatLLM=Vicuna-13B, glue layer=QLLAMA2023.1258.5
Unified-IOOCR pipeline input=false, Model Category=Generalist2023.1257.4
SotA2022.0457.2
Unrestricted SotA (Lin et al.)Evaluation Protocol=Fine-tuned2022.0457.2
InternVL-ChatLLM=Vicuna-13B, glue layer=MLP2023.1254.6
GIT_BParameters=0.1B2022.0554.6
LOVA³-7BLLM=Vicuna-7B2024.0554.2
LLaVA-1.5LLM=Vicuna-13B2023.1253.6
FasterVLMReduction Ratio=90%, Vision Tokens=582026.0253
M3Reduction Ratio=75%, Vision Tokens=362026.0252.8
Mask-LLaVAReduction Ratio=97%, Vision Tokens=152026.0252.8
FasterVLMReduction Ratio=95%, Vision Tokens=292026.0252.6
InternVL-ChatLLM=Vicuna-7B, glue layer=MLP2023.1252.5
Mask-LLaVAReduction Ratio=97%, Vision Tokens=422026.0252.1
M3Reduction Ratio=75%, Vision Tokens=92026.0251.9
Mask-LLaVAReduction Ratio=90%, Vision Tokens=572026.0251.8
MQTReduction Ratio=88%, Vision Tokens=642026.0251.5
MQTReduction Ratio=88%, Vision Tokens=362026.0251
FitPruneReduction Ratio=90%, Vision Tokens=582026.0250.8
InstructBLIPLLM=Vicuna-7B2023.1250.1
LLaVA-1.5LLM=Vicuna-7B2023.1250
LLaVA-1.5LLM=Vicuna-7B2024.0550
LLaVA-1.5-7BReduction Ratio=0%, Vision Tokens=5762026.0250
FlamingoFew-shot=32 shots2022.0449.8
Flamingo 32 Shotshots=32, trained_on_QA_datasets=true2024.0349.8
MQTReduction Ratio=88%, Vision Tokens=162026.0249.8
SparseVLMReduction Ratio=90%, Vision Tokens=582026.0249.3
FitPruneReduction Ratio=95%, Vision Tokens=292026.0248.6
FlexCap-LLMzero-shot=true2024.0341.8
Qwen-VL-ChatLLM=Qwen-7B2023.1238.9
SFT-CM3Leon-7Bshots=0-shot, SFT=true2023.0937.6
LOVA³-1.5BLLM=Phi-1.5B2024.0536.9
IDEFICS-80BLLM=LLaMA-65B2023.1236
InstructBLIPzero-shot=true2024.0334.5
Emuzero-shot=true2024.0334.2
LLaVA-Phi-1.5LLM=Phi-1.5B2024.0533.8
Flamingozero-shot=true2024.0331.6
Flamingo-9Bshots=0-shot2023.0928.8
IDEFICS-80B-ILLM=LLaMA-65B2023.1226
BLIP-2LLM=Vicuna-13B2023.1219.6