List your company
ResearchFormerly WizWand

Visual Question Answering on OCR-VQA

75.8AccuracyLyrics
39.19248.69658.267.704Oct 7, 2022Jan 16, 2023Apr 27, 2023Aug 6, 2023Nov 15, 2023Feb 24, 2024Jun 4, 2024
Updated 8mo ago

Evaluation results

MethodLinks
LyricsBackbone=Vicuna-13B, Evaluation protocol=0-shot2023.1275.8
Qwen-VL (Qwen-7B)Model type=Generalist Models, Backbone=Qwen-7B2023.0875.7
QWen-VL-7BLLM=Qwen-7B2023.1175.7
PALI-X-55BModel type=Specialist SOTAs, Configuration=Single-task fine-tuning, without OCR Pipeline, Parameters=55B2023.0875
InfMLLM-7BLLM=Vicuna-7B2023.1173.52
InstructBLIPLLM Backbone=FlanT5-XXL, Evaluation Protocol=finetuning2023.0573.3
InstructBLIPLLM Backbone=Vicuna-7B, Evaluation Protocol=finetuning2023.0572.8
BLIP-2LLM Backbone=FlanT5-XXL, Evaluation Protocol=finetuning2023.0572.7
Pix2Struct-LargeInput type=Pixel-only, Scale=Large2022.1071.3
Pix2Struct-Large (1.3B)Model type=Generalist Models, Parameters=1.3B2023.0871.3
Qwen-VL-ChatModel type=Generalist Models2023.0870.5
Qwen-VL-ChatParam.=9.6B, Res.=448, Data=1.4B, Inference Speed=17.0 t/s, Evaluation Protocol=In-domain2024.0370.5
Qwen-VLBackbone=Qwen-7B, Evaluation protocol=0-shot2023.1270.5
GIT2Input type=Pixel-only, Additional labeled data=true2022.1070.3
GITEvaluation Protocol=finetuning2023.0570.3
Pix2Struct-BaseInput type=Pixel-only, Scale=Base2022.1069.1
BLIP-2LLM Backbone=Vicuna-7B, Evaluation Protocol=finetuning2023.0569.1
LLaVA-HR-XParam.=14B, Res.=1024, Data=1.2M, Inference Speed=12.9 t/s, Evaluation Protocol=In-domain2024.0369
LLaVA-HRParam.=7.4B, Res.=1024, Data=1.2M, Inference Speed=19.7 t/s, Evaluation Protocol=In-domain2024.0368.4
LLaVA-HRParam.=13.4B, Res.=1024, Data=1.2M, Inference Speed=15.0 t/s, Evaluation Protocol=In-domain2024.0367.7
LATrInput type=Pipelined2022.1067.5
DonutInput type=Pixel-only2022.1066
M4CInput type=Pipelined2022.1063.9
VisInContextText Source=Rendered Image2024.0658.4
mPLUG-DocOwlLLM=LLaMA-7B2023.1152.6
Open-Flamingo-9B BaselineText Source=Raw Text2024.0651.5
BLIP-2Param.=14.2B, Res.=224, Data=129M, Evaluation Protocol=In-domain2024.0340.6