Visual Question Answering on OCR-VQA
75.8AccuracyLyrics
Evaluation Results
| Method | Links | |
|---|---|---|
| LyricsBackbone=Vicuna-13B, Evaluation protocol=0-shot2023.12 | 75.8 | |
| Qwen-VL (Qwen-7B)Model type=Generalist Models, Backbone=Qwen-7B2023.08 | 75.7 | |
| QWen-VL-7BLLM=Qwen-7B2023.11 | 75.7 | |
| PALI-X-55BModel type=Specialist SOTAs, Configuration=Single-task fine-tuning, without OCR Pipeline, Parameters=55B2023.08 | 75 | |
| InfMLLM-7BLLM=Vicuna-7B2023.11 | 73.52 | |
| InstructBLIPLLM Backbone=FlanT5-XXL, Evaluation Protocol=finetuning2023.05 | 73.3 | |
| InstructBLIPLLM Backbone=Vicuna-7B, Evaluation Protocol=finetuning2023.05 | 72.8 | |
| BLIP-2LLM Backbone=FlanT5-XXL, Evaluation Protocol=finetuning2023.05 | 72.7 | |
| Pix2Struct-LargeInput type=Pixel-only, Scale=Large2022.10 | 71.3 | |
| Pix2Struct-Large (1.3B)Model type=Generalist Models, Parameters=1.3B2023.08 | 71.3 | |
| Qwen-VL-ChatModel type=Generalist Models2023.08 | 70.5 | |
| Qwen-VL-ChatParam.=9.6B, Res.=448, Data=1.4B, Inference Speed=17.0 t/s, Evaluation Protocol=In-domain2024.03 | 70.5 | |
| Qwen-VLBackbone=Qwen-7B, Evaluation protocol=0-shot2023.12 | 70.5 | |
| GIT2Input type=Pixel-only, Additional labeled data=true2022.10 | 70.3 | |
| GITEvaluation Protocol=finetuning2023.05 | 70.3 | |
| Pix2Struct-BaseInput type=Pixel-only, Scale=Base2022.10 | 69.1 | |
| BLIP-2LLM Backbone=Vicuna-7B, Evaluation Protocol=finetuning2023.05 | 69.1 | |
| LLaVA-HR-XParam.=14B, Res.=1024, Data=1.2M, Inference Speed=12.9 t/s, Evaluation Protocol=In-domain2024.03 | 69 | |
| LLaVA-HRParam.=7.4B, Res.=1024, Data=1.2M, Inference Speed=19.7 t/s, Evaluation Protocol=In-domain2024.03 | 68.4 | |
| LLaVA-HRParam.=13.4B, Res.=1024, Data=1.2M, Inference Speed=15.0 t/s, Evaluation Protocol=In-domain2024.03 | 67.7 | |
| LATrInput type=Pipelined2022.10 | 67.5 | |
| DonutInput type=Pixel-only2022.10 | 66 | |
| M4CInput type=Pipelined2022.10 | 63.9 | |
| VisInContextText Source=Rendered Image2024.06 | 58.4 | |
| mPLUG-DocOwlLLM=LLaMA-7B2023.11 | 52.6 | |
| Open-Flamingo-9B BaselineText Source=Raw Text2024.06 | 51.5 | |
| BLIP-2Param.=14.2B, Res.=224, Data=129M, Evaluation Protocol=In-domain2024.03 | 40.6 |