Visual Question Answering on DocVQA (val)
89.2ANLSQwen2-VL-2B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen2-VL-2BParams (B)=2.12024.10 | 89.2 | |
| NSCEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 87.1 | |
| PaLI-XOCR pipeline input=true2023.05 | 86.8 | |
| InternVL2-2BParams (B)=2.12024.10 | 86.2 | |
| GPT-4o-202408062024.10 | 86.1 | |
| AdaMergingEvaluation Protocol=Merged, Token Strategy=Full Token, Backbone=Vicuna 7B, LoRA rank=162026.03 | 85.8 | |
| SVDEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 85.5 | |
| KnOTS-TIESEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 85.5 | |
| KnOTS-DARE-TIESEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 85.3 | |
| TAEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 85.2 | |
| Method [54]OCR pipeline input=true2023.05 | 84.7 | |
| DARE-TIESEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 84.4 | |
| Phi-3-Vision - MicrosoftParams (B)=4.22024.10 | 84.3 | |
| EMR-MergingEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 84.3 | |
| H2OVL-Mississippi-2BParams (B)=2.12024.10 | 83.8 | |
| LoRA-LEGOEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 82.5 | |
| TIESEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 82.3 | |
| RobustMergeEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 82.1 | |
| AdaMergingEvaluation Protocol=Merged, Token Strategy=Single Token, Backbone=Vicuna 7B, LoRA rank=162026.03 | 82.1 | |
| DeepStack-HD+LLM=Vicuna-13B, Visual Tokens=14400, Context Length=2880, Pre-training=748K, Instruction-tuning=748K2024.06 | 81 | |
| PaLI-XOCR pipeline input=false2023.05 | 80 | |
| LinearEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 79.2 | |
| DeepStack-HD+LLM=Vicuna-7B, Visual Tokens=14400, Context Length=2880, Pre-training=558K, Instruction-tuning=748K2024.06 | 78.8 | |
| Vary-baseSFT data=LLaVA-80k2023.12 | 78.2 | |
| Vary-baseSFT data=LLaVA-CC665k2023.12 | 78.1 | |
| LLaVA-NextLLM=Vicuna-13B, Visual Tokens=2880, Context Length=2880, Pre-training=558K, Instruction-tuning=765K2024.06 | 77.5 | |
| Method [27]OCR pipeline input=false2023.05 | 76.6 | |
| FR-MergingEvaluation Protocol=Merged, Backbone=Vicuna 7B, LoRA rank=162026.03 | 76.2 | |
| LLaVA-NextLLM=Vicuna-7B, Visual Tokens=2880, Context Length=2880, Pre-training=558K, Instruction-tuning=765K2024.06 | 74.4 | |
| PaliGemma-3B-mix-448Params (B)=2.92024.10 | 73.9 | |
| Cambrian-13BParams (B)=132024.10 | 73.7 | |
| MiniCPM-V-2Params (B)=2.82024.10 | 69.6 | |
| Vary-toySize=1.8B2024.01 | 65.6 | |
| Qwen-VL-chatSize=7B2024.01 | 65.1 | |
| mPLUG-DocOwlSize=7B2024.01 | 62.2 | |
| VisInContextText Source=Rendered Image2024.06 | 48.5 | |
| Dessurt2023.12 | 46.5 | |
| Dessurt2024.01 | 46.5 | |
| Open-Flamingo-9B BaselineText Source=Raw Text2024.06 | 45.3 | |
| DeepStack-LLLM=Vicuna-13B, Visual Tokens=2880, Context Length=576, Pre-training=558K, Instruction-tuning=665K2024.06 | 43.1 | |
| DeepStack-VLLM=Vicuna-13B, Visual Tokens=2880, Context Length=576, Pre-training=558K, Instruction-tuning=665K2024.06 | 41.7 | |
| DeepStack-VLLM=Vicuna-7B, Visual Tokens=2880, Context Length=576, Pre-training=558K, Instruction-tuning=665K2024.06 | 41 | |
| FinetunedEvaluation Protocol=Finetuned, Backbone=Vicuna 7B, LoRA rank=162026.03 | 40.7 | |
| DeepStack-LLLM=Vicuna-7B, Visual Tokens=2880, Context Length=576, Pre-training=558K, Instruction-tuning=665K2024.06 | 39.3 | |
| LLaVA-1.5LLM=Vicuna-13B, Visual Tokens=576, Context Length=576, Pre-training=558K, Instruction-tuning=665K2024.06 | 30.3 | |
| LLaVA-1.5LLM=Vicuna-7B, Visual Tokens=576, Context Length=576, Pre-training=558K, Instruction-tuning=665K2024.06 | 28.1 | |
| Zero-ShotEvaluation Protocol=Zero-Shot2026.03 | 24.3 |