Document-Oriented Visual Question Answering on DocVQA
94.9AccuracyVanilla
Evaluation Results
| Method | Links | |
|---|---|---|
| VanillaAvg. Tokens=100%2026.02 | 94.9 | |
| IVC-PruneAvg. Tokens=50%2026.02 | 94.4 | |
| FastVAvg. Tokens=52%2026.02 | 93.9 | |
| PDropAvg. Tokens=52%2026.02 | 93.8 | |
| LLaVA-OneVisionSize=72B2024.09 | 93.1 | |
| MaLoRAModel=Qwen3-VL-8B, Training examples=39.5k2025.10 | 92.88 | |
| Oryx-1.5Size=32B2024.09 | 92.7 | |
| BaseModel=Qwen3-VL-8B, Training examples=39.5k2025.10 | 92.67 | |
| LoRAModel=Qwen3-VL-8B, Training examples=39.5k2025.10 | 92.54 | |
| QLoRAModel=Qwen3-VL-8B, Training examples=39.5k2025.10 | 92.05 | |
| MaLoRAModel=Qwen2.5-VL-7B, Training examples=39.5k2025.10 | 91.83 | |
| LoRAModel=Qwen2.5-VL-7B, Training examples=39.5k2025.10 | 91.64 | |
| OryxSize=34B2024.09 | 91.4 | |
| QLoRAModel=Qwen2.5-VL-7B, Training examples=39.5k2025.10 | 90.3 | |
| Oryx-1.5Size=7B2024.09 | 90.1 | |
| OryxSize=7B2024.09 | 89 | |
| Florence-VL 8B# Vis tok.=5762024.12 | 84.9 | |
| Magma-8B (Full)Number of Parameters=8B, Finetuning=true, Pre-training configuration=Full with SoM/ToM2025.02 | 84.8 | |
| FastVLM-1.7BVision Encoder=FastViTHD, LLM=Qw.2, #Visual Tokens=256, Vis. Enc. Size (M)=125, Evaluation Library=lmms-eval2024.12 | 84.2 | |
| Magma-8B (Actw/o)Number of Parameters=8B, Finetuning=true, Pre-training configuration=Action without SoM/ToM2025.02 | 84.1 | |
| LLaVA-NeXTSize=34B2024.09 | 84 | |
| Magma-8B (Fullw/o)Number of Parameters=8B, Finetuning=true, Pre-training configuration=Full without SoM/ToM2025.02 | 83.8 | |
| Florence-VL 3B# Vis tok.=5762024.12 | 82.1 | |
| FlorenceVL-3BVision Encoder=DaViT, LLM=Phi-3, #Visual Tokens=576, Vis. Enc. Size (M)=770, Evaluation Library=lmms-eval2024.12 | 82.1 | |
| MM1.5-1BVision Encoder=ViT-H, #Visual Tokens=1440, Vis. Enc. Size (M)=632, Evaluation Library=lmms-eval2024.12 | 81 | |
| Magma-8B (SFT)Number of Parameters=8B, Finetuning=true, Pre-training configuration=SFT2025.02 | 80.4 | |
| LLaVA-NeXT+CALLLM=Vicuna-13B, Resolution setting=High resolution2024.05 | 80.1 | |
| SmolVLM2-2.2BVision Encoder=ViT-SO400M, LLM=SmolLM2, #Visual Tokens=2106*, Vis. Enc. Size (M)=430, Evaluation Library=lmms-eval2024.12 | 80 | |
| FastVLM-0.6BVision Encoder=FastViTHD, LLM=Qw.2, #Visual Tokens=256, Vis. Enc. Size (M)=125, Evaluation Library=lmms-eval2024.12 | 79.1 | |
| LLaVA-NeXTLLM=Vicuna-13B, Resolution setting=High resolution2024.05 | 78.4 | |
| BaseModel=Qwen2.5-VL-7B, Training examples=39.5k2025.10 | 78.24 | |
| LLaVA-NeXTSize=8B2024.09 | 78.2 | |
| MGM-HD+CALLLM=Vicuna-13B, Resolution setting=High resolution2024.05 | 78 | |
| Cambrian 8B# Vis tok.=5762024.12 | 77.8 | |
| Cambrian-1Size=8B2024.09 | 77.8 | |
| MGM-HDLLM=Vicuna-13B, Resolution setting=High resolution2024.05 | 77.7 | |
| MolmoE A1B-7BVision Encoder=ViT-SO400M, LLM=OLMOE., #Visual Tokens=1728, Vis. Enc. Size (M)=430, Evaluation Library=lmms-eval2024.12 | 77.7 | |
| LLaVA-NeXT+CALLLM=Vicuna-7B, Resolution setting=High resolution2024.05 | 77.3 | |
| Phi 3.5 Vision2024.12 | 75.9 | |
| Cambrian-1Size=34B2024.09 | 75.5 | |
| LLaVA-NeXTLLM=Vicuna-7B, Resolution setting=High resolution2024.05 | 75.1 | |
| Mini-Gemini-HD 8B# Vis tok.=28802024.12 | 74.6 | |
| LLaVA-Next-7BNumber of Parameters=7B, Finetuning=true2025.02 | 74.4 | |
| MGM-HD+CALLLM=Vicuna-7B, Resolution setting=High resolution2024.05 | 73.4 | |
| MGM-HDLLM=Vicuna-7B, Resolution setting=High resolution2024.05 | 72 | |
| SmolVLM2-0.5BVision Encoder=ViT-B, LLM=SmolLM2, #Visual Tokens=1088*, Vis. Enc. Size (M)=93, Evaluation Library=VLMEvalKit2024.12 | 70.5 | |
| LLaVA next 8B# Vis tok.=28802024.12 | 69.3 | |
| MGMLLM=Vicuna-13B, Resolution setting=Low resolution2024.05 | 61.7 | |
| MGM+CALLLM=Vicuna-13B, Resolution setting=Low resolution2024.05 | 61.6 | |
| MGM+CALLLM=Vicuna-7B, Resolution setting=Low resolution2024.05 | 58 | |
| MGMLLM=Vicuna-7B, Resolution setting=Low resolution2024.05 | 57.7 | |
| MonkeyModel Generation Capability=Text only2024.07 | 50.1 | |
| TextHarmony-ChatModel Generation Capability=Text and image, Slide-LoRA=true2024.07 | 49.8 | |
| DocPediaModel Generation Capability=Text only2024.07 | 47.1 | |
| TextHarmonyModel Generation Capability=Text and image, Slide-LoRA=true2024.07 | 47.1 | |
| MGM+CALLLM=Gemma-2B, Resolution setting=Low resolution2024.05 | 44.8 | |
| TextHarmony*Model Generation Capability=Text and image, Slide-LoRA=false2024.07 | 44 | |
| MGMLLM=Gemma-2B, Resolution setting=Low resolution2024.05 | 39.8 | |
| InternLM-XComposer2Model Generation Capability=Text only2024.07 | 39.7 | |
| LLaVA-1.5+CALLLM=Vicuna-13B, Resolution setting=Low resolution2024.05 | 32.6 | |
| LLaVA-1.5LLM=Vicuna-13B, Resolution setting=Low resolution2024.05 | 31.1 | |
| LLaVA-1.5+CALLLM=Vicuna-7B, Resolution setting=Low resolution2024.05 | 30.6 | |
| AIMv2architecture=ViT-3B/14, #patches=576, resolution=336px2024.11 | 30.4 | |
| InternVLModel Generation Capability=Text only2024.07 | 28.7 | |
| LLaVA-1.5LLM=Vicuna-7B, Resolution setting=Low resolution2024.05 | 28.5 | |
| AIMv2architecture=ViT-1B/14, #patches=576, resolution=336px2024.11 | 28.5 | |
| LLaVA-1.5-7BNumber of Parameters=7B, Finetuning=true2025.02 | 28.1 | |
| AIMv2architecture=ViT-H/14, #patches=576, resolution=336px2024.11 | 27.8 | |
| AIMv2architecture=ViT-L/14, #patches=576, resolution=336px2024.11 | 26.6 | |
| OpenAI CLIParchitecture=ViT-L/14, #patches=5762024.11 | 25.6 | |
| MaLoRAModel=LLaVA-1.5-7B, Training examples=39.5k2025.10 | 25.37 | |
| LoRAModel=LLaVA-1.5-7B, Training examples=39.5k2025.10 | 24.45 | |
| QLoRAModel=LLaVA-1.5-7B, Training examples=39.5k2025.10 | 23.7 | |
| BaseModel=LLaVA-1.5-7B, Training examples=39.5k2025.10 | 21.82 | |
| SigLIParchitecture=ViT-So400M/14, #patches=7522024.11 | 19.2 | |
| mPLUG-Owl2Model Generation Capability=Text only2024.07 | 17.9 | |
| SigLIParchitecture=ViT-L/14, #patches=5762024.11 | 16.9 | |
| LLaVARModel Generation Capability=Text only2024.07 | 12.3 | |
| LLaVA1.5-7BModel Generation Capability=Text only2024.07 | 8.5 | |
| DINOv2architecture=ViT-g/14, #patches=30342024.11 | 8.2 | |
| MM-InterleavedModel Generation Capability=Text and image2024.07 | 8.1 | |
| UniDocModel Generation Capability=Text only2024.07 | 7.7 | |
| SEED-LLaMA-14BModel Generation Capability=Text and image2024.07 | 6.5 | |
| MiniGPT5Model Generation Capability=Text and image2024.07 | 1.6 |