Visual Question Answering on InfoSeek (test)
57.9AccuracyQwen2-VL-Oracle
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2-VL-OracleLLM=Qwen2-7B, KB=Wiki, Setting=Zero-shot2025.06 | 57.9 | 57.9 | 57.9 | — | |
| MAD-RAGBackbone=Qwen2.5-VL-3B, Context Setting=Oracle Contexts2026.01 | 54.06 | — | — | — | |
| DoLABackbone=Qwen2.5-VL-3B, Context Setting=Oracle Contexts2026.01 | 52.25 | — | — | — | |
| AdaCADBackbone=Qwen2.5-VL-3B, Context Setting=Oracle Contexts2026.01 | 52.04 | — | — | — | |
| VCDBackbone=Qwen2.5-VL-3B, Context Setting=Oracle Contexts2026.01 | 51.68 | — | — | — | |
| MAD-RAGBackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 51.59 | — | — | — | |
| Vanilla RAGBackbone=Qwen2.5-VL-3B, Context Setting=Oracle Contexts2026.01 | 51.44 | — | — | — | |
| ALFARBackbone=Qwen2.5-VL-3B, Context Setting=Oracle Contexts2026.01 | 51.44 | — | — | — | |
| MAD-RAGBackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 50.19 | — | — | — | |
| MAD-RAGBackbone=Qwen2.5-VL-7B, Context Setting=Oracle Contexts2026.01 | 48.43 | — | — | — | |
| ALFARBackbone=Qwen2.5-VL-7B, Context Setting=Oracle Contexts2026.01 | 48.09 | — | — | — | |
| CADBackbone=Qwen2.5-VL-3B, Context Setting=Oracle Contexts2026.01 | 47.98 | — | — | — | |
| DoLABackbone=Qwen2.5-VL-7B, Context Setting=Oracle Contexts2026.01 | 47.28 | — | — | — | |
| VCDBackbone=Qwen2.5-VL-7B, Context Setting=Oracle Contexts2026.01 | 46.98 | — | — | — | |
| CoRe-MMRAGLLM=Qwen2-7B, KB=Wiki, Setting=Fine-tuned2025.06 | 46.5 | 45.2 | 46.9 | — | |
| AdaCADBackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 46.45 | — | — | — | |
| AdaCADBackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 46.45 | — | — | — | |
| Vanilla RAGBackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 46.33 | — | — | — | |
| AdaCADBackbone=Qwen2.5-VL-7B, Context Setting=Oracle Contexts2026.01 | 46.33 | — | — | — | |
| Vanilla RAGBackbone=Qwen2.5-VL-7B, Context Setting=Oracle Contexts2026.01 | 45.71 | — | — | — | |
| SPINBackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 45.35 | — | — | — | |
| ALFARBackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 45.11 | — | — | — | |
| DoLABackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 44.69 | — | — | — | |
| CADBackbone=Qwen2.5-VL-7B, Context Setting=Oracle Contexts2026.01 | 44.12 | — | — | — | |
| OPERABackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 44.03 | — | — | — | |
| Vanilla RAGBackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 44.01 | — | — | — | |
| VCDBackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 43.86 | — | — | — | |
| VCDBackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 43.61 | — | — | — | |
| OPERABackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 43.03 | — | — | — | |
| DoLABackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 43.01 | — | — | — | |
| Qwen2-VL-1-StageLLM=Qwen2-7B, KB=Wiki, Setting=Fine-tuned2025.06 | 43 | 42.3 | 43.4 | — | |
| CoRe-MMRAGLLM=Qwen2-7B, KB=Wiki, Setting=Zero-shot2025.06 | 42.9 | 42.3 | 43.3 | — | |
| SPINBackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 40.97 | — | — | — | |
| Qwen2-VL-1-StageLLM=Qwen2-7B, KB=Wiki, Setting=Zero-shot2025.06 | 40.9 | 40.8 | 40.9 | — | |
| VHRBackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 40.19 | — | — | — | |
| LLaVA-mR2AGLLM=Vicuna-7B, KB=Wiki, Setting=Fine-tuned2025.06 | 39.4 | 39.1 | 39.7 | — | |
| CADBackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 39.06 | — | — | — | |
| CADBackbone=LLaVA-1.5-13B, Context Setting=Oracle Contexts2026.01 | 39.06 | — | — | — | |
| ALFARBackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 38.7 | — | — | — | |
| VHRBackbone=LLaVA-1.5-7B, Context Setting=Oracle Contexts2026.01 | 38.06 | — | — | — | |
| Qwen2-VL-2-StageLLM=Qwen2-7B, KB=Wiki, Setting=Fine-tuned2025.06 | 37.3 | 36.8 | 37.6 | — | |
| Qwen2-VL-MMSTaRLLM=Qwen2-7B, KB=Wiki, Setting=Fine-tuned2025.06 | 35.1 | 34.8 | 35.2 | — | |
| Qwen2-VL-2-StageLLM=Qwen2-7B, KB=Wiki, Setting=Zero-shot2025.06 | 35 | 34.9 | 35.1 | — | |
| Qwen2-VL-MMSTaRLLM=Qwen2-7B, KB=Wiki, Setting=Zero-shot2025.06 | 33.9 | 33.4 | 34 | — | |
| RA-VQAv2 w/ PreFLMRRAG System=RA-VQAv2, Retriever=PreFLMR2026.02 | 32.1 | — | — | — | |
| V-Retrver-7BModel Scale=7B2026.02 | 31.9 | — | — | — | |
| Wiki-LLaVALLM=Vicuna-7B, KB=Wiki, Setting=Fine-tuned2025.06 | 28.9 | 30.1 | 27.8 | — | |
| LamRA-7BModel Scale=7B2026.02 | 28.8 | — | — | — | |
| VHRBackbone=Qwen2.5-VL-3B, Context Setting=Oracle Contexts2026.01 | 28.16 | — | — | — | |
| EchoSightLLM=LLAMA3-8B, KB=Wiki, Setting=Fine-tuned2025.06 | 27.7 | — | — | — | |
| RORA-VLMLLM=Vicuna-7B, KB=Wiki+Web, Setting=Fine-tuned2025.06 | 26.9 | 27.3 | 25.1 | — | |
| VHRBackbone=Qwen2.5-VL-7B, Context Setting=Oracle Contexts2026.01 | 24.04 | — | — | — | |
| MMA-RAGModel=Qwen2VL-7B2026.02 | 23.9 | — | — | — | |
| RIRModel=Qwen2VL-7B2026.02 | 23.3 | — | — | — | |
| P(true)Model=Qwen2VL-7B2026.02 | 22.6 | — | — | — | |
| Qwen2-VL-ParamLLM=Qwen2-7B, Setting=Zero-shot2025.06 | 22.1 | 23.1 | 21.8 | — | |
| PaLI-XOCR pipeline input=false, Resolution=224x2242023.05 | 21.8 | — | — | — | |
| CLIPModel=Qwen2VL-7B2026.02 | 20.9 | — | — | — | |
| CoTModel=Qwen2VL-7B2026.02 | 20.4 | — | — | — | |
| MMA-RAGModel=Idefics2-8B2026.02 | 20.3 | — | — | — | |
| Zero shotModel=Qwen2VL-7B2026.02 | 19.9 | — | — | — | |
| Closed-book (parametric)Backbone=Qwen2.5-VL-7B, Context Setting=Parametric only2026.01 | 19.58 | — | — | — | |
| MMA-RAGModel=Idefics3-8B2026.02 | 18.1 | — | — | — | |
| Closed-book (parametric)Backbone=Qwen2.5-VL-3B, Context Setting=Parametric only2026.01 | 17.88 | — | — | — | |
| Method [48]OCR pipeline input=false, Resolution=224x2242023.05 | 17.7 | — | — | — | |
| RIRModel=Idefics3-8B2026.02 | 17.5 | — | — | — | |
| RIRModel=Idefics2-8B2026.02 | 17.2 | — | — | — | |
| CLIPModel=Idefics2-8B2026.02 | 16.4 | — | — | — | |
| Few shotModel=Qwen2VL-7B2026.02 | 15.9 | — | — | — | |
| CoTModel=Idefics2-8B2026.02 | 15.5 | — | — | — | |
| Zero shotModel=Idefics2-8B2026.02 | 15.1 | — | — | — | |
| CoTModel=Idefics3-8B2026.02 | 14.3 | — | — | — | |
| Few shotModel=Idefics2-8B2026.02 | 14.2 | — | — | — | |
| P(true)Model=Idefics2-8B2026.02 | 14.1 | — | — | — | |
| P(true)Model=Idefics3-8B2026.02 | 12.9 | — | — | — | |
| CLIPModel=Idefics3-8B2026.02 | 12.6 | — | — | — | |
| Zero shotModel=Idefics3-8B2026.02 | 12.5 | — | — | — | |
| LLaVA-1.5LLM=Vicuna-7B, Setting=Zero-shot2025.06 | 12.2 | 13 | 10.3 | — | |
| Few shotModel=Idefics3-8B2026.02 | 8.5 | — | — | — | |
| Closed-book (parametric)Backbone=LLaVA-1.5-13B, Context Setting=Parametric only2026.01 | 8.14 | — | — | — | |
| Closed-book (parametric)Backbone=LLaVA-1.5-7B, Context Setting=Parametric only2026.01 | 8.01 | — | — | — | |
| EchoSightGenerator Model=Mistral-7B | LLaMA3-8B, Gen. FT=false, Ret. FT=true2026.04 | — | — | — | 31.3 | |
| LLM-RAGenerator Model=BLIP2-Flan-T5XL, Gen. FT=true, Ret. FT=true2026.04 | — | 26.12 | 20.9 | 23.14 | |
| mR2AGGenerator Model=LLaVA-1.5-7B, Gen. FT=true, Ret. FT=true2026.04 | — | 40.6 | 39.8 | 40.2 | |
| OMGMGenerator Model=LLaVA-1.5-7B, Gen. FT=true, Ret. FT=true2026.04 | — | 43.46 | 43.53 | 43.49 | |
| ReflectiVAGenerator Model=LLaVA-MORE-8B, Gen. FT=true, Ret. FT=true2026.04 | — | 40.4 | 39.8 | 40.1 | |
| RoRA-VLMGenerator Model=LLaVA-1.5-7B, Gen. FT=true, Ret. FT=false2026.04 | — | 27.34 | 25.1 | 26.9 | |
| Wiki-LLaVAGenerator Model=LLaVA-1.5-7B, Gen. FT=true, Ret. FT=false2026.04 | — | 30.1 | 27.8 | 28.9 | |
| WikiSeekerGenerator Model=Qwen3-VL-8B-Instruct, Gen. FT=false, Ret. FT=true2026.04 | — | 38.7 | 37.69 | 38.19 | |
| WikiSeekerGenerator Model=Qwen2.5-7B-Instruct, Gen. FT=false, Ret. FT=true2026.04 | — | 35.99 | 34.93 | 35.46 | |
| WikiSeekerGenerator Model=Inspector+Qwen2.5-7B-Instruct, Gen. FT=true, Ret. FT=true2026.04 | — | 43.82 | 45.64 | 44.72 |