Knowledge-based Visual Question Answering on INFOSEEK Unseen Question
46.5AccuracyDBAgent (SFT: InfoSeek)
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| DBAgent (SFT: InfoSeek)LLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B + bge-m32026.04 | 46.5 | — | — | — | |
| DBAgent (SFT: Mixed)LLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B + bge-m32026.04 | 43.6 | — | — | — | |
| ReflectiVALLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B2026.04 | 43.5 | — | — | — | |
| VLM-PRFLLM=InternVL3-8B, Retriever=EVA-CLIP-8B2026.04 | 43.5 | — | — | — | |
| VLM-PRF w/ RLLLM/MLLM=InternVL3-8B, Retriever=EVA-CLIP-8B, Feature=V+T2026.05 | 43.5 | — | — | — | |
| CgRAGLLM/MLLM=InternVL3-8B, Retriever=EVA-CLIP-8B, Feature=V+T2026.05 | 43.5 | — | — | — | |
| VLM-PRFLLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B2026.04 | 43.3 | — | — | — | |
| DBAgent (SFT: E-VQA)LLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B + bge-m32026.04 | 43.1 | — | — | — | |
| CgRAGLLM/MLLM=LLaVA-NeXT-8B, Retriever=EVA-CLIP-8B, Feature=V+T2026.05 | 41.9 | — | — | — | |
| CgRAGLLM/MLLM=LLaVA-NeXT-7B, Retriever=EVA-CLIP-8B, Feature=V+T2026.05 | 41.7 | — | — | — | |
| CgRAGLLM/MLLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B, Feature=V+T2026.05 | 41.3 | — | — | — | |
| CgRAGLLM/MLLM=Qwen2-VL-7B, Retriever=EVA-CLIP-8B, Feature=V+T2026.05 | 40.8 | — | — | — | |
| ReflectVALLM=LLaMA-3.1-8B, Retriever=EVA-CLIP-8B2026.04 | 40.4 | — | — | — | |
| VLM-PRFLLM/MLLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B, Feature=V+T2026.05 | 40 | — | — | — | |
| IBA-LLaVABackbone=Llama-3.1-8B, Category=Our proposed IBA2026.06 | 37.8 | 38.7 | 13.5 | 45.9 | |
| IBA-QwenBackbone=Qwen-2.5-VL, Category=Our proposed IBA2026.06 | 37.2 | 34.5 | 7.8 | 47.5 | |
| MMKB-RAGLLM/MLLM=Qwen2-VL-7B, Retriever=EVA-CLIP-8B, Feature=V2026.05 | 36.4 | — | — | — | |
| ReflectiVABackbone=Llama-3.1-8B, Category=Retrieve Augmented Models requiring Fine-tuning2026.06 | 36.4 | 29.7 | 10.4 | 45.6 | |
| 2Stage-QwenBackbone=Qwen-2.5-VL, Category=Zero-shot with Re-rank2026.06 | 35.9 | 8.4 | 0 | 49 | |
| 1Stage-QwenBackbone=Qwen-2.5-VL, Category=Zero-shot with Re-rank2026.06 | 34.6 | 32.7 | 1.5 | 44.8 | |
| Article-qwenBackbone=Qwen-2.5-VL, Category=Zero-shot with Retrieval2026.06 | 34.5 | 14.3 | 0.2 | 46.8 | |
| MMhops-R1Model Size=7B2025.12 | 33.8 | — | — | — | |
| EchoSightBackbone=Llama-3.1-8B, Category=Retrieve Augmented Models requiring Fine-tuning2026.06 | 33.8 | 24.9 | 12.3 | 41.7 | |
| Wiki-LLaVA2025.12 | 30.1 | — | — | — | |
| Wiki-LLaVALLM=LLaVA-v1.5-7B, Retriever=CLIP ViT-L/14 + Contriever2026.04 | 30.1 | — | — | — | |
| Wiki-LLaVALLM/MLLM=Vicuna-7B, Retriever=CLIP ViT-L/14 + Contriever, Feature=T2026.05 | 30.1 | — | — | — | |
| Wiki-LLaVALLM/MLLM=LLaMA-3.1-8B, Retriever=CLIP ViT-L/14 + Contriever, Feature=T2026.05 | 28.6 | — | — | — | |
| ReflectiVALLM/MLLM=LLaMA-3.1-8B, Retriever=EVA-CLIP-8B, Feature=V2026.05 | 28.6 | — | — | — | |
| 2Stage-llavaBackbone=Llama-3.1-8B, Category=Zero-shot with Re-rank2026.06 | 27.2 | 6.9 | 0.9 | 38.5 | |
| Para-QwenBackbone=Qwen-2.5-VL, Category=Zero-shot base models2026.06 | 25.5 | 10.9 | 0 | 35.5 | |
| RORA-VLMLLM=LLaVA-v1.5-7B, Retriever=CLIP ViT-L/142026.04 | 25.1 | — | — | — | |
| RORA-VLMLLM/MLLM=Vicuna-7B, Retriever=CLIP + Google Search, Feature=V2026.05 | 25.1 | — | — | — | |
| DBAgent (w/o SFT)LLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B + bge-m32026.04 | 24.9 | — | — | — | |
| LLaVA-NeXT-8BMode=Zero-shot MLLMs2026.05 | 23.7 | — | — | — | |
| CLIP-FiD2025.12 | 23.3 | — | — | — | |
| LLaVA-NeXT-7BMode=Zero-shot MLLMs2026.05 | 23.2 | — | — | — | |
| Qwen2.5-VL-7B (Base)LLM=Qwen2.5-VL-7B (Base), Retriever=–2026.04 | 22.8 | — | — | — | |
| Qwen2.5-VL-7BMode=Zero-shot MLLMs2026.05 | 22.8 | — | — | — | |
| CLIP-PaLM2025.12 | 22.7 | — | — | — | |
| Qwen2.5-VL-3BMode=Zero-shot MLLMs2026.05 | 20.2 | — | — | — | |
| Article-llavaBackbone=Llama-3.1-8B, Category=Zero-shot with Retrieval2026.06 | 18.4 | 0 | 0 | 26.4 | |
| EchoSightLLM/MLLM=LLaMA-3.1-8B, Retriever=EVA-CLIP-8B, Feature=V2026.05 | 18 | — | — | — | |
| Qwen2-VL-7BMode=Zero-shot MLLMs2026.05 | 15.4 | — | — | — | |
| GPT-4VLLM=GPT-4V, Retriever=–2026.04 | 15 | — | — | — | |
| GPT-4VMode=Zero-shot MLLMs2026.05 | 15 | — | — | — | |
| BLIP-2LLM=Flan-T5XL, Retriever=–2026.04 | 12.7 | — | — | — | |
| BLIP-2LLM/MLLM=Flan-T5XL, Mode=Zero-shot MLLMs2026.05 | 12.7 | — | — | — | |
| 1Stage-llavaBackbone=Llama-3.1-8B, Category=Zero-shot with Re-rank2026.06 | 10.5 | 0 | 0 | 15 | |
| LLaVA-v1.5LLM=Vicuna-7B, Retriever=–2026.04 | 9.6 | — | — | — | |
| LLaVA-1.5LLM/MLLM=Vicuna-7B, Mode=Zero-shot MLLMs2026.05 | 9.6 | — | — | — | |
| Para-LlavaBackbone=Llama-3.1-8B, Category=Zero-shot base models2026.06 | 9 | 0.9 | 0.5 | 12.4 | |
| InstructBLIPLLM=Flan-T5XL, Retriever=–2026.04 | 8.9 | — | — | — | |
| InstructBLIPLLM/MLLM=Flan-T5XL, Mode=Zero-shot MLLMs2026.05 | 8.9 | — | — | — | |
| LLaVA-1.5LLM/MLLM=LLaMA-3.1-8B, Mode=Zero-shot MLLMs2026.05 | 8.3 | — | — | — |