Knowledge-based Visual Question Answering on INFOSEEK (Unseen Entity)
51AccuracyDBAgent (SFT: InfoSeek)
Evaluation results
| Method | Links | |
|---|---|---|
| DBAgent (SFT: InfoSeek)LLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B + bge-m32026.04 | 51 | |
| DBAgent (SFT: Mixed)LLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B + bge-m32026.04 | 50.4 | |
| DBAgent (SFT: E-VQA)LLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B + bge-m32026.04 | 50.2 | |
| ReflectiVALLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B2026.04 | 44.3 | |
| VLM-PRFLLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B2026.04 | 42.7 | |
| VLM-PRFLLM=InternVL3-8B, Retriever=EVA-CLIP-8B2026.04 | 42.1 | |
| ReflectVALLM=LLaMA-3.1-8B, Retriever=EVA-CLIP-8B2026.04 | 39.8 | |
| MMhops-R1Model Size=7B2025.12 | 32.6 | |
| Wiki-LLaVA2025.12 | 27.8 | |
| Wiki-LLaVALLM=LLaVA-v1.5-7B, Retriever=CLIP ViT-L/14 + Contriever2026.04 | 27.8 | |
| RORA-VLMLLM=LLaVA-v1.5-7B, Retriever=CLIP ViT-L/142026.04 | 27.3 | |
| DBAgent (w/o SFT)LLM=Qwen2.5-VL-7B, Retriever=EVA-CLIP-8B + bge-m32026.04 | 24.2 | |
| Qwen2.5-VL-7B (Base)LLM=Qwen2.5-VL-7B (Base), Retriever=–2026.04 | 24.1 | |
| CLIP-FiD2025.12 | 19.1 | |
| CLIP-PaLM2025.12 | 18.5 | |
| GPT-4VLLM=GPT-4V, Retriever=–2026.04 | 14.3 | |
| BLIP-2LLM=Flan-T5XL, Retriever=–2026.04 | 12.3 | |
| LLaVA-v1.5LLM=Vicuna-7B, Retriever=–2026.04 | 9.4 | |
| InstructBLIPLLM=Flan-T5XL, Retriever=–2026.04 | 7.4 |