Multi-hop Question Answering on MMhops Bridging (test)
58.8String Success RateGemini-2.5-pro
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemini-2.5-pro2025.12 | 58.8 | 50.83 | 57.32 | 53.98 | |
| Gemini-2.5-flash2025.12 | 51.08 | 43.51 | 50.1 | 46.58 | |
| MMhops-R1Base Model=Qwen2.5-vl-7B-Instruct, Retriever=Image, Text2025.12 | 44.66 | 55.33 | 47.94 | 51.35 | |
| GPT-4o2025.12 | 41.66 | 33.6 | 39.28 | 36.62 | |
| Zero-shotBase Model=Qwen2.5-vl-72B-Instruct2025.12 | 37.51 | 32.11 | 37.32 | 34.39 | |
| OmniSearchBase Model=GPT-4o, Retriever=Image, Text2025.12 | 31.02 | 49.77 | 36.5 | 42.65 | |
| GPT-4o-mini2025.12 | 29.67 | 20.54 | 26.08 | 23.8 | |
| Self-AskBase Model=GPT-4o, Retriever=Caption, Text2025.12 | 27.59 | 31.41 | 31.13 | 30.42 | |
| Vanilla mRAGBase Model=Qwen2.5-vl-7B-Instruct, Retriever=Image, Text2025.12 | 26.52 | 25.68 | 28.97 | 26.49 | |
| Zero-shotBase Model=Qwen2.5-vl-7B-Instruct2025.12 | 24.21 | 15.24 | 26.6 | 19.53 | |
| EchoSightBase Model=LLaMA3, Retriever=Image, Text2025.12 | 19.14 | 11.83 | 11.86 | 13.63 | |
| Search-r1Base Model=Qwen2.5-7b-Instruct, Retriever=Caption, Text2025.12 | 14.45 | 23.05 | 17.85 | 19.98 | |
| Vanilla mRAGBase Model=Qwen2.5-vl-7B-Instruct, Retriever=Text2025.12 | 14.37 | 14.65 | 14.95 | 14.63 |