Multimodal Deep Search on HLE-VL
19Accuracy (HLE-VL)Gemini-2.5 Pro
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini-2.5 ProEvaluation Setting=Direct Reasoning2026.05 | 19 | — | — | — | — | — | — | — | — | — | |
| GPT-5Evaluation Setting=Agent Workflow2026.05 | 18.1 | — | — | — | — | — | — | — | — | — | |
| Gemini-2.5 ProEvaluation Setting=Agent Workflow2026.05 | 17.3 | — | — | — | — | — | — | — | — | — | |
| GPT-5Evaluation Setting=Direct Reasoning2026.05 | 15.8 | — | — | — | — | — | — | — | — | — | |
| WebWatcher-32BEvaluation Setting=Deep Search Agent2026.05 | 13.6 | — | — | — | — | — | — | — | — | — | |
| Gemini-2.5 FlashEvaluation Setting=Direct Reasoning2026.05 | 12 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-8B + ODE-RLEvaluation Setting=Visual-Native Agent Harness, Training Strategy=ODE-RL2026.05 | 11.4 | — | — | — | — | — | — | — | — | — | |
| WebWatcher-7BEvaluation Setting=Deep Search Agent2026.05 | 10.6 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-30B + ODE-RLEvaluation Setting=Visual-Native Agent Harness, Training Strategy=ODE-RL2026.05 | 10.5 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-30B + ODE-SFTEvaluation Setting=Visual-Native Agent Harness, Training Strategy=ODE-SFT2026.05 | 9.9 | — | — | — | — | — | — | — | — | — | |
| Gemini-2.5 FlashEvaluation Setting=Agent Workflow2026.05 | 8.5 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-30BEvaluation Setting=Visual-Native Agent Harness2026.05 | 8.5 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-8B + ODE-SFTEvaluation Setting=Visual-Native Agent Harness, Training Strategy=ODE-SFT2026.05 | 8.2 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-8BEvaluation Setting=Direct Reasoning2026.05 | 6.1 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-8BEvaluation Setting=Visual-Native Agent Harness2026.05 | 6.1 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-30BEvaluation Setting=Direct Reasoning2026.05 | 5 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-8BEvaluation Setting=Agent Workflow2026.05 | 5 | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-30BEvaluation Setting=Agent Workflow2026.05 | 4.4 | — | — | — | — | — | — | — | — | — | |
| Claude-3.7-SonnetProtocol=Direct Inference2026.06 | — | 1.7 | 4.8 | 0 | 2 | 0 | 0 | 0 | 12.3 | 2.8 | |
| Flash-SearcherProtocol=Agent Workflow, Backbone=GPT-4o2026.06 | — | 8.5 | 3.2 | 0 | 3.9 | 8 | 6.7 | 0 | 3.5 | 4.9 | |
| Flash-SearcherProtocol=Agent Workflow, Backbone=Gemini-2.5-Pro2026.06 | — | 11.9 | 9.7 | 6.7 | 3.9 | 4 | 17.8 | 7.1 | 5.2 | 8.8 | |
| Flash-SearcherProtocol=Agent Workflow, Backbone=GPT-52026.06 | — | 18.6 | 14.5 | 0 | 11.5 | 16 | 22.2 | 21.4 | 12.1 | 15.2 | |
| Gemini-2.5-FlashProtocol=Direct Inference2026.06 | — | 12.1 | 1.6 | 0 | 0 | 4 | 0 | 14.3 | 0 | 4.9 | |
| Gemini-2.5-ProProtocol=Direct Inference2026.06 | — | 23.7 | 17.7 | 13.3 | 11.5 | 8 | 13.3 | 14.3 | 15.5 | 15.8 | |
| GPT-4oProtocol=Direct Inference2026.06 | — | 13.8 | 0 | 0 | 3.9 | 12 | 6.8 | 7.1 | 7 | 6.5 | |
| GPT-5Protocol=Direct Inference2026.06 | — | 13.6 | 9.7 | 0 | 1.9 | 8 | 13.3 | 0 | 3.5 | 7.6 | |
| OmniSearchProtocol=Agent Workflow, Backbone=GPT-4o2026.06 | — | 15.5 | 8.2 | 0 | 2.2 | 8 | 6.8 | 21.4 | 12.1 | 9.3 | |
| Qwen2.5-VL-32BProtocol=Direct Inference2026.06 | — | 3.4 | 6.5 | 0 | 3.9 | 8 | 2.3 | 7.1 | 0 | 3.7 | |
| Qwen2.5-VL-72BProtocol=Direct Inference2026.06 | — | 3.4 | 8 | 0 | 5.9 | 8 | 0 | 0 | 7 | 4.9 | |
| Qwen2.5-VL-7BProtocol=Direct Inference2026.06 | — | 3.4 | 3.2 | 7.1 | 0 | 4 | 2.3 | 7.1 | 0 | 2.6 | |
| Struct-SearcherProtocol=Agent Workflow, Backbone=GPT-4o2026.06 | — | 13.6 | 12.9 | 0 | 0 | 0 | 8.9 | 7.1 | 8.6 | 7.9 | |
| Struct-SearcherProtocol=Agent Workflow, Backbone=Gemini-2.5-Pro2026.06 | — | 10.2 | 14.5 | 6.7 | 9.6 | 4 | 22.2 | 21.4 | 8.6 | 12.4 | |
| Struct-SearcherProtocol=Agent Workflow, Backbone=GPT-52026.06 | — | 20.3 | 14.5 | 13.3 | 11.5 | 20 | 22.2 | 28.6 | 15.5 | 17.3 | |
| WebWatcher-32BProtocol=Agent Workflow2026.06 | — | 33.8 | 9.7 | 0 | 5.8 | 8 | 8.9 | 14.3 | 13.8 | 13.6 | |
| WebWatcher-7BProtocol=Agent Workflow2026.06 | — | 18.6 | 6.5 | 6.7 | 7.7 | 4 | 6.7 | 7.1 | 17.2 | 10.6 |