Question Answering on BrowseComp-Plus
88.33Accuracy (Avg)Perfect Retrieval
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Perfect RetrievalExecutive Model=Gemini-3-Flash2026.05 | 88.33 | — | — | — | — | |
| Perfect RetrievalExecutive Model=Qwen2.5-32B-Instruct2026.05 | 79.67 | — | — | — | — | |
| MEMOExecutive Model=Gemini-3-Flash, MEMORY model=Qwen2.5-14B-Instruct2026.05 | 66.67 | — | — | — | — | |
| HippoRAG2Executive Model=Gemini-3-Flash2026.05 | 66.33 | — | — | — | — | |
| NV-Embed-V2Executive Model=Gemini-3-Flash2026.05 | 57 | — | — | — | — | |
| HippoRAG2Executive Model=Qwen2.5-32B-Instruct2026.05 | 56.11 | — | — | — | — | |
| MEMOExecutive Model=Qwen2.5-32B-Instruct, MEMORY model=Qwen2.5-14B-Instruct2026.05 | 54.22 | — | — | — | — | |
| NV-Embed-V2Executive Model=Qwen2.5-32B-Instruct2026.05 | 50.67 | — | — | — | — | |
| BM25Executive Model=Gemini-3-Flash2026.05 | 27 | — | — | — | — | |
| BACM-RLBackbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 14.7 | 52 | 16.4 | 4.2 | — | |
| SummaryBackbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 13.7 | 48 | 16.4 | 1.9 | — | |
| ReACTBackbone=Qwen3-235B-A22B-Inst-2507, Context=128k2026.04 | 13.6 | 38 | 16.4 | 3.4 | — | |
| MEM1Backbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 13.1 | 36 | 14.1 | 6.9 | — | |
| ReACTBackbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 13 | 44 | 14.8 | 3.4 | — | |
| Search-R1(RL)Backbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 12.8 | 42 | 15 | 2.7 | — | |
| BACM-RLBackbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 12.7 | 42 | 14.6 | 3.1 | — | |
| ReACTBackbone=Qwen3-235B-A22B-Inst-2507, Context=8k2026.04 | 11.8 | 42 | 14.3 | 1.1 | — | |
| Search-R1(RL)Backbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 9.9 | 40 | 11.1 | 1.5 | — | |
| ReACTBackbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 8.9 | 32 | 9.8 | 2.7 | — | |
| BACM (w/o RL)Backbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 8 | 20 | 10 | 1.5 | — | |
| SummaryBackbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 7.8 | 34 | 8.5 | 1.5 | — | |
| MEM1(RL)Backbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 3.5 | 4 | 4.4 | 1.5 | — | |
| BACM (w/o RL)Backbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 3.1 | 16 | 2.7 | 1.5 | — | |
| BM25Executive Model=Qwen2.5-32B-Instruct2026.05 | 1.11 | — | — | — | — | |
| CartridgesExecutive Model=Qwen2.5-32B-Instruct2026.05 | 0 | — | — | — | — | |
| Adaptive-RAGBudget Goal=50%2026.05 | — | — | — | — | 24.8 | |
| Best Static Config. (Murakkab)Budget Goal=50%2026.05 | — | — | — | — | 52.05 | |
| BRANEBudget Goal=50%2026.05 | — | — | — | — | 62 | |
| CARROT-KNN (Extended)Budget Goal=50%2026.05 | — | — | — | — | 40.1 | |
| CARROT-KNN (LLM-only)Budget Goal=50%2026.05 | — | — | — | — | 22.3 | |
| CARROT-RoBERTa (Extended)Budget Goal=50%2026.05 | — | — | — | — | 43.1 | |
| CARROT-RoBERTa (LLM-only)Budget Goal=50%2026.05 | — | — | — | — | 23.8 | |
| METISBudget Goal=50%2026.05 | — | — | — | — | 47 |