Search-based Question Answering on BrowseComp-Plus
83.1AccuracyPI-SERINI (m)
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PI-SERINI (m)LLM=gpt-5.5, Retriever=bm252026.05 | 83.1 | 15.7 | 291.6 | 94.7 | 94.4 | 73.6 | 72.9 | 58.9 | 56.1 | |
| PI-SERINI (i)LLM=gpt-5, Retriever=bm252026.05 | 74.6 | 7.2 | 94.9 | 90.5 | 93.8 | 62.7 | 70 | 45.7 | 56.2 | |
| PI-SERINI (l)LLM=gpt-5.4, Retriever=bm252026.05 | 73.3 | 9.2 | 175.5 | 93.8 | 95.3 | 70.3 | 66.9 | 51.8 | 58.1 | |
| DR-DCI + Workspace-Preserving Context ResetAvg. Tools=38.37, Avg. Turns=34.06, Avg. Wall=176.13s, Cost=$39.352026.06 | 73.25 | — | — | — | — | — | — | — | — | |
| Chen et al. (2025) (d)LLM=gpt-5, Retriever=qwen3-embed-8b2026.05 | 73 | 9.7 | 360.7 | 79 | 81.3 | — | — | — | — | |
| PI-SERINI (o)LLM=deepseek-v4-pro, Retriever=bm252026.05 | 71.4 | 7 | 55.1 | 91.3 | 92.5 | 60 | 63 | 45.4 | 50.6 | |
| DR-DCIAvg. Tools=30.94, Avg. Turns=31.46, Avg. Wall=146.16s, Cost=$34.912026.06 | 71.2 | — | — | — | — | — | — | — | — | |
| PI-SERINI (j)LLM=gpt-5.2, Retriever=bm252026.05 | 70.5 | 6.2 | 122.2 | 89.9 | 92.8 | 60.5 | 67.3 | 44.8 | 54.5 | |
| PI-SERINI (h)LLM=claude-opus-4.7, Retriever=bm252026.05 | 69.8 | 10.2 | 246.6 | 81.2 | 86.8 | 43.3 | 52.5 | 30.4 | 43 | |
| Chen et al. (2026) (f)LLM=Tongyi-DR, Retriever=AgentIR-4B2026.05 | 68.1 | — | — | 79.2 | — | — | — | — | — | |
| PI-SERINI (k)LLM=gpt-5.4-mini, Retriever=bm252026.05 | 68.1 | 13.7 | 86.2 | 91.9 | 94.1 | 60.1 | 65.4 | 43.1 | 52.1 | |
| PI-SERINI (n)LLM=deepseek-v4-flash, Retriever=bm252026.05 | 68.1 | 15.5 | 28.9 | 94.5 | 95.7 | 67.9 | 69.9 | 55.2 | 60.6 | |
| Chen et al. (2025) (b)LLM=o3, Retriever=qwen3-embed-8b2026.05 | 66.3 | 32.7 | 740.8 | 73.2 | 76.3 | — | — | — | — | |
| Raw-DCI / DCI-Agent-LiteAvg. Tools=37.53, Avg. Turns=38.24, Avg. Wall=3139.10s, Cost=$88.132026.06 | 62.9 | — | — | — | — | — | — | — | — | |
| Chen et al. (2025) (c)LLM=gpt-5, Retriever=bm252026.05 | 58.3 | 13.5 | 400.4 | 61.7 | 66.5 | — | — | — | — | |
| PI-SERINI (g)LLM=claude-haiku-4.5, Retriever=bm252026.05 | 54.8 | 17.3 | 193.5 | 94.1 | 95.4 | 58.2 | 60.1 | 40.9 | 46.3 | |
| Chen et al. (2025) (a)LLM=o3, Retriever=bm252026.05 | 50.8 | 39.1 | 836.4 | 56.6 | 61.7 | — | — | — | — | |
| Meng et al. (2026) (e)LLM=gpt-5.2, Retriever=qwen3-embed-8b2026.05 | 45.1 | — | 1 | — | 74.7 | — | — | — | — |