Deep Search on Deep Search Tasks (test)
91.3Pass@1SALE w/o memory
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SALE w/o memoryTask complexity (tau(t))=<= 0.12026.02 | 91.3 | 0.24 | |
| SALETask complexity (tau(t))=<= 0.12026.02 | 91.3 | 0.22 | |
| SALETask complexity (tau(t))=<= 0.52026.02 | 88.5 | 0.22 | |
| Best single agentTask complexity (tau(t))=<= 0.12026.02 | 87.5 | 0.36 | |
| Best single agentTask complexity (tau(t))=<= 0.52026.02 | 87.5 | 0.36 | |
| SALE w/o memoryTask complexity (tau(t))=<= 0.52026.02 | 87.5 | 0.24 | |
| TO-RouterTask complexity (tau(t))=<= 0.12026.02 | 86.3 | 0.28 | |
| FrugalGPTTask complexity (tau(t))=<= 0.12026.02 | 86.3 | 0.47 | |
| WTPTask complexity (tau(t))=<= 0.52026.02 | 86.3 | 0.33 | |
| CARROTTask complexity (tau(t))=<= 0.52026.02 | 86.3 | 0.28 | |
| TO-RouterTask complexity (tau(t))=<= 0.52026.02 | 86.3 | 0.32 | |
| CARROTTask complexity (tau(t))=<= 0.12026.02 | 85 | 0.27 | |
| WTPTask complexity (tau(t))=<= 0.12026.02 | 83.8 | 0.32 | |
| FrugalGPTTask complexity (tau(t))=<= 0.52026.02 | 81.3 | 0.48 | |
| SALETask complexity (tau(t))=<= 2.52026.02 | 73.5 | 0.23 | |
| SALE w/o memoryTask complexity (tau(t))=<= 2.52026.02 | 72.5 | 0.25 | |
| Best single agentTask complexity (tau(t))=<= 2.52026.02 | 68.8 | 0.36 | |
| WTPTask complexity (tau(t))=<= 2.52026.02 | 67.5 | 0.31 | |
| TO-RouterTask complexity (tau(t))=<= 2.52026.02 | 67.5 | 0.34 | |
| SALETask complexity (tau(t))=All2026.02 | 67.3 | 0.21 | |
| SALE w/o memoryTask complexity (tau(t))=All2026.02 | 66.4 | 0.24 | |
| CARROTTask complexity (tau(t))=<= 2.52026.02 | 66.3 | 0.29 | |
| FrugalGPTTask complexity (tau(t))=<= 2.52026.02 | 66.3 | 0.53 | |
| Best single agentTask complexity (tau(t))=All2026.02 | 63.8 | 0.36 | |
| TO-RouterTask complexity (tau(t))=All2026.02 | 63 | 0.33 | |
| WTPTask complexity (tau(t))=All2026.02 | 62.4 | 0.32 | |
| CARROTTask complexity (tau(t))=All2026.02 | 61.3 | 0.28 | |
| FrugalGPTTask complexity (tau(t))=All2026.02 | 61 | 0.51 | |
| SALETask complexity (tau(t))=<= 12.52026.02 | 37.1 | 0.17 | |
| SALE w/o memoryTask complexity (tau(t))=<= 12.52026.02 | 35.4 | 0.19 | |
| WTPTask complexity (tau(t))=<= 12.52026.02 | 34.2 | 0.32 | |
| Best single agentTask complexity (tau(t))=<= 12.52026.02 | 32.9 | 0.36 | |
| TO-RouterTask complexity (tau(t))=<= 12.52026.02 | 32.9 | 0.36 | |
| FrugalGPTTask complexity (tau(t))=<= 12.52026.02 | 30.5 | 0.5 | |
| CARROTTask complexity (tau(t))=<= 12.52026.02 | 29.3 | 0.29 | |
| SALETask complexity (tau(t))=<= 602026.02 | 16.3 | 0.23 | |
| SALE w/o memoryTask complexity (tau(t))=<= 602026.02 | 15.6 | 0.26 | |
| Best single agentTask complexity (tau(t))=<= 602026.02 | 12.5 | 0.36 | |
| TO-RouterTask complexity (tau(t))=<= 602026.02 | 12.5 | 0.36 | |
| FrugalGPTTask complexity (tau(t))=<= 602026.02 | 12.5 | 0.6 | |
| WTPTask complexity (tau(t))=<= 602026.02 | 9.4 | 0.31 | |
| CARROTTask complexity (tau(t))=<= 602026.02 | 9.4 | 0.32 |