Agentic Tool-use on tau2-bench Retail
82Pass@1CODEDELEGATOR
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CODEDELEGATORModel=DeepseekV3.22026.01 | 82 | 71.2 | 63.4 | 57 | — | — | — | |
| +PACESLM=Qwen3.5-9B2026.05 | 81.7 | 74.5 | 69.8 | 66.1 | — | — | — | |
| GPT-5Model Category=Proprietary Models2026.03 | 81.6 | 71.8 | 64.7 | 58.8 | — | — | — | |
| CODEDELEGATORModel=GPT5.02026.01 | 80.9 | 70.3 | 64.5 | 60.5 | — | — | — | |
| ReActModel=DeepseekV3.22026.01 | 79.6 | 69.9 | 63.4 | 58.8 | — | — | — | |
| VanillaSLM=Qwen3.5-9B2026.05 | 79.1 | 70.6 | 65.9 | 62.6 | — | — | — | |
| ReActModel=GPT5.02026.01 | 78.7 | 68.9 | 62.9 | 58.8 | — | — | — | |
| +PESLM=Qwen3.5-9B2026.05 | 78.5 | 70.4 | 65.2 | 61.7 | — | — | — | |
| CodeActModel=GPT5.02026.01 | 75 | 65.3 | 59.3 | 57.1 | — | — | — | |
| CodeActModel=DeepseekV3.22026.01 | 70.2 | 59 | 50 | 47 | — | — | — | |
| Sonnet 3.52026.05 | 69.2 | 57.6 | 50.9 | 46.2 | — | — | — | |
| GPT-4oModel Category=Proprietary Models2026.03 | 64 | 50.8 | 43 | 37.7 | — | — | — | |
| Qwen3-235B-A22B-Inst.Model Category=Open-Source Models, Parameter Scale=≥70B2026.03 | 63.6 | 51.2 | 44.5 | 39.5 | — | — | — | |
| Simia-Tau-32BModel Category=Open-Source Models, Parameter Scale=~30B2026.03 | 62.5 | 50.1 | 41.9 | 36 | — | — | — | |
| GPT-4o2026.05 | 60.4 | 49.1 | 43 | 38.3 | — | — | — | |
| xLAM-2-70b-fc-rModel Category=Open-Source Models, Parameter Scale=≥70B2026.03 | 59.7 | 44.1 | 35.8 | 30.3 | — | — | — | |
| CoVe-4BModel Category=Open-Source Models, Parameter Scale=≤8B2026.03 | 59.4 | 50 | 45 | 41.2 | — | — | — | |
| GEM-32BModel Category=Open-Source Models, Parameter Scale=~30B2026.03 | 55.5 | — | — | — | — | — | — | |
| Qwen3-30B-A3B-Inst.Model Category=Open-Source Models, Parameter Scale=~30B2026.03 | 53.5 | 41.2 | 34.4 | 29.8 | — | — | — | |
| Simia-Tau-8BModel Category=Open-Source Models, Parameter Scale=≤8B2026.03 | 52.4 | 39.6 | 32.7 | 28.1 | — | — | — | |
| Simia-Tau-RL-8BModel Category=Open-Source Models, Parameter Scale=≤8B2026.03 | 52.4 | 41.1 | 34.9 | 30.7 | — | — | — | |
| xLAM-2-32b-fc-rModel Category=Open-Source Models, Parameter Scale=~30B2026.03 | 52.1 | 37.1 | 29.2 | 24.6 | — | — | — | |
| +PACESLM=Qwen3-4B2026.05 | 51.3 | 37.8 | 30.7 | 27 | — | — | — | |
| xLAM-2-8b-fc-rModel Category=Open-Source Models, Parameter Scale=≤8B2026.03 | 48.9 | 34.5 | 27.5 | 22.5 | — | — | — | |
| +PESLM=Qwen3-4B2026.05 | 47.4 | 34.1 | 27.8 | 22.6 | — | — | — | |
| VanillaSLM=Qwen3-4B2026.05 | 46.5 | 33.2 | 26.1 | 21.7 | — | — | — | |
| GEM-8BModel Category=Open-Source Models, Parameter Scale=≤8B2026.03 | 44.5 | — | — | — | — | — | — | |
| Qwen3-4B-Inst.Model Category=Open-Source Models, Parameter Scale=≤8B2026.03 | 40.1 | 27.3 | 21.7 | 18.4 | — | — | — | |
| Simia-Tau-3BModel Category=Open-Source Models, Parameter Scale=≤8B2026.03 | 32 | — | — | — | — | — | — | |
| xLAM-2-3b-fc-rModel Category=Open-Source Models, Parameter Scale=≤8B2026.03 | 15.1 | 8.8 | 6.4 | 5.3 | — | — | — | |
| Agent-CBackbone=Qwen3-8B2026.03 | — | — | — | — | 42.2 | 0 | 234.7 | |
| DeepSeek-V3.2Mode=Non-Reasoning, Architecture=MoE, Total Params=671B, Activated Params=37B2026.01 | — | — | — | — | 80.9 | — | — | |
| EXAONE 4.0Mode=Non-Reasoning, Architecture=Dense, Total Params=32B, Activated Params=32B2026.01 | — | — | — | — | 61.4 | — | — | |
| K-EXAONEMode=Non-Reasoning, Architecture=MoE, Total Params=236B, Activated Params=23B2026.01 | — | — | — | — | 73.2 | — | — | |
| LLMBackbone=Qwen3-8B2026.03 | — | — | — | — | 11.3 | 76.3 | 146.6 | |
| Qwen3-235B-A22B Instruct-2507Architecture=MoE, Total Params=235B, Activated Params=22B2026.01 | — | — | — | — | 74.6 | — | — | |
| SEVerAConstraints=without, Backbone=Qwen3-8B2026.03 | — | — | — | — | 49.4 | 10.3 | 238.3 | |
| SEVerABackbone=Qwen3-8B2026.03 | — | — | — | — | 53.6 | 0 | 212.4 |