Agentic Tool Use on Tau2-Bench (Domain Scores)
90.4Retail ScoreSeed 2.0
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Seed 2.0Model Category=Frontier Proprietary Models2026.04 | 90.4 | 94.2 | 64.4 | 83 | |
| Claude Sonnet-4.5Model Category=Frontier Proprietary Models2026.04 | 86.2 | 98 | 70.1 | 84.7 | |
| Gemini-3 ProModel Category=Frontier Proprietary Models2026.04 | 85.3 | 98 | 72.7 | 85.4 | |
| GPT-5.2 HighModel Category=Frontier Proprietary Models2026.04 | 81.6 | 95.8 | 62.5 | 80.2 | |
| GPT-5Model Identifier=gpt-5-2025-08-072026.05 | 81.1 | 96.7 | 62 | — | |
| MAVENBase Model=GPT-OSS-120b2026.05 | 77.19 | 66.67 | 62 | — | |
| Agent-World-14BModel Category=Open-Source Environment Scaling Methods, Parameter Scale=14B2026.04 | 74.5 | 56.1 | 52 | 65.4 | |
| Kimi-K2Model Identifier=Kimi-K22026.05 | 73.9 | 65.8 | 51.2 | — | |
| Agent-World-8BModel Category=Open-Source Environment Scaling Methods, Parameter Scale=8B2026.04 | 72.8 | 50.9 | 40 | 61.8 | |
| Qwen3-235B-A22BModel Category=Open-Source Foundation Models, Parameter Scale=235B2026.04 | 71.9 | 58 | 45.6 | 58.5 | |
| o4-miniModel Identifier=o4-mini-2025-04-162026.05 | 70.4 | 46.5 | 46 | — | |
| o3Model Identifier=o3-2025-04-162026.05 | 70.4 | 58.2 | 52 | — | |
| Gemini-2.5Model Identifier=gemini-2.5-pro-preview-05-062026.05 | 68.7 | 27.2 | 44 | — | |
| GPT-OSS-120BModel Category=Open-Source Foundation Models, Parameter Scale=120B2026.04 | 67.8 | 49.2 | 48 | 55 | |
| Qwen3-Th-235BModel Identifier=Qwen3-235B2026.05 | 67.8 | 45.6 | 46 | — | |
| DeepSeek-V3.1Model Identifier=DeepSeek-V3.12026.05 | 66.1 | 38.5 | 40 | — | |
| AWM-14BModel Category=Open-Source Environment Scaling Methods, Parameter Scale=14B2026.04 | 63.6 | 17.8 | 31.5 | 39 | |
| Qwen3-32BModel Category=Open-Source Foundation Models, Parameter Scale=32B2026.04 | 59.5 | 27.2 | 48 | 44.9 | |
| Qwen3-14BModel Category=Open-Source Foundation Models, Parameter Scale=14B2026.04 | 55.3 | 14.9 | 27 | 32.4 | |
| ScaleEnv-8BModel Category=Open-Source Environment Scaling Methods, Parameter Scale=8B2026.04 | 50.9 | 27.2 | 37.5 | 38.5 | |
| EnvScaler-8BModel Category=Open-Source Environment Scaling Methods, Parameter Scale=8B2026.04 | 49.6 | 32.7 | 31.5 | 37.9 | |
| GAISBackbone=Qwen3-14B-B., Thinking Mode=false2026.06 | 47.4 | 8.8 | 40 | — | |
| GAISBackbone=Qwen3-8B-Base, Thinking Mode=false2026.06 | 46.5 | 15.8 | 24 | — | |
| GAISBackbone Scale=8B, Fine-tuning Dataset=GAIS2026.06 | 46.5 | 15.8 | 24 | — | |
| GAISBackbone Scale=14B, Fine-tuning Dataset=GAIS2026.06 | 46.5 | 19.3 | 28 | — | |
| Qwen3-8BBackbone=Qwen3-8B-Base, Thinking Mode=true2026.06 | 43.9 | 17.5 | 28 | — | |
| GAISBackbone=Qwen3-14B-B., Thinking Mode=true2026.06 | 43.9 | 17.5 | 40 | — | |
| AWM-8BModel Category=Open-Source Environment Scaling Methods, Parameter Scale=8B2026.04 | 41.2 | 38.5 | 23.5 | 34.4 | |
| Qwen3-14BBackbone=Qwen3-14B-B., Thinking Mode=true2026.06 | 40.4 | 21.1 | 38 | — | |
| NemotronBackbone Scale=14B, Fine-tuning Dataset=Nemotron2026.06 | 39.5 | 14 | 20 | — | |
| Qwen3-8BBackbone=Qwen3-8B-Base, Thinking Mode=false2026.06 | 38.6 | 7 | 16 | — | |
| Qwen3-8BBackbone Scale=8B, Fine-tuning Dataset=Official instruction-tuned2026.06 | 38.6 | 7 | 16 | — | |
| GAISBackbone=Qwen3-4B-B., Thinking Mode=true2026.06 | 37.7 | 7.9 | 22 | — | |
| GAISBackbone=Qwen3-8B-Base, Thinking Mode=true2026.06 | 36.8 | 11.4 | 26 | — | |
| Qwen3-14BBackbone=Qwen3-14B-B., Thinking Mode=false2026.06 | 36.8 | 16.7 | 14 | — | |
| Qwen3-14BBackbone Scale=14B, Fine-tuning Dataset=Official instruction-tuned2026.06 | 36.8 | 16.7 | 14 | — | |
| NemotronBackbone=Qwen3-14B-B., Thinking Mode=true2026.06 | 35.1 | 15.8 | 32 | — | |
| NemotronBackbone=Qwen3-14B-B., Thinking Mode=false2026.06 | 34.2 | 7.9 | 28 | — | |
| Qwen3-8BModel Category=Open-Source Foundation Models, Parameter Scale=8B2026.04 | 34 | 18 | 26.5 | 26.2 | |
| NemotronBackbone=Qwen3-4B-B., Thinking Mode=true2026.06 | 33.3 | 4.4 | 20 | — | |
| NemotronBackbone=Qwen3-8B-Base, Thinking Mode=true2026.06 | 32.5 | 15.8 | 24 | — | |
| GAISBackbone=Qwen3-4B-B., Thinking Mode=false2026.06 | 32.5 | 5.3 | 26 | — | |
| GAISBackbone Scale=4B, Fine-tuning Dataset=GAIS2026.06 | 32.5 | 13.2 | 24 | — | |
| Simulator-8BModel Category=Open-Source Environment Scaling Methods, Parameter Scale=8B2026.04 | 32.2 | 29.2 | 34 | 31.8 | |
| Qwen3-4BBackbone=Qwen3-4B-B., Thinking Mode=true2026.06 | 30.7 | 9.7 | 32 | — | |
| NemotronBackbone=Qwen3-8B-Base, Thinking Mode=false2026.06 | 25.4 | 15.8 | 20 | — | |
| GAISBackbone=Llama, Thinking Mode=false2026.06 | 25.4 | 9.6 | 18 | — | |
| NemotronBackbone Scale=8B, Fine-tuning Dataset=Nemotron2026.06 | 25.4 | 15.8 | 20 | — | |
| Qwen3-4BBackbone=Qwen3-4B-B., Thinking Mode=false2026.06 | 23.7 | 13.2 | 24 | — | |
| Qwen3-4BBackbone Scale=4B, Fine-tuning Dataset=Official instruction-tuned2026.06 | 23.7 | 13.2 | 24 | — | |
| NemotronBackbone Scale=4B, Fine-tuning Dataset=Nemotron2026.06 | 23.7 | 8.8 | 18 | — | |
| TOUCAN-7BModel Category=Open-Source Environment Scaling Methods, Parameter Scale=7B2026.04 | 22.8 | 10.5 | 20 | 17.7 | |
| NemotronBackbone=Qwen3-4B-B., Thinking Mode=false2026.06 | 20.2 | 7.9 | 18 | — | |
| NemotronBackbone=Llama, Thinking Mode=false2026.06 | 12.3 | 5.3 | 14 | — | |
| Llama-3.1-8B-Ins.Backbone=Llama, Thinking Mode=false2026.06 | 7 | 5.3 | 8 | — | |
| ToolLLMBackbone=Qwen3-8B-Base, Thinking Mode=false2026.06 | 0 | 0 | 0 | — | |
| ToolACEBackbone=Qwen3-8B-Base, Thinking Mode=false2026.06 | 0 | 0 | 0 | — | |
| ToolLLMBackbone Scale=8B, Fine-tuning Dataset=ToolLLM2026.06 | 0 | 0 | 0 | — | |
| ToolACEBackbone Scale=8B, Fine-tuning Dataset=ToolACE2026.06 | 0 | 0 | 0 | — | |
| DeepSeek-V3.2-685BModel Category=Open-Source Foundation Models, Parameter Scale=685B2026.04 | — | — | — | 80.3 |