Agent Execution on tau-Bench (test)
59Execution AccuracyGemini-2.5 Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-2.5 ProPrompting=2-shot2026.03 | 59 | |
| Claude-3.5-SonnetPrompting=2-shot2026.03 | 56 | |
| GPT-4oPrompting=2-shot2026.03 | 54 | |
| Qwen3-8B Agentic GRPOTraining Strategy=Agentic GRPO, Training Data=<1K2026.03 | 42 | |
| Qwen3-8B SFTTraining Strategy=SFT, Training Data=<1K2026.03 | 36 | |
| Qwen3-8B BasePrompting=2-shot2026.03 | 33 | |
| xLAM-2-70BTraining Data=60K-trained2026.03 | 17 | |
| ToolAceTraining Data=26K-trained2026.03 | 15 |