Agentic Task Completion on τ2-Bench
84Airline Success RateClaude-Opus-4.5
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Claude-Opus-4.52026.05 | 84 | 79.6 | 70.2 | 92.3 | 24.7 | |
| MAESTRO2026.05 | 84 | 85.1 | 72.9 | 95.6 | 26.9 | |
| GPT-5.22026.05 | 80.5 | 67.5 | 55.5 | 61.6 | 12.4 | |
| Gemini-2.5-Flash2026.05 | 76 | 50.9 | 48.1 | 56.1 | 9.3 | |
| TRACE2026.04 | 44 | 48.2 | 47 | — | — | |
| GEPA2026.04 | 38 | 40.4 | 39.6 | — | — | |
| Single Capability GRPO2026.04 | 34 | 43 | 40.3 | — | — | |
| GRPO on Target2026.04 | 32 | 40.4 | 37.8 | — | — | |
| AWM2026.04 | 32 | 41.2 | 38.4 | — | — | |
| PROVEModel=Qwen3-8B2026.06 | 30 | 34.2 | 29.3 | 23.7 | — | |
| ADP2026.04 | 28 | 34.2 | 32.3 | — | — | |
| PROVEModel=Qwen3-4B2026.06 | 28 | 42.1 | 28.6 | 15.8 | — | |
| BaseModel=Qwen3-4B2026.06 | 26.8 | 35.1 | 25 | 13.2 | — | |
| PROVEModel=Qwen2.5-7B2026.06 | 26 | 16.7 | 21.8 | 22.8 | — | |
| Base Model2026.04 | 24 | 36.8 | 32.9 | — | — | |
| BaseModel=Qwen3-8B2026.06 | 24 | 28.9 | 25.8 | 24.6 | — | |
| PROVEModel=Granite-4.1-8B2026.06 | 22 | 47.4 | 31.3 | 24.6 | — | |
| BaseModel=Qwen2.5-7B2026.06 | 16 | 12.3 | 15 | 16.7 | — | |
| BaseModel=Granite-4.1-8B2026.06 | 8 | 49.1 | 24.9 | 17.5 | — |