Customer Support Interaction on τ-Bench Airline Verified (test)
82Pass@1Gemini-3-flash
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Gemini-3-flashUser Simulator=GPT-5.22026.05 | 82 | 68 | |
| Gemini-3-flashUser Simulator=Gemini-3-flash2026.05 | 72 | 56 | |
| claude-sonnet-4.6User Simulator=Gemini-3-flash2026.05 | 72 | — | |
| Gemini-2.5-flashUser Simulator=GPT-5.22026.05 | 66 | 40 | |
| Gemini-2.5-flashUser Simulator=Gemini-3-flash2026.05 | 58 | 40 | |
| GPT-5.2User Simulator=GPT-5.22026.05 | 58 | 36 | |
| GPT-5.2User Simulator=Gemini-3-flash2026.05 | 57 | 34 | |
| Qwen-32BUser Simulator=Gemini-3-flash2026.05 | 50 | 26 | |
| deepseek-3.1User Simulator=Gemini-3-flash2026.05 | 49 | 30 | |
| deepseek-3.1User Simulator=GPT-5.22026.05 | 48 | 24 | |
| Qwen-32BUser Simulator=GPT-5.22026.05 | 36 | 22 |