Sister Tasks on extended tau-Bench Airline domain
74Pass@1 RatePolicyBank
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PolicyBankModel=Gemini-3-Pro2026.04 | 74 | 60 | 52 | 48 | |
| PolicyBankModel=Claude-4.5-Opus2026.04 | 72 | 55 | 53 | 40 | |
| ReasoningBankModel=Claude-4.5-Opus2026.04 | 45 | 32 | 28 | 16 | |
| SynapseModel=Claude-4.5-Opus2026.04 | 33 | 15 | 14 | 10 | |
| AWMModel=Claude-4.5-Opus2026.04 | 31 | 17 | 11 | 10 | |
| No memoryModel=Claude-4.5-Opus2026.04 | 30 | 18 | 13 | 9 | |
| ReasoningBankModel=Gemini-3-Pro2026.04 | 23 | 17 | 10 | 10 | |
| SynapseModel=Gemini-3-Pro2026.04 | 2 | 1 | 0 | 0 | |
| No memoryModel=Gemini-3-Pro2026.04 | 1 | 0 | 0 | 0 | |
| AWMModel=Gemini-3-Pro2026.04 | 1 | 0 | 0 | 0 |