Original Tasks on tau-Bench Airline domain extended
70Pass Rate @ Threshold 1ReasoningBank
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ReasoningBankModel=Gemini-3-Pro2026.04 | 70 | 63 | 60 | 60 | |
| PolicyBankModel=Gemini-3-Pro2026.04 | 70 | 64 | 62 | 59 | |
| ReasoningBankModel=Claude-4.5-Opus2026.04 | 69 | 61 | 53 | 48 | |
| PolicyBankModel=Claude-4.5-Opus2026.04 | 69 | 59 | 51 | 49 | |
| SynapseModel=Gemini-3-Pro2026.04 | 67 | 60 | 55 | 47 | |
| AWMModel=Gemini-3-Pro2026.04 | 67 | 63 | 59 | 55 | |
| AWMModel=Claude-4.5-Opus2026.04 | 67 | 54 | 51 | 48 | |
| No memoryModel=Gemini-3-Pro2026.04 | 66 | 62 | 59 | 58 | |
| No memoryModel=Claude-4.5-Opus2026.04 | 62 | 52 | 47 | 44 | |
| SynapseModel=Claude-4.5-Opus2026.04 | 62 | 54 | 50 | 47 |