User Simulation Behavioral Alignment on tau2-Bench Airline (test)
90.3HLHumans
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| HumansUser Simulator=GPT-5.4-Mini2026.05 | 90.3 | 58.4 | 74.4 | 91.3 | 98.1 | 84.8 | 77.8 | 88 | |
| PPol: EvolvedUser Simulator=GPT-5.4-Mini2026.05 | 60.4 | 54.5 | 57.4 | 68.7 | 87.7 | 66.9 | 66.7 | 72.5 | |
| PPol: InitialUser Simulator=GPT-5.4-Mini2026.05 | 45.7 | 16.4 | 31 | 59.3 | 65.6 | 57.6 | 54.4 | 59.2 | |
| DP PersonasUser Simulator=GPT-5.4-Mini2026.05 | 35.8 | 19.6 | 27.7 | 44.5 | 59.4 | 62.6 | 42.9 | 52.4 | |
| Base-simulatorUser Simulator=GPT-5.4-Mini2026.05 | 32.2 | 16.3 | 24.3 | 47.2 | 62.3 | 61.1 | 46.8 | 54.3 |