User Simulation Behavioral Alignment on tau2-bench Retail + Airline (test)
95.8HL ScoreHumans
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| HumansUser Simulator=DeepSeek-V3.12026.05 | 95.8 | 62.3 | 79 | 94.9 | 97.8 | 88.6 | 92.2 | 93.4 | |
| PPol: EvolvedUser Simulator=DeepSeek-V3.12026.05 | 57 | 65.7 | 61.4 | 65.4 | 93.7 | 84 | 54.8 | 74.5 | |
| PPol: InitialUser Simulator=DeepSeek-V3.12026.05 | 41 | 32.3 | 36.7 | 42.7 | 85.3 | 43.8 | 33.2 | 51.2 | |
| DP PersonasUser Simulator=DeepSeek-V3.12026.05 | 29.2 | 34 | 31.6 | 37.1 | 75.9 | 49.8 | 32.9 | 48.9 | |
| Base-simulatorUser Simulator=DeepSeek-V3.12026.05 | 12.3 | 14.6 | 13.5 | 32 | 88.7 | 53.4 | 54.4 | 57.1 |