tau2-bench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
tau2-Bench
81.1Retail Performance Score
102
tau2-bench Airline
63.5Pass@1
30
tau2-bench Retail
82Pass@1
30
tau2-Bench held-out (test)
41.9Average Score
16
tau2-bench Telecom 1.0
0.809AUROC
15
tau2-bench Retail 1.0
0.707AUROC
15
tau2-bench Airline 1.0
74.2AUROC
15
tau2-bench
86.69Score
12
tau2-Bench
75.2Retail Success Rate
12
tau2-bench
72Airline Success Rate
9
tau2-bench telecom
67Pass Rate
9
tau2-bench airline
60Pass Rate
9
tau2-bench Retail Domain
61.4Total Pass Rate
9
tau2-bench
81.2Pass@1
9
Tau2-Bench Telecom subset Solo mode
43.3Success Rate
7
tau2-Bench (test)
22.3Success Rate
7
tau2-bench v1 v2 (test)
89Recall
6
tau2-Bench Telecom
98.2Score
6
Tau2-Bench
57.4Success Rate
6
tau2-bench Retail + Airline (test)
95.8HL Score
5
tau2-bench Airline (test)
90.3HL
5
tau2-bench Transfer held-out
67.4ROC-AUC
4
tau2-bench Within corpus
0.719ROC AUC
4
tau2-bench
43Productivity
4
tau2-bench Telecom
85.1Task Completion Score (TCS)
4