τ2-bench
Benchmarks
TaskDatasetSOTA resultTrendResults
τ2-Bench Airline 1.0 (test)
96.4CAP
48
τ2-Bench Retail 1.0 (test)
91.4Completion Accuracy (CAP)
48
τ2-bench
76.5Airline Success Rate
43
τ2-Bench
85.4Accuracy
41
τ2-bench Retail
0.899AUROC
32
τ2-bench Airline
86.5AUROC
32
τ2-BENCH (test)
0.921Average Task Reward
27
τ2-Bench
46.9Telecom Success Rate
23
τ2-bench Airline
97Success Rate
22
τ2-Bench
91.6Score
20
τ2-Bench
84Airline Success Rate
19
τ2-bench
38Airline Score
19
τ2-Bench Telecom
45Avg@2 Score
8
τ2-Bench Airline
60Avg@4
8
τ2-Bench Retail
69.7Avg@4
8
τ2 Bench Retail, Telecom, Airline
48.3Retail Score
5
τ2-bench Retail
100Success Rate
4
τ2-BENCH Telecom (test)
30PASS^4
3
τ2-BENCH Telecom base
20.2PASS^4
3
τ2-BENCH Retail (base)
59.6PASS^4
3
τ2-bench (full)
82.46Retail Success Rate
3
τ2-Bench
90.5Attack Hit Rate (AHR)
3
τ2-bench Telecom
100Success Rate
3
τ2-bench (test)
38.3Success Rate (Airline)
2