List your company

Task failure prediction and selective task completion on tau2-bench Telecom 1.0

0.809AUROCTRACER
0.407560.511780.6160.72022Feb 11, 2026

Evaluation results

MethodLinks
TRACERModel=gemini-2.5-flash2026.020.8090.52
TRACERModel=gpt-4.1-mini2026.020.7650.613
TRACERModel=gemini-2.5-pro2026.020.6910.517
Semantic EntropyModel=gpt-4.1-mini2026.020.6860.394
Self-Assessed ConfidenceModel=gpt-4.1-mini2026.020.6830.389
SAUPModel=gemini-2.5-flash2026.020.6730.446
Semantic EntropyModel=gemini-2.5-pro2026.020.6510.395
Normalized EntropyModel=gemini-2.5-pro2026.020.6490.392
SAUPModel=gemini-2.5-pro2026.020.6390.379
Semantic EntropyModel=gemini-2.5-flash2026.020.6210.392
Normalized EntropyModel=gemini-2.5-flash2026.020.5590.358
Normalized EntropyModel=gpt-4.1-mini2026.020.5480.27
SAUPModel=gpt-4.1-mini2026.020.5340.347
Self-Assessed ConfidenceModel=gemini-2.5-pro2026.020.5070.283
Self-Assessed ConfidenceModel=gemini-2.5-flash2026.020.4230.257