List your company

Task Failure Prediction and Selective Completion on tau2-bench Airline 1.0

74.2AUROCTRACER
27.19239.39651.663.804Feb 11, 2026

Evaluation results

MethodLinks
TRACERModel=gpt-4.1-mini2026.0274.261.5
TRACERModel=gemini-2.5-pro2026.0273.562.9
TRACERModel=gemini-2.5-flash2026.0272.569.7
Semantic EntropyModel=gemini-2.5-flash2026.0266.664.8
Self-Assessed ConfidenceModel=gemini-2.5-flash2026.0266.364.5
Semantic EntropyModel=gemini-2.5-pro2026.0260.351.6
Normalized EntropyModel=gemini-2.5-pro2026.026051.4
SAUPModel=gemini-2.5-pro2026.0259.551.7
Normalized EntropyModel=gemini-2.5-flash2026.0256.861.2
Semantic EntropyModel=gpt-4.1-mini2026.0254.142.7
SAUPModel=gemini-2.5-flash2026.025459.9
Normalized EntropyModel=gpt-4.1-mini2026.0253.840.2
SAUPModel=gpt-4.1-mini2026.0253.140.3
Self-Assessed ConfidenceModel=gemini-2.5-pro2026.0246.245.1
Self-Assessed ConfidenceModel=gpt-4.1-mini2026.022923.6