Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Failure Prediction research benchmarks · SOTA2 Research

Failure Prediction

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

CIFAR100-LT IF=10 (test)

0.709Acc

28

Feb 26, 2026

CIFAR10-LT IF=10 (test)

93.91Accuracy

28

Feb 26, 2026

Real-robot trajectories 80 rollouts (test)

91.2AUC

9

Apr 21, 2026

ImageNet-LT (test)

67.2AUC (All)

9

Feb 26, 2026

CIFAR-100-LT (test)

77.9AUC (All)

9

Feb 26, 2026

CIFAR-10-LT (test)

83.9AUC (All)

9

Feb 26, 2026

AI4I 2020 (test)

Threshold classifier

94.1Recall (failure)

8

Jun 18, 2026

SWE-agent (Transfer held-out)

0.69ROC-AUC

7

Jun 23, 2026

web (Transfer held-out)

0.649ROC AUC

4

Jun 23, 2026

tau2-bench Transfer held-out

67.4ROC-AUC

4

Jun 23, 2026

OpenHands Transfer (held-out)

69.5ROC-AUC

4

Jun 23, 2026

SWE-Gym (Transfer held-out)

60ROC-AUC

4

Jun 23, 2026

tau-bench Transfer held-out

57.4ROC-AUC

4

Jun 23, 2026

web (Within corpus)

78.5ROC-AUC

4

Jun 23, 2026

tau2-bench Within corpus

0.719ROC AUC

4

Jun 23, 2026

OpenHands (Within corpus)

69.7ROC-AUC

4

Jun 23, 2026

SWE-Gym (Within corpus)

85.3ROC-AUC

4

Jun 23, 2026

SWE-agent (Within corpus)

0.713ROC-AUC

4

Jun 23, 2026

tau-bench Within corpus

63.3ROC-AUC

4

Jun 23, 2026

Popular datasets

CIFAR Tau-bench Web tau2-bench SWE-Gym

Follow for updates

Join SOTA2 Contact