Counterfactual reasoning
Benchmarks
DatasetSOTA methodMetricTrendResultsLast updated
63.4Mean Score
52
Feb 26, 2026
71.41Accuracy
40
Feb 27, 2026
71.6EM
30
Feb 26, 2026
80.8Basic Score
20
Apr 14, 2026
100Accuracy
16
Feb 26, 2026
99.7Accuracy
16
Feb 26, 2026
94.53Exact Match Performance
11
Feb 26, 2026
83.64Accuracy
8
Feb 26, 2026
80.05Accuracy
8
Feb 26, 2026
90.75AP (Exact Counterfactual Accuracy)
6
May 11, 2026
16,515MSE
5
Mar 5, 2026
599MSE
5
Mar 5, 2026
150MSE
5
Mar 5, 2026
48.8MSE
5
Mar 5, 2026
72.1Safe Precision
4
Feb 26, 2026
80.6Numerical Direct Accuracy
4
Feb 26, 2026
0.18Mean Error (PN)
3
Jun 11, 2026
100Joint Accuracy (A ∧ B)
3
May 18, 2026
28Infeasible Rate
1
Jun 11, 2026
—
—Primary metric
0
Feb 26, 2026
—
—Primary metric
0
Feb 26, 2026