Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Symbolic Reasoning research benchmarks – Page 2 · SOTA2 Research

Symbolic Reasoning

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

93.3Solve Rate

4

Feb 26, 2026

Maze Hard (test)

87.7Accuracy

3

Mar 6, 2026

Sudoku-Extreme (test)

89.34Accuracy

3

Mar 6, 2026

ARC-AGI 2 (test)

9.9Pass@1

3

Mar 6, 2026

84Accuracy (BBH Symbolic Reasoning)

3

Mar 5, 2026

Coinflip OOD: 4

90.2Accuracy

3

Feb 27, 2026

Coinflip OOD: 3

98.6Accuracy

3

Feb 27, 2026

Last Letter 12 words

100Accuracy

3

Feb 27, 2026

Last Letter Concatenation OOD: 4 (test)

63Accuracy

3

Feb 27, 2026

Last Letter Concatenation OOD: 3 (test)

94.8Accuracy

3

Feb 27, 2026

95.1Solve Rate

3

Feb 26, 2026

Coin Flip (OOD)

RULEREASONER-8B + DADS

95.1pass@1

2

Feb 26, 2026

Coin Flip 4 times

91.4Accuracy (format-specific)

2

Feb 27, 2026

Last Letter 4 words

57.6Accuracy (format-specific)

2

Feb 27, 2026

MMLU-Hard abstract_algebra 100 items

Uncoded single call

41.7Baseline Accuracy

1

May 12, 2026

Coinflip OOD: 4 (test)

—

—Accuracy

0

Feb 26, 2026

Coinflip OOD: 3 (test)

—

—Accuracy

0

Feb 26, 2026

—

—Accuracy

0

Feb 26, 2026

Page 2 of 2

Popular datasets

Coin Flip Coinflip Last Letter Concatenation Last Letter MMLU

Follow for updates

Join SOTA2 Contact