Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

PaperBench dataset and benchmarks · SOTA2 Research

Research Datasets

PaperBench

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Software Engineering

PaperBench Code (dev)

78.2Score

15

Repository-level paper reproduction

PaperBench Code lite (dev)

49.26MU-DPO

12

Experiment Reproduction

PaperBench Code (dev)

64.1Score

9

Software Engineering

PaperBench

66.8Score

9

Paper-to-Code Reproduction

PaperBench Code (dev)

78.6Final Score

9

Long-horizon Research Task Reproduction

PaperBench Code (dev)

72.22FRE Score

7

ML research engineering

PaperBench

33.26Adaptive Pruning Score

6

Paper-to-code reproduction

PaperBench Code ICML 2024 (dev)

0.786Average Score

6

Detail Knowledge Extraction

PaperBench Category B: Detail

92.6Accuracy

2

Fidelity Knowledge Extraction

PaperBench Category A: Fidelity

96.7Accuracy

2

Popular tasks

Software Engineering Paper-to-Code Reproduction Long-horizon Research Task Reproduction ML research engineering Repository-level paper reproduction

Follow for updates

Join SOTA2 Contact