Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Research Idea Evaluation research benchmarks · SOTA2 Research

Research Idea Evaluation

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

ScholarIdeas-AI contribution rubrics

ScholarEval-GPT-4.1

3.45Coverage (mean)

31

Mar 4, 2026

ScholarIdeas (Evaluation)

GPT-5.1 Instant

23.33Reference Inv.

11

Mar 4, 2026

ScholarIdeas Ecology

ScholarEval Claude

2.9Coverage

11

Mar 4, 2026

ScholarIdeas Biochemistry

ScholarEval Claude

2.64Coverage

11

Mar 4, 2026

ScholarIdeas Neuroscience

ScholarEval GPT-4.1

2.74Coverage

11

Mar 4, 2026

D_point (randomly sampled 60 instances)

0.885Rationality Win Rate

5

Feb 26, 2026

5-seed blinded subset (test)

6.24Novelty

4

May 15, 2026

ScholarIdeas-AI

ScholarEval Claude

76.6Evidence

2

Mar 4, 2026

Expert User Study

2.66Citations

2

Mar 4, 2026

Popular datasets

ScholarIdeas ScholarIdeas-AI D_point Expert User Study 5-seed blinded

Follow for updates

Join SOTA2 Contact