Loading the SOTA2 catalog…
SOTA2 Research · benchmarks
Compare state-of-the-art methods across the tasks and datasets used to measure AI progress.
| Task | Dataset | Domain | Trend | Results | Last Update |
|---|---|---|---|---|---|
| Structured Distribution Sampling | OOD distribution families Bernoulli, Poisson, Maxwell, TruncNorm, Chi, and Weibull (Held-out) | 36 | May 13, 2026 | ||
| Patience-discounted reward evaluation | NoveltyBench | 36 | May 13, 2026 | ||
| Random-generation diversity evaluation | Open-ended random generation | 36 | May 13, 2026 | ||
| Question Answering | Overall (NQ, PopQA, 2WikiQA, Musique, LOCOMO) (test) | 36 | May 12, 2026 | ||
| Conversation Question Answering | LOCOMO (test) |
| 36 |
| May 12, 2026 |
| Simple Question Answering | PopQA (test) | 36 | May 12, 2026 |
|---|
| Simple Question Answering | NQ* (test) | 36 | May 12, 2026 |
|---|
| Bandit Optimization | Setting 2 | 36 | May 12, 2026 |
|---|
| Bandit Optimization | Setting 1 | 36 | May 12, 2026 |
|---|
| Traitor win probability estimation | The Traitors Game Small configurations Migdał value based (n <= 11) | 36 | May 12, 2026 |
|---|
| Image Classification | CIFAR-10 8000 images (test) | 36 | May 12, 2026 |
|---|
| Sequential Model Editing | ZsRE T = 300 | 36 | May 12, 2026 |
|---|
| Sequential Model Editing | CounterFact T = 300 | 36 | May 12, 2026 |
|---|
| Language Modeling | Llama decoder 10M | 36 | May 12, 2026 |
|---|
| Multimodal time-series forecasting | Agriculture textual-numerical time-series dataset | 36 | May 12, 2026 |
|---|
| Classification | Devnagari dev (test) | 36 | May 12, 2026 |
|---|
| Math Reasoning | Olympiad Bench | 36 | May 12, 2026 |
|---|
| Macro-average Exact Match | ShellOps-Pro | 36 | May 11, 2026 |
|---|
| Robotic Manipulation | RoboTwin (random-scene) | 36 | May 11, 2026 |
|---|
| Relocalization | Grass-Slope Scene | 36 | May 11, 2026 |
|---|
| Modular Addition | Modular Addition Token Embedding | 36 | May 11, 2026 |
|---|
| Modular Addition | Modular Addition Angular Embedding (test) | 36 | May 11, 2026 |
|---|
| Causal Discovery | Lorenz-96 | 36 | May 11, 2026 |
|---|
| ECG Classification | CODE-15% (test) | 36 | May 11, 2026 |
|---|
| ECG Classification | MIMIC-IV-ECG (test) | 36 | May 11, 2026 |
|---|