Loading the SOTA2 catalog…
SOTA2 Research · benchmarks
Compare state-of-the-art methods across the tasks and datasets used to measure AI progress.
| Task | Dataset | Domain | Trend | Results | Last Update |
|---|---|---|---|---|---|
| Training Data Reconstruction | SST | 32 | Feb 26, 2026 | ||
| Training Data Reconstruction | CoLA | 32 | Feb 26, 2026 | ||
| Memory Fidelity Evaluation | Memory-based Experiment Seen Features | 32 | Feb 26, 2026 | ||
| Mathematics | GSM8K 0 (test) | 32 | Feb 26, 2026 | ||
| Commonsense Reasoning | CommonsenseQA Non-Math | 32 | Feb 26, 2026 | ||
| Mathematical Reasoning |
| AMC 2023 |
| 32 |
| Feb 26, 2026 |
| Visual Instruction Following | FronTalk Multi-Turn 1.0 (test) | 32 | Feb 26, 2026 |
|---|
| Competitive Programming | Codeforces 2501 - 2507 | 32 | Mar 20, 2026 |
|---|
| Mathematics | Base Aggregate Math (test) | 32 | Feb 26, 2026 |
|---|
| General Question Answering | NQ (Natural Questions) | 32 | Apr 21, 2026 |
|---|
| Long-context Reasoning | ∞ Bench | 32 | Feb 26, 2026 |
|---|
| Mathematical Reasoning | CHAMP | 32 | Feb 26, 2026 |
|---|
| Localization | RefCOCO+ (val) | 32 | Feb 26, 2026 |
|---|
| Safety Classification | ToxicChat | 32 | Jun 26, 2026 |
|---|
| Uncertainty Quantification | MulFactTrap (test) | 32 | Feb 26, 2026 |
|---|
| Uncertainty Quantification | Mixed Dataset (real and fake biographies) | 32 | Feb 26, 2026 |
|---|
| Aspect-conditioned Similarity | ASPECTSIM Sentence Level 1.0 (test) | 32 | Feb 26, 2026 |
|---|
| Open-book generation under knowledge conflict | ConFiQA 1,500 subset | 32 | Feb 26, 2026 |
|---|
| Automatic Speech Recognition | H-O out-of-domain Sudanese dataset (holdout) | 32 | Feb 26, 2026 |
|---|
| Automatic Speech Recognition | OOOK-Eval | 32 | Feb 26, 2026 |
|---|
| Long-form Dialogue | SCM4LLMs | 32 | Feb 26, 2026 |
|---|
| Long-form Dialogue | MT-Bench+ | 32 | Feb 26, 2026 |
|---|
| Long-form Dialogue | LoCoMo | 32 | Feb 26, 2026 |
|---|
| Detoxification | AttaQ benchmark | 32 | Feb 26, 2026 |
|---|
| Detoxification | RealToxicityPrompts challenging | 32 | Feb 26, 2026 |
|---|