Loading the SOTA2 catalog…
SOTA2 Research · benchmarks
Compare state-of-the-art methods across the tasks and datasets used to measure AI progress.
| Task | Dataset | Domain | Trend | Results | Last Update |
|---|---|---|---|---|---|
| Image Classification | CIFAR-100 60% symmetric label noise | 36 | Apr 23, 2026 | ||
| Univariate Anomaly Detection | NAB | 36 | Apr 23, 2026 | ||
| Reasoning | GPQA Diamond | 36 | Jun 2, 2026 | ||
| Synthetic Image Detection | Chameleon | 36 | Jun 8, 2026 | ||
| Image Classification | ImageNet-1K (val) | 36 | Apr 23, 2026 |
| Reasoning and Code Generation | Reasoning Evaluation Suite (GSM8K, MATH500, AIME24, HumanEval) (test) | 36 | Apr 23, 2026 |
|---|
| Over-refusal Evaluation | Over-refusal (test) | 36 | Apr 23, 2026 |
|---|
| Harmfulness Evaluation | Harmfulness Evaluation Suite JBB, SR, WJ, GCG, JBC, PAIR (test) | 36 | Apr 23, 2026 |
|---|
| Math and Science Reasoning | Average | 36 | Apr 21, 2026 |
|---|
| Math Reasoning | AIME 2025 | 36 | Apr 21, 2026 |
|---|
| Math Reasoning | AIME 2024 | 36 | Apr 21, 2026 |
|---|
| Math Reasoning | MATH 500 | 36 | Apr 21, 2026 |
|---|
| Adversarial Attack | Medical Imaging Dataset 1,000 images 1.0 (test) | 36 | Apr 21, 2026 |
|---|
| Jailbreak Attack | AdvBench and VulMine 520 harmful behaviors and curated prompts | 36 | Apr 21, 2026 |
|---|
| Message filtering | Blood request dataset | 36 | Apr 21, 2026 |
|---|
| Multi-hop Question Answering | WebQSP | 36 | Apr 21, 2026 |
|---|
| Multi-hop Question Answering | Mintaka | 36 | Apr 21, 2026 |
|---|
| Multi-hop Question Answering | SimpleQA | 36 | Apr 21, 2026 |
|---|
| Multi-hop Question Answering | ExaQT | 36 | Apr 21, 2026 |
|---|
| Multi-hop Question Answering | CWQ | 36 | Apr 21, 2026 |
|---|
| Reasoning | MMLU PRO | 36 | Jun 2, 2026 |
|---|
| Reasoning | GPQA Diamond | 36 | May 19, 2026 |
|---|
| Novel Emerging Segmentation | NEST | 36 | Apr 16, 2026 |
|---|
| Open-world Knowledge Graph Question Answering | GLOW-Bench 1-Hop | 36 | Apr 16, 2026 |
|---|
| Software Engineering | SWE-bench Lite | 36 | May 29, 2026 |
|---|