Loading the SOTA2 catalog…
SOTA2 Research · benchmarks
Compare state-of-the-art methods across the tasks and datasets used to measure AI progress.
| Task | Dataset | Domain | Trend | Results | Last Update |
|---|---|---|---|---|---|
| Attribute Inference | Real-world Reddit-derived dataset 1.0 (test) | 30 | Feb 26, 2026 | ||
| Language Modeling | WikiText v1 (test) | 30 | Apr 27, 2026 | ||
| Zero-shot Classification | HellaSwag, PIQA, COPA, ARC-C, ARC-E, WinoGrande | 30 | Feb 26, 2026 | ||
| Language Modeling | WikiText2, PTB3, WikiText103 | 30 | Feb 26, 2026 | ||
| Spoken Language Modeling | English Spoken Language Modeling sWUGGY, sBLIMP, tSC (test) |
| 30 |
| Feb 26, 2026 |
| Comprehensive Chinese Evaluation | C-Eval | 30 | Jun 30, 2026 |
|---|
| Linguistic Implicatures Decoding | Open PL LLM Leaderboard Implicatures component base models | 30 | Feb 26, 2026 |
|---|
| Sentiment Transfer | YELP (test) | 30 | Feb 26, 2026 |
|---|
| Long-term Memory Evaluation | LongMemEval S (test) | 30 | May 6, 2026 |
|---|
| Mathematical Reasoning | MATH500 | 30 | Feb 26, 2026 |
|---|
| Fine-grained Hallucination Detection | InFi-Check-FG (test) | 30 | Feb 26, 2026 |
|---|
| Question Answering | ARC | 30 | Feb 26, 2026 |
|---|
| Machine Translation Meta-evaluation | MENT EN-ZH | 30 | Feb 26, 2026 |
|---|
| Machine Translation Meta-evaluation | MENT ZH-EN | 30 | Feb 26, 2026 |
|---|
| Multiple Choice Question Answering | OpenBookQA | 30 | Jun 29, 2026 |
|---|
| Node Classification | Sports | 30 | Feb 26, 2026 |
|---|
| Agentic Tool-use | tau2-bench Airline | 30 | May 25, 2026 |
|---|
| Agentic Tool-use | tau2-bench Retail | 30 | May 25, 2026 |
|---|
| Perturbed Policy Refusal | General Data Protection Regulation (GDPR) | 30 | Feb 26, 2026 |
|---|
| Policy Perturbation Detection | General Data Protection Regulation (GDPR) | 30 | Feb 26, 2026 |
|---|
| Perturbed Policy Refusal | Health Insurance Portability and Accountability Act (HIPAA) | 30 | Feb 26, 2026 |
|---|
| Policy Perturbation Detection | Health Insurance Portability and Accountability Act (HIPAA) | 30 | Feb 26, 2026 |
|---|
| Question Answering | All QA Datasets Average | 30 | Apr 28, 2026 |
|---|
| Machine Translation (xx -> zh) | FLORES+ latest (test) | 30 | Feb 26, 2026 |
|---|
| Generative Performance | HH Golden 42.5k (test) | 30 | Feb 26, 2026 |
|---|