Loading the SOTA2 catalog…
SOTA2 Research · benchmarks
Compare state-of-the-art methods across the tasks and datasets used to measure AI progress.
| Task | Dataset | Domain | Trend | Results | Last Update |
|---|---|---|---|---|---|
| Multiple Choice Question Answering | MMLU English (test) | 32 | Mar 30, 2026 | ||
| Visual Question Answering | GQA | 32 | Jun 16, 2026 | ||
| Video Question Answering | ActivityNet | 32 | Jun 24, 2026 | ||
| Science Question Answering | ARC-C | 32 | Mar 10, 2026 | ||
| OpenBook Question Answering | OBQA | 32 | May 14, 2026 | ||
| MRQA 2019 (dev) |
| 32 |
| Mar 12, 2026 |
| Question Answering | GPQA Diamond v1 (test) | 32 | Feb 26, 2026 |
|---|
| Medical Question Answering | PubMedQA | 32 | May 28, 2026 |
|---|
| Conversational Question Answering | COQA zero-shot (test) | 32 | Feb 26, 2026 |
|---|
| Mathematics Question Answering | GSM8K zero-shot (test) | 32 | Feb 26, 2026 |
|---|
| Zero-shot Question Answering and Reasoning | LM-Eval-Harness Suite (PIQA, HellaSwag, LAMBADA, ARC-e, ARC-c, SciQ, Race, MMLU) zero-shot | 32 | Feb 26, 2026 |
|---|
| Visual Question Answering | AnimalKB 1.0 (test) | 32 | Feb 26, 2026 |
|---|
| Visual Question Answering | CLEAR 1.0 (Retain) | 32 | Feb 26, 2026 |
|---|
| Embodied Question Answering | OpenEQA v1.0 (test) | 32 | Feb 26, 2026 |
|---|
| Visual Question Answering | UCIT | 32 | Jun 16, 2026 |
|---|
| Visual Question Answering | MLLM-DCL | 32 | Jun 16, 2026 |
|---|
| Video Question Answering | Video-MME | 32 | Jul 3, 2026 |
|---|
| General Visual Question Answering | RealWorldQA | 32 | Jul 7, 2026 |
|---|
| General Question Answering | NQ (Natural Questions) | 32 | Apr 21, 2026 |
|---|
| Question Answering | ARC (25-shot), MMLU (5-shot), HellaSwag (10-shot), TruthfulQA (0-shot), and WinoGrande (0-shot) (test) | 32 | Feb 26, 2026 |
|---|
| Multi-hop Question Answering | HotpotQA | 32 | Feb 26, 2026 |
|---|
| Question Answering | Overall NQ, TriviaQA, BioASQ, PopQA | 32 | Feb 26, 2026 |
|---|
| Visual Question Answering | MMHalSnowball HalluConv 1.0 | 32 | Feb 26, 2026 |
|---|
| Visual Question Answering | MMHalSnowball CleanConv 1.0 | 32 | Feb 26, 2026 |
|---|
| Visual Question Answering | GQA balanced (test-dev) | 32 | Feb 26, 2026 |
|---|