Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Zero-shot Language Understanding research benchmarks · SOTA2 Research

Zero-shot Language Understanding

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

ARC-c, ARC-e, PIQA, Winogrande, Hellaswag

6,257Mean Accuracy

121

Jun 12, 2026

ARC-Easy, ARC-Challenge, HellaSwag, LAMBADA, PIQA lm-eval 0.4.11 (test)

81.5Average Accuracy

42

Mar 31, 2026

Evaluation Suite Zero-shot (LMB, HellA, PIQA, ARC-E, ARC-C, WINO, Open, MMLU)

BLT Distill (Teacher)

83.4ARC-E Accuracy

25

Feb 26, 2026

Reasoning Suite Zero-shot (BoolQ, WinoG., PIQA, OBQA, HellaS., ARC-e, ARC-c)

82.63BoolQ Accuracy

24

Mar 9, 2026

Zero-shot Benchmarks

73.07Average Zero-shot Accuracy

21

Jun 1, 2026

LM Evaluation Harness Downstream Suite (HellaSwag, PIQA, WinoGrande, OpenBookQA, SIQA, BoolQ, TriviaQA, MMLU, ARC-Challenge, ARC-Easy, MathQA, SciQ)

Llama-3.1-8B-Instruct

72.52HellaSwag Accuracy

18

Jun 2, 2026

Qwen3-0.6B Zero-shot Evaluation Suite (AE, AC, SciQ, MM, MM-P, HS, OBQA, PIQA, RACE, WG, CSQA, AGI) (test)

68.64Accuracy (AE)

15

May 19, 2026

LLM Evaluation Suite MMLU, GSM8k, HellaSwag, WinoGrande

72.8MMLU

12

Mar 4, 2026

BoolQ, ARC-e, ARC-c, WinoGrande, HellaSwag

83.08ARC-e Accuracy

8

Apr 14, 2026

Commonsense Reasoning and Language Modeling Suite (PIQA, SIQA, HellaSwag, ARC-E, ARC-C, WinoGrande, LAMBADA) zero-shot

70.46PIQA Accuracy

6

May 12, 2026

Combined Zero-shot

63.71Average Accuracy

4

Apr 23, 2026

LLM Evaluation Suite (AE, AC, SciQ, MM, MM-P, HS, OBQA, PIQA, RACE, WG, CSQA, AGI) zero-shot

67.97AE Score

2

May 19, 2026

Popular datasets

LLM Evaluation Suite Evaluation Suite LM Evaluation Harness Reasoning Suite Combined

Follow for updates

Join SOTA2 Contact