Commonsense and Reading Comprehension Suite

Benchmarks

TaskDatasetSOTA resultTrendResults

7-benchmark commonsense and reading-comprehension suite (ARC-Easy, ARC-Challenge, HellaSwag, WinoGrande, PIQA, BoolQ, and OpenBookQA) LM Evaluation Harness default (test)

68.77Accuracy

108

Zero-shot Question Answering and Language Modeling

Commonsense and Reading Comprehension Suite (ARC-c, ARC-e, BoolQ, HellaSwag, LAMBADA, PIQA, RACE) zero-shot

26.37ARC-c Accuracy