GPT-3 Evaluation Suite

Benchmarks

TaskDatasetSOTA resultTrendResults

GPT-3 Evaluation Suite (11 tasks: HellaSwag, LAMBADA, TriviaQA, WebQs, Winogrande, PIQA, ARC Challenge, ARC Easy, ANLI R1, ANLI R2, ANLI R3) zero-shot

33.6Average Accuracy

Few-shot Language Understanding

GPT-3 Evaluation Suite Few-shot

48.1Accuracy

Zero-shot Evaluation

GPT-3 Evaluation Suite (LAMBADA, TriviaQA, WebQs, PIQA, RACE-h, BoolQ) 1.3B various (test val)

44.4Overall Accuracy