Commonsense Reasoning Suite
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Commonsense Reasoning Suite BoolQ, PIQA, HellaS, WinoG, ARC-e, ARC-c, OBQA
72.8Average Accuracy
98
Commonsense Reasoning Suite (test)
0.959HellaSwag Accuracy
87
Commonsense Reasoning Suite
35OpenBookQA Accuracy
48
Commonsense Reasoning Suite Boolq, PIQA, SIQA, Win, OBQA, HellaSwag, ARC-E, ARC-C
77.5BoolQ Accuracy
44
Commonsense Reasoning Suite BoolQ, PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c
87.49BoolQ Accuracy
43
Commonsense Reasoning Suite (PIQA, ARC-e, ARC-c, HellaSwag, WinoGrande)
74.1PIQA Accuracy
35
Commonsense Reasoning Suite (ARC-c, BoolQ, PIQA, HellaSwag, WinoGrande) zero-shot
55.03ARC-c Accuracy
35
Commonsense Reasoning Suite
73.18BoolQ Accuracy
32
Commonsense Reasoning Suite
75.24BoolQ Accuracy
26
Commonsense Reasoning Suite Zero-shot
82.81PIQA Accuracy
25
Commonsense Reasoning Suite (ARC-e, OBQA, SIQA, ARC-c, WinoG., PIQA)
88ARC-e Accuracy
24
Commonsense Reasoning Suite (OBQA, PIQA, BoolQ, SIQA, Hellaswag, ARC-E, ARC-C, Winogrande) (test)
44.8OBQA Accuracy
21
Commonsense Reasoning Suite LM Eval Harness
48.89LM Eval Score
20
Commonsense Reasoning Suite (PIQA, SIQA, HellaSwag, ARC-E, ARC-C, WinoGrande, LAMBADA) zero-shot
66.92PIQA Accuracy
18
Commonsense Reasoning Suite (PIQA, HellaSwag, WinoGrande, ARC-Easy, ARC-Challenge) zero-shot LLaMA-2-7B
78PIQA Accuracy
17
Commonsense Reasoning Suite (BoolQ, PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c, OBQ) (test)
63.27BoolQ Accuracy
16
Commonsense Reasoning Suite (PiQA, Arc-C, WinoGrande, HellaSwag, SciQ, OBQA, BoolQ, Arc-E) (test)
80.79PiQA Accuracy
15
Commonsense Reasoning Suite (PiQA, Arc-C, WinoGrande, HellaSwag, SciQ, OBQA, BoolQ, Arc-E)
82.21PiQA Accuracy
15
Commonsense Reasoning Suite (ARC-e, ARC-c, BoolQ, OBQA, PIQA) (test)
77.7ARC-e
8
Commonsense Reasoning Suite HellaSwag, PIQA, OBQA, COPA, WinoGrande
27.3HellaSwag Accuracy
7
Commonsense Reasoning Suite (HellaSwag, PIQA, OBQA, COPA, WinoGrande)
32.8HellaSwag Accuracy
7
Commonsense Reasoning Suite (PIQA, WinoGrande, HellaSwag, ARC) Zero-shot Llama-2-70B
82.7PIQA Accuracy (Zero-shot)
7
Commonsense Reasoning Suite OBQA, WinoGrande, ARC-c, ARC-e, HellaSwag, SIQA, PIQA
29.6OBQA Score
5
Commonsense Reasoning Suite (test)
48.27ARC-c
5
Commonsense Reasoning Suite (MMLU, ARC, PIQA, HellaSwag, OpenBookQA, Winogrande)
63.84Average Accuracy
4