# Commonsense Reasoning and Knowledge on ARC-C, BoolQ, Lambada, PIQA, Winogrande Source: [Commonsense Reasoning and Knowledge on ARC-C, BoolQ, Lambada, PIQA, Winogrande]() Collection index: [llms.txt]() Data updated: 2026-03-31T09:10:41.469Z Evaluation of model performance across five specific commonsense reasoning benchmarks: ARC-Challenge (ARC-C), BoolQ, Lambada, PIQA, and Winogrande, reporting individual scores and the overall average. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Commonsense Reasoning and Knowledge | | Dataset (including any stated split) | lm-eval ARC-C, BoolQ, Lambada, PIQA, Winogrande | | Metric | ARC-C Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 8 of 8 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ARC-C Accuracy](): Higher is better; unit: PERCENT. - [BoolQ Accuracy](): Higher is better; unit: PERCENT. - [Lambada Accuracy](): Higher is better; unit: PERCENT. - [PIQA Accuracy](): Higher is better; unit: PERCENT. - [Winogrande Accuracy](): Higher is better; unit: PERCENT. - [Average Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | ARC-C Accuracy | Result date | Report paper | | --- | --- | --- | --- | | FP16 | 53.58 | Not recorded | 68fa7a6a1e209131cea83140 | | MicroMix | 52.22 | Not recorded | 68fa7a6a1e209131cea83140 | | FlatQuant | 50.17 | Not recorded | 68fa7a6a1e209131cea83140 | | QUIK | 49.66 | Not recorded | 68fa7a6a1e209131cea83140 | | INT6 | 49.15 | Not recorded | 68fa7a6a1e209131cea83140 | | Atom | 47.95 | Not recorded | 68fa7a6a1e209131cea83140 | | AMXFP4 | 46.84 | Not recorded | 68fa7a6a1e209131cea83140 | | QuaRot | 44.88 | Not recorded | 68fa7a6a1e209131cea83140 | ### FP16 ARC-C Accuracy: 53.58. Result ID: 69cb8e9347f34ce924b41108. [Reporting paper]() Recorded evaluation configuration: {"Avg. Bits":"16.00","Base Model":"Llama3.1-8B","Evaluation Protocol":"0-shot","Quantization":"None"}. This record does not assert independent reproduction. ### MicroMix ARC-C Accuracy: 52.22. Result ID: 69cb8e9347f34ce924b4110f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Avg. Bits":"5.51","Base Model":"Llama3.1-8B","Evaluation Protocol":"0-shot","KV Cache Quantization":"Asymmetric INT4 (group size 64)"}. This record does not assert independent reproduction. ### FlatQuant ARC-C Accuracy: 50.17. Result ID: 69cb8e9347f34ce924b4110c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Avg. Bits":"4.19","Base Model":"Llama3.1-8B","Evaluation Protocol":"0-shot","KV Cache Quantization":"Asymmetric INT4 (group size 64)"}. This record does not assert independent reproduction. ### QUIK ARC-C Accuracy: 49.66. Result ID: 69cb8e9347f34ce924b4110a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Avg. Bits":"5.95","Base Model":"Llama3.1-8B","Evaluation Protocol":"0-shot","KV Cache Quantization":"Asymmetric INT4 (group size 64)"}. This record does not assert independent reproduction. ### INT6 ARC-C Accuracy: 49.15. Result ID: 69cb8e9347f34ce924b4110e. [Reporting paper]() Recorded evaluation configuration: {"Avg. Bits":"6.00","Base Model":"Llama3.1-8B","Evaluation Protocol":"0-shot","KV Cache Quantization":"Asymmetric INT4 (group size 64)"}. This record does not assert independent reproduction. ### Atom ARC-C Accuracy: 47.95. Result ID: 69cb8e9347f34ce924b4110b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Avg. Bits":"4.25","Base Model":"Llama3.1-8B","Evaluation Protocol":"0-shot","KV Cache Quantization":"Asymmetric INT4 (group size 64)"}. This record does not assert independent reproduction. ### AMXFP4 ARC-C Accuracy: 46.84. Result ID: 69cb8e9347f34ce924b4110d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Avg. Bits":"5.00","Base Model":"Llama3.1-8B","Evaluation Protocol":"0-shot","KV Cache Quantization":"Asymmetric INT4 (group size 64)"}. This record does not assert independent reproduction. ### QuaRot ARC-C Accuracy: 44.88. Result ID: 69cb8e9347f34ce924b41109. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Avg. Bits":"4.12","Base Model":"Llama3.1-8B","Evaluation Protocol":"0-shot","KV Cache Quantization":"Asymmetric INT4 (group size 64)"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.