# Zero-shot Commonsense Reasoning Suite Subset (ARC-c, BoolQ, PIQA, HellaSwag, WinoGrande) Source: [Zero-shot Commonsense Reasoning Suite Subset (ARC-c, BoolQ, PIQA, HellaSwag, WinoGrande)]() Collection index: [llms.txt]() Data updated: 2026-05-27T08:50:43.267Z Evaluation of zero-shot performance across a specific subset of the Commonsense Reasoning Suite, comprising ARC-Challenge (ARC-c), BoolQ, PIQA, HellaSwag, and WinoGrande. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Commonsense Reasoning | | Dataset (including any stated split) | Commonsense Reasoning Suite (ARC-c, BoolQ, PIQA, HellaSwag, WinoGrande) zero-shot | | Metric | ARC-c Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 35 of 35 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ARC-c Accuracy](): Higher is better; unit: PERCENT. - [BoolQ Accuracy](): Higher is better; unit: PERCENT. - [PIQA Accuracy](): Higher is better; unit: PERCENT. - [HellaSwag Accuracy](): Higher is better; unit: PERCENT. - [WinoGrande Accuracy](): Higher is better; unit: PERCENT. - [Average Zero-shot Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | ARC-c Accuracy | Result date | Report paper | | --- | --- | --- | --- | | FP | 55.03 | Not recorded | 6a167a7edbd7150506f7ff06 | | WINDQuant | 49.9 | Not recorded | 6a167a7edbd7150506f7ff06 | | FP | 46.3 | Not recorded | 6a167a7edbd7150506f7ff06 | | AQLM | 45.7 | Not recorded | 6a167a7edbd7150506f7ff06 | | WINDQuant | 37.8 | Not recorded | 6a167a7edbd7150506f7ff06 | | AQLM | 37.6 | Not recorded | 6a167a7edbd7150506f7ff06 | | FP | 36.7 | Not recorded | 6a167a7edbd7150506f7ff06 | | SignRound | 33.1 | Not recorded | 6a167a7edbd7150506f7ff06 | | SignRound | 31.3 | Not recorded | 6a167a7edbd7150506f7ff06 | | EfficientQAT | 31.1 | Not recorded | 6a167a7edbd7150506f7ff06 | | WINDQuant | 28 | Not recorded | 6a167a7edbd7150506f7ff06 | | SpinQuant | 27 | Not recorded | 6a167a7edbd7150506f7ff06 | | OmniQ | 26.2 | Not recorded | 6a167a7edbd7150506f7ff06 | | AWQ | 26 | Not recorded | 6a167a7edbd7150506f7ff06 | | SignRound | 26 | Not recorded | 6a167a7edbd7150506f7ff06 | | SpinQuant | 25.7 | Not recorded | 6a167a7edbd7150506f7ff06 | | LLM-QAT | 25.7 | Not recorded | 6a167a7edbd7150506f7ff06 | | AWQ | 25.7 | Not recorded | 6a167a7edbd7150506f7ff06 | | OmniQ | 25.7 | Not recorded | 6a167a7edbd7150506f7ff06 | | SpinQuant | 25.7 | Not recorded | 6a167a7edbd7150506f7ff06 | | AWQ | 25.5 | Not recorded | 6a167a7edbd7150506f7ff06 | | LLM-QAT | 25.2 | Not recorded | 6a167a7edbd7150506f7ff06 | | GPTQ | 25 | Not recorded | 6a167a7edbd7150506f7ff06 | | EfficientQAT | 24.8 | Not recorded | 6a167a7edbd7150506f7ff06 | | TesseraQ | 24.6 | Not recorded | 6a167a7edbd7150506f7ff06 | | EfficientQAT | 24.3 | Not recorded | 6a167a7edbd7150506f7ff06 | | OmniQ | 24.2 | Not recorded | 6a167a7edbd7150506f7ff06 | | GPTQ | 24.2 | Not recorded | 6a167a7edbd7150506f7ff06 | | GPTQ | 24 | Not recorded | 6a167a7edbd7150506f7ff06 | | GPTVQ | 23.8 | Not recorded | 6a167a7edbd7150506f7ff06 | | TesseraQ | 23.3 | Not recorded | 6a167a7edbd7150506f7ff06 | | TesseraQ | 22.1 | Not recorded | 6a167a7edbd7150506f7ff06 | | GPTVQ | 21.9 | Not recorded | 6a167a7edbd7150506f7ff06 | | LLM-QAT | 21.6 | Not recorded | 6a167a7edbd7150506f7ff06 | | GPTVQ | 20.7 | Not recorded | 6a167a7edbd7150506f7ff06 | ### FP ARC-c Accuracy: 55.03. Result ID: 6a16afdafbc373592c0fed50. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"16"}. This record does not assert independent reproduction. ### WINDQuant ARC-c Accuracy: 49.9. Result ID: 6a16afdafbc373592c0fed51. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"1.96"}. This record does not assert independent reproduction. ### FP ARC-c Accuracy: 46.3. Result ID: 6a16afdafbc373592c0fed44. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"16"}. This record does not assert independent reproduction. ### AQLM ARC-c Accuracy: 45.7. Result ID: 6a16afdafbc373592c0fed58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2.27"}. This record does not assert independent reproduction. ### WINDQuant ARC-c Accuracy: 37.8. Result ID: 6a16afdafbc373592c0fed45. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"1.99"}. This record does not assert independent reproduction. ### AQLM ARC-c Accuracy: 37.6. Result ID: 6a16afdafbc373592c0fed4c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2.27"}. This record does not assert independent reproduction. ### FP ARC-c Accuracy: 36.7. Result ID: 6a16afdafbc373592c0fed39. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"16"}. This record does not assert independent reproduction. ### SignRound ARC-c Accuracy: 33.1. Result ID: 6a16afdafbc373592c0fed5a. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2"}. This record does not assert independent reproduction. ### SignRound ARC-c Accuracy: 31.3. Result ID: 6a16afdafbc373592c0fed4e. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2"}. This record does not assert independent reproduction. ### EfficientQAT ARC-c Accuracy: 31.1. Result ID: 6a16afdafbc373592c0fed57. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2.12"}. This record does not assert independent reproduction. ### WINDQuant ARC-c Accuracy: 28. Result ID: 6a16afdafbc373592c0fed3a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2.01"}. This record does not assert independent reproduction. ### SpinQuant ARC-c Accuracy: 27. Result ID: 6a16afdafbc373592c0fed4a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2"}. This record does not assert independent reproduction. ### OmniQ ARC-c Accuracy: 26.2. Result ID: 6a16afdafbc373592c0fed3e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2"}. This record does not assert independent reproduction. ### AWQ ARC-c Accuracy: 26. Result ID: 6a16afdafbc373592c0fed3b. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2"}. This record does not assert independent reproduction. ### SignRound ARC-c Accuracy: 26. Result ID: 6a16afdafbc373592c0fed42. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2"}. This record does not assert independent reproduction. ### SpinQuant ARC-c Accuracy: 25.7. Result ID: 6a16afdafbc373592c0fed3f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2"}. This record does not assert independent reproduction. ### LLM-QAT ARC-c Accuracy: 25.7. Result ID: 6a16afdafbc373592c0fed48. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2"}. This record does not assert independent reproduction. ### AWQ ARC-c Accuracy: 25.7. Result ID: 6a16afdafbc373592c0fed52. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2"}. This record does not assert independent reproduction. ### OmniQ ARC-c Accuracy: 25.7. Result ID: 6a16afdafbc373592c0fed55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2"}. This record does not assert independent reproduction. ### SpinQuant ARC-c Accuracy: 25.7. Result ID: 6a16afdafbc373592c0fed56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2"}. This record does not assert independent reproduction. ### AWQ ARC-c Accuracy: 25.5. Result ID: 6a16afdafbc373592c0fed46. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2"}. This record does not assert independent reproduction. ### LLM-QAT ARC-c Accuracy: 25.2. Result ID: 6a16afdafbc373592c0fed3d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2"}. This record does not assert independent reproduction. ### GPTQ ARC-c Accuracy: 25. Result ID: 6a16afdafbc373592c0fed3c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2"}. This record does not assert independent reproduction. ### EfficientQAT ARC-c Accuracy: 24.8. Result ID: 6a16afdafbc373592c0fed4b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2.12"}. This record does not assert independent reproduction. ### TesseraQ ARC-c Accuracy: 24.6. Result ID: 6a16afdafbc373592c0fed4f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2"}. This record does not assert independent reproduction. ### EfficientQAT ARC-c Accuracy: 24.3. Result ID: 6a16afdafbc373592c0fed40. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2.12"}. This record does not assert independent reproduction. ### OmniQ ARC-c Accuracy: 24.2. Result ID: 6a16afdafbc373592c0fed49. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2"}. This record does not assert independent reproduction. ### GPTQ ARC-c Accuracy: 24.2. Result ID: 6a16afdafbc373592c0fed53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2"}. This record does not assert independent reproduction. ### GPTQ ARC-c Accuracy: 24. Result ID: 6a16afdafbc373592c0fed47. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2"}. This record does not assert independent reproduction. ### GPTVQ ARC-c Accuracy: 23.8. Result ID: 6a16afdafbc373592c0fed41. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2"}. This record does not assert independent reproduction. ### TesseraQ ARC-c Accuracy: 23.3. Result ID: 6a16afdafbc373592c0fed43. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 1B","#Bits":"2"}. This record does not assert independent reproduction. ### TesseraQ ARC-c Accuracy: 22.1. Result ID: 6a16afdafbc373592c0fed5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2"}. This record does not assert independent reproduction. ### GPTVQ ARC-c Accuracy: 21.9. Result ID: 6a16afdafbc373592c0fed59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2"}. This record does not assert independent reproduction. ### LLM-QAT ARC-c Accuracy: 21.6. Result ID: 6a16afdafbc373592c0fed54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 8B","#Bits":"2"}. This record does not assert independent reproduction. ### GPTVQ ARC-c Accuracy: 20.7. Result ID: 6a16afdafbc373592c0fed4d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3 3B","#Bits":"2"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.