# Commonsense Reasoning on Commonsense Reasoning Suite (Zero-shot) Source: [Commonsense Reasoning on Commonsense Reasoning Suite (Zero-shot)]() Collection index: [llms.txt]() Data updated: 2026-06-26T08:06:02.968Z Evaluation of zero-shot performance on commonsense reasoning, reporting scores for PIQA, SIQA, HellaSwag, ARC-E, ARC-C, WinoGrande, and LAMBADA. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Commonsense Reasoning | | Dataset (including any stated split) | Commonsense Reasoning Suite Zero-shot | | Metric | PIQA Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [PIQA Accuracy](): Higher is better; unit: PERCENT. - [SIQA Accuracy](): Higher is better; unit: PERCENT. - [HellaSwag Accuracy](): Higher is better; unit: PERCENT. - [ARC-E Accuracy](): Higher is better; unit: PERCENT. - [ARC-C Accuracy](): Higher is better; unit: PERCENT. - [WinoGrande Accuracy](): Higher is better; unit: PERCENT. - [LAMBADA Accuracy](): Higher is better; unit: PERCENT. - [Average Accuracy (Zero-shot)](): Higher is better; unit: PERCENT. ## Recorded results | Method | PIQA Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Llama2-70B | 82.81 | Not recorded | 6a3e07bb206542251d7cb62a | | CAT-Q | 80.34 | Not recorded | 6a3e07bb206542251d7cb62a | | EfficientQAT | 80.2 | Not recorded | 6a3e07bb206542251d7cb62a | | DB-LLM | 79.27 | Not recorded | 6a3e07bb206542251d7cb62a | | Llama2-7B | 78.84 | Not recorded | 6a3e07bb206542251d7cb62a | | SliderQuant | 78.79 | Not recorded | 6a3e07bb206542251d7cb62a | | OmniQuant | 74.1 | Not recorded | 6a3e07bb206542251d7cb62a | | DB-LLM | 73.18 | Not recorded | 6a3e07bb206542251d7cb62a | | CAT-Q | 72.91 | Not recorded | 6a3e07bb206542251d7cb62a | | SliderQuant | 70.78 | Not recorded | 6a3e07bb206542251d7cb62a | | OmniQuant | 65.13 | Not recorded | 6a3e07bb206542251d7cb62a | | DECO (NG) | 64.36 | Not recorded | 6a02b4b4dbc836b87d824fcd | | BlockFFN | 63.38 | Not recorded | 6a02b4b4dbc836b87d824fcd | | DeepSeek-V3 (NG) | 63.28 | Not recorded | 6a02b4b4dbc836b87d824fcd | | ReMoE (NG) | 63.22 | Not recorded | 6a02b4b4dbc836b87d824fcd | | Dense | 63 | Not recorded | 6a02b4b4dbc836b87d824fcd | | DeepSeek-V3 (GA) | 62.79 | Not recorded | 6a02b4b4dbc836b87d824fcd | | ReMoE (GA) | 62.62 | Not recorded | 6a02b4b4dbc836b87d824fcd | | DECO (GA) | 61.86 | Not recorded | 6a02b4b4dbc836b87d824fcd | | TopP | 61.37 | Not recorded | 6a02b4b4dbc836b87d824fcd | | BiLLM | 60.6 | Not recorded | 6a3e07bb206542251d7cb62a | | GPTQ | 58.32 | Not recorded | 6a3e07bb206542251d7cb62a | | PB-LLM | 55.22 | Not recorded | 6a3e07bb206542251d7cb62a | | AWQ | 50 | Not recorded | 6a3e07bb206542251d7cb62a | | GPTQ | 49.51 | Not recorded | 6a3e07bb206542251d7cb62a | ### Llama2-70B PIQA Accuracy: 82.81. Result ID: 6a3e32e9fbc373592c5fd2ec. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Bits":"W16A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-70B"}. This record does not assert independent reproduction. ### CAT-Q PIQA Accuracy: 80.34. Result ID: 6a3e32e9fbc373592c5fd2f2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W1.58A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-70B"}. This record does not assert independent reproduction. ### EfficientQAT PIQA Accuracy: 80.2. Result ID: 6a3e32e9fbc373592c5fd2f0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Bits":"W2A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-70B"}. This record does not assert independent reproduction. ### DB-LLM PIQA Accuracy: 79.27. Result ID: 6a3e32e9fbc373592c5fd2ef. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W1\*A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-70B"}. This record does not assert independent reproduction. ### Llama2-7B PIQA Accuracy: 78.84. Result ID: 6a3e32e9fbc373592c5fd2e3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Bits":"W16A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-7B"}. This record does not assert independent reproduction. ### SliderQuant PIQA Accuracy: 78.79. Result ID: 6a3e32e9fbc373592c5fd2f1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W2A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-70B"}. This record does not assert independent reproduction. ### OmniQuant PIQA Accuracy: 74.1. Result ID: 6a3e32e9fbc373592c5fd2ee. [Reporting paper]() Recorded evaluation configuration: {"#Bits":"W2A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-70B"}. This record does not assert independent reproduction. ### DB-LLM PIQA Accuracy: 73.18. Result ID: 6a3e32e9fbc373592c5fd2e9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W1\*A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-7B"}. This record does not assert independent reproduction. ### CAT-Q PIQA Accuracy: 72.91. Result ID: 6a3e32e9fbc373592c5fd2eb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W1.58A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-7B"}. This record does not assert independent reproduction. ### SliderQuant PIQA Accuracy: 70.78. Result ID: 6a3e32e9fbc373592c5fd2ea. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W2A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-7B"}. This record does not assert independent reproduction. ### OmniQuant PIQA Accuracy: 65.13. Result ID: 6a3e32e9fbc373592c5fd2e6. [Reporting paper]() Recorded evaluation configuration: {"#Bits":"W2A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-7B"}. This record does not assert independent reproduction. ### DECO (NG) PIQA Accuracy: 64.36. Result ID: 6a035d8b47f34ce924e906a8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"Medium","Routing Strategy":"NG"}. This record does not assert independent reproduction. ### BlockFFN PIQA Accuracy: 63.38. Result ID: 6a035d8b47f34ce924e906a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Medium"}. This record does not assert independent reproduction. ### DeepSeek-V3 (NG) PIQA Accuracy: 63.28. Result ID: 6a035d8b47f34ce924e906a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Medium","Routing Strategy":"NG"}. This record does not assert independent reproduction. ### ReMoE (NG) PIQA Accuracy: 63.22. Result ID: 6a035d8b47f34ce924e906a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Medium","Routing Strategy":"NG"}. This record does not assert independent reproduction. ### Dense PIQA Accuracy: 63. Result ID: 6a035d8b47f34ce924e906a7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"Medium","Model Architecture":"Dense"}. This record does not assert independent reproduction. ### DeepSeek-V3 (GA) PIQA Accuracy: 62.79. Result ID: 6a035d8b47f34ce924e906a1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Medium","Routing Strategy":"GA"}. This record does not assert independent reproduction. ### ReMoE (GA) PIQA Accuracy: 62.62. Result ID: 6a035d8b47f34ce924e906a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Medium","Routing Strategy":"GA"}. This record does not assert independent reproduction. ### DECO (GA) PIQA Accuracy: 61.86. Result ID: 6a035d8b47f34ce924e906a6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"Medium","Routing Strategy":"GA"}. This record does not assert independent reproduction. ### TopP PIQA Accuracy: 61.37. Result ID: 6a035d8b47f34ce924e906a0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Medium"}. This record does not assert independent reproduction. ### BiLLM PIQA Accuracy: 60.6. Result ID: 6a3e32e9fbc373592c5fd2e8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W1\*A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-7B"}. This record does not assert independent reproduction. ### GPTQ PIQA Accuracy: 58.32. Result ID: 6a3e32e9fbc373592c5fd2e5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W2A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-7B"}. This record does not assert independent reproduction. ### PB-LLM PIQA Accuracy: 55.22. Result ID: 6a3e32e9fbc373592c5fd2e7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W1\*A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-7B"}. This record does not assert independent reproduction. ### AWQ PIQA Accuracy: 50. Result ID: 6a3e32e9fbc373592c5fd2e4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W2A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-7B"}. This record does not assert independent reproduction. ### GPTQ PIQA Accuracy: 49.51. Result ID: 6a3e32e9fbc373592c5fd2ed. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Bits":"W2A16","Zero-shot evaluation protocol":"true","Backbone":"Llama2-70B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.