# Question Answering and Commonsense Reasoning on LM Eval ARCC, ARCE, HellaSwag, PIQA (test) Source: [Question Answering and Commonsense Reasoning on LM Eval ARCC, ARCE, HellaSwag, PIQA (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:23:39.419Z Evaluation covering multiple commonsense reasoning and question answering tasks (ARCC, ARCE, HellaSwag, PIQA) using the standard test splits. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering and Commonsense Reasoning | | Dataset (including any stated split) | LM Eval ARCC, ARCE, HellaSwag, PIQA 0.4.4 standard (test) | | Metric | ARCC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ARCC](): Higher is better; unit: PERCENT. - [ARCE](): Higher is better; unit: PERCENT. - [HellaSwag](): Higher is better; unit: PERCENT. - [PIQA](): Higher is better; unit: PERCENT. ## Recorded results | Method | ARCC | Result date | Report paper | | --- | --- | --- | --- | | Meta 'FP8' | 61.6 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 61.5 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 61.3 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 60.7 | Not recorded | 68fa798a1e209131cea7e211 | | BF16 | 56.7 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 56.3 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 55.1 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 54.4 | Not recorded | 68fa798a1e209131cea7e211 | | PV-Tuning | 52.7 | Not recorded | 68fa798a1e209131cea7e211 | | BF16 | 51.6 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 50.7 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 50.4 | Not recorded | 68fa798a1e209131cea7e211 | | PV-Tuning | 46.2 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 45.1 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 43.5 | Not recorded | 68fa798a1e209131cea7e211 | | BF16 | 43.3 | Not recorded | 68fa798a1e209131cea7e211 | | BF16 | 36 | Not recorded | 68fa798a1e209131cea7e211 | | QTIP | 34.8 | Not recorded | 68fa798a1e209131cea7e211 | ### Meta 'FP8' ARCC: 61.6. Result ID: 698b531d8d67a41d0a6d1aff. [Reporting paper]() Recorded evaluation configuration: {"Bits":"16","Model":"Llama 3.1 405B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 61.5. Result ID: 698b531d8d67a41d0a6d1b01. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"3","Model":"Llama 3.1 405B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 61.3. Result ID: 698b531d8d67a41d0a6d1b00. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"4","Model":"Llama 3.1 405B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 60.7. Result ID: 698b531d8d67a41d0a6d1b02. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"2","Model":"Llama 3.1 405B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### BF16 ARCC: 56.7. Result ID: 698b531d8d67a41d0a6d1b03. [Reporting paper]() Recorded evaluation configuration: {"Bits":"16","Model":"Llama 3.1 70B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 56.3. Result ID: 698b531d8d67a41d0a6d1b04. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"4","Model":"Llama 3.1 70B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 55.1. Result ID: 698b531d8d67a41d0a6d1b05. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"3","Model":"Llama 3.1 70B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 54.4. Result ID: 698b531d8d67a41d0a6d1b06. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"2","Model":"Llama 3.1 70B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### PV-Tuning ARCC: 52.7. Result ID: 698b531d8d67a41d0a6d1b07. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"2.01","Model":"Llama 3.1 70B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### BF16 ARCC: 51.6. Result ID: 698b531d8d67a41d0a6d1b08. [Reporting paper]() Recorded evaluation configuration: {"Bits":"16","Model":"Llama 3.1 8B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 50.7. Result ID: 698b531d8d67a41d0a6d1b09. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"4","Model":"Llama 3.1 8B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 50.4. Result ID: 698b531d8d67a41d0a6d1b0a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"3","Model":"Llama 3.1 8B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### PV-Tuning ARCC: 46.2. Result ID: 698b531d8d67a41d0a6d1b0c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"2.07","Model":"Llama 3.1 8B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 45.1. Result ID: 698b531d8d67a41d0a6d1b0b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"2","Model":"Llama 3.1 8B Instruct","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### QTIP ARCC: 43.5. Result ID: 698c52dd37894ef7d9cd810f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"Llama 3.2 3B","Size (GB)":"2.1","Precision":"4-bit"}. This record does not assert independent reproduction. ### BF16 ARCC: 43.3. Result ID: 698c52dd37894ef7d9cd810e. [Reporting paper]() Recorded evaluation configuration: {"Backbone Model":"Llama 3.2 3B","Size (GB)":"6","Precision":"16-bit"}. This record does not assert independent reproduction. ### BF16 ARCC: 36. Result ID: 698c52dd37894ef7d9cd8110. [Reporting paper]() Recorded evaluation configuration: {"Backbone Model":"Llama 3.2 1B","Size (GB)":"2.4","Precision":"16-bit"}. This record does not assert independent reproduction. ### QTIP ARCC: 34.8. Result ID: 698c52dd37894ef7d9cd8111. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"Llama 3.2 1B","Size (GB)":"0.97","Precision":"4-bit"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.