# Zero-shot Reasoning and Question Answering on (ARC-C, ARC-E, H'SWAG, LAM'DA, OPENBKQA, PIQA, W'GRANDE) Source: [Zero-shot Reasoning and Question Answering on (ARC-C, ARC-E, H'SWAG, LAM'DA, OPENBKQA, PIQA, W'GRANDE)]() Collection index: [llms.txt]() Data updated: 2026-03-05T07:42:19.357Z Evaluation of zero-shot performance across a suite of seven reasoning and question answering benchmarks: ARC-Challenge, ARC-Easy, HellaSwag, LAMBADA, OpenBookQA, PIQA, and Winogrande. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Zero-shot Reasoning and Question Answering | | Dataset (including any stated split) | Downstream Tasks (ARC-C, ARC-E, H'SWAG, LAM'DA, OPENBKQA, PIQA, W'GRANDE) zero-shot | | Metric | ARC-C Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 9 of 9 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ARC-C Score](): Higher is better; unit: SCALAR. - [ARC-E Score](): Higher is better; unit: SCALAR. - [H'SWAG Score](): Higher is better; unit: SCALAR. - [LAM'DA Score](): Higher is better; unit: SCALAR. - [OpenBookQA Score](): Higher is better; unit: SCALAR. - [PIQA Score](): Higher is better; unit: SCALAR. - [W'GRANDE Score](): Higher is better; unit: SCALAR. - [Average Score (Zero-shot)](): Higher is better; unit: SCALAR. ## Recorded results | Method | ARC-C Score | Result date | Report paper | | --- | --- | --- | --- | | ASVD | 32.76 | Not recorded | 69a90dec2d603e6de3e406b6 | | SVD-LLM | 31.66 | Not recorded | 69a90dec2d603e6de3e406b6 | | Dobi-SVD | 31.31 | Not recorded | 69a90dec2d603e6de3e406b6 | | Dobi-SVD | 30.72 | Not recorded | 69a90dec2d603e6de3e406b6 | | SVD-LLM | 28.41 | Not recorded | 69a90dec2d603e6de3e406b6 | | Dobi-SVD | 28.16 | Not recorded | 69a90dec2d603e6de3e406b6 | | SVD-LLM | 26.19 | Not recorded | 69a90dec2d603e6de3e406b6 | | ASVD | 24.66 | Not recorded | 69a90dec2d603e6de3e406b6 | | ASVD | 22.61 | Not recorded | 69a90dec2d603e6de3e406b6 | ### ASVD ARC-C Score: 32.76. Result ID: 69a933cb6dbb7075b9524fbd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Optimizer":"NUMUON","Compression Rate":"40%","Backbone Model":"Llama3-1.8B"}. This record does not assert independent reproduction. ### SVD-LLM ARC-C Score: 31.66. Result ID: 69a933cb6dbb7075b9524fc0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Optimizer":"NUMUON","Compression Rate":"40%","Backbone Model":"Llama3-1.8B"}. This record does not assert independent reproduction. ### Dobi-SVD ARC-C Score: 31.31. Result ID: 69a933cb6dbb7075b9524fc3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Optimizer":"NUMUON","Compression Rate":"40%","Backbone Model":"Llama3-1.8B"}. This record does not assert independent reproduction. ### Dobi-SVD ARC-C Score: 30.72. Result ID: 69a933cb6dbb7075b9524fc1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Optimizer":"ADAMW","Compression Rate":"40%","Backbone Model":"Llama3-1.8B"}. This record does not assert independent reproduction. ### SVD-LLM ARC-C Score: 28.41. Result ID: 69a933cb6dbb7075b9524fbe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Optimizer":"ADAMW","Compression Rate":"40%","Backbone Model":"Llama3-1.8B"}. This record does not assert independent reproduction. ### Dobi-SVD ARC-C Score: 28.16. Result ID: 69a933cb6dbb7075b9524fc2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Optimizer":"MUON","Compression Rate":"40%","Backbone Model":"Llama3-1.8B"}. This record does not assert independent reproduction. ### SVD-LLM ARC-C Score: 26.19. Result ID: 69a933cb6dbb7075b9524fbf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Optimizer":"MUON","Compression Rate":"40%","Backbone Model":"Llama3-1.8B"}. This record does not assert independent reproduction. ### ASVD ARC-C Score: 24.66. Result ID: 69a933cb6dbb7075b9524fbc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Optimizer":"MUON","Compression Rate":"40%","Backbone Model":"Llama3-1.8B"}. This record does not assert independent reproduction. ### ASVD ARC-C Score: 22.61. Result ID: 69a933cb6dbb7075b9524fbb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Optimizer":"ADAMW","Compression Rate":"40%","Backbone Model":"Llama3-1.8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.