# Zero-shot Question Answering on Reasoning Suite (ARC-C, ARC-E, BoolQ, HellaSwag, PIQA, WinoGrande)

Source: [Zero-shot Question Answering on Reasoning Suite (ARC-C, ARC-E, BoolQ, HellaSwag, PIQA, WinoGrande)](<https://www.sota2.com/research/sota/zero-shot-question-answering-on-reasoning-suite-arc-c-arc-e-boolq-hellaswag-piqa-winogrande>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-11T07:55:45.035Z

Evaluation of zero-shot question answering performance across the specific suite of reasoning benchmarks: ARC-Challenge, ARC-Easy, BoolQ, HellaSwag, PIQA, and WinoGrande.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Zero-shot Question Answering |
| Dataset (including any stated split) | Reasoning Benchmarks ARC-C, ARC-E, BoolQ, HellaSwag, PIQA, WinoGrande |
| Metric | ARC-C Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 3 of 3 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [ARC-C Accuracy](<https://www.sota2.com/research/sota/zero-shot-question-answering-on-reasoning-suite-arc-c-arc-e-boolq-hellaswag-piqa-winogrande.md?metric=ARC-C%20Accuracy>): Higher is better; unit: PERCENT.

- [ARC-E Accuracy](<https://www.sota2.com/research/sota/zero-shot-question-answering-on-reasoning-suite-arc-c-arc-e-boolq-hellaswag-piqa-winogrande.md?metric=ARC-E%20Accuracy>): Higher is better; unit: PERCENT.

- [BoolQ Accuracy](<https://www.sota2.com/research/sota/zero-shot-question-answering-on-reasoning-suite-arc-c-arc-e-boolq-hellaswag-piqa-winogrande.md?metric=BoolQ%20Accuracy>): Higher is better; unit: PERCENT.

- [HellaSwag Accuracy](<https://www.sota2.com/research/sota/zero-shot-question-answering-on-reasoning-suite-arc-c-arc-e-boolq-hellaswag-piqa-winogrande.md?metric=HellaSwag%20Accuracy>): Higher is better; unit: PERCENT.

- [PIQA Accuracy](<https://www.sota2.com/research/sota/zero-shot-question-answering-on-reasoning-suite-arc-c-arc-e-boolq-hellaswag-piqa-winogrande.md?metric=PIQA%20Accuracy>): Higher is better; unit: PERCENT.

- [WinoGrande Accuracy](<https://www.sota2.com/research/sota/zero-shot-question-answering-on-reasoning-suite-arc-c-arc-e-boolq-hellaswag-piqa-winogrande.md?metric=WinoGrande%20Accuracy>): Higher is better; unit: PERCENT.

- [Average Accuracy (Overall)](<https://www.sota2.com/research/sota/zero-shot-question-answering-on-reasoning-suite-arc-c-arc-e-boolq-hellaswag-piqa-winogrande.md?metric=Average%20Accuracy%20(Overall)>): Higher is better; unit: PERCENT.

- [Performance Ratio](<https://www.sota2.com/research/sota/zero-shot-question-answering-on-reasoning-suite-arc-c-arc-e-boolq-hellaswag-piqa-winogrande.md?metric=Performance%20Ratio>): Higher is better; unit: SCALAR.

## Recorded results

| Method | ARC-C Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| FP16 | 60.15 | Not recorded | 6a28f32990966ade919c16b9 |
| LC-QAT | 54.69 | Not recorded | 6a28f32990966ade919c16b9 |
| LC-PTQ | 51.62 | Not recorded | 6a28f32990966ade919c16b9 |

### FP16

ARC-C Accuracy: 60.15. Result ID: 6a2a69edfbc373592c274bcf.

[Reporting paper](<https://www.sota2.com/research/paper/6a28f32990966ade919c16b9.md>)

Recorded evaluation configuration: {"Model Scale":"14B","Calibration Tokens":"58M"}. This record does not assert independent reproduction.

### LC-QAT

ARC-C Accuracy: 54.69. Result ID: 6a2a69edfbc373592c274bd1.

[Reporting paper](<https://www.sota2.com/research/paper/6a28f32990966ade919c16b9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a28f32990966ade919c16b9.md>)

Recorded evaluation configuration: {"Model Scale":"14B","Calibration Tokens":"58M"}. This record does not assert independent reproduction.

### LC-PTQ

ARC-C Accuracy: 51.62. Result ID: 6a2a69edfbc373592c274bd0.

[Reporting paper](<https://www.sota2.com/research/paper/6a28f32990966ade919c16b9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a28f32990966ade919c16b9.md>)

Recorded evaluation configuration: {"Model Scale":"14B","Calibration Tokens":"58M"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
