# Zero-shot Commonsense Reasoning Suite (PIQA, HellaSwag, WinoGrande, ARC, SocialIQA, BoolQ)

Source: [Zero-shot Commonsense Reasoning Suite (PIQA, HellaSwag, WinoGrande, ARC, SocialIQA, BoolQ)](<https://www.sota2.com/research/sota/zero-shot-commonsense-reasoning-suite-piqa-hellaswag-winogrande-arc-socialiqa-boolq>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-14T08:40:55.754Z

Evaluation of zero-shot model performance across a suite of six core commonsense reasoning benchmarks: PIQA, HellaSwag, WinoGrande, ARC (including both easy and challenge splits), SocialIQA, and BoolQ.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Commonsense Reasoning |
| Dataset (including any stated split) | PIQA, HellaSwag, WinoGrande, ARC, SocialIQA, BoolQ zero-shot |
| Metric | PIQA Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 12 of 12 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [PIQA Accuracy](<https://www.sota2.com/research/sota/zero-shot-commonsense-reasoning-suite-piqa-hellaswag-winogrande-arc-socialiqa-boolq.md?metric=PIQA%20Accuracy>): Higher is better; unit: PERCENT.

- [HellaSwag Accuracy](<https://www.sota2.com/research/sota/zero-shot-commonsense-reasoning-suite-piqa-hellaswag-winogrande-arc-socialiqa-boolq.md?metric=HellaSwag%20Accuracy>): Higher is better; unit: PERCENT.

- [WinoGrande Accuracy](<https://www.sota2.com/research/sota/zero-shot-commonsense-reasoning-suite-piqa-hellaswag-winogrande-arc-socialiqa-boolq.md?metric=WinoGrande%20Accuracy>): Higher is better; unit: PERCENT.

- [ARC-e Accuracy](<https://www.sota2.com/research/sota/zero-shot-commonsense-reasoning-suite-piqa-hellaswag-winogrande-arc-socialiqa-boolq.md?metric=ARC-e%20Accuracy>): Higher is better; unit: PERCENT.

- [ARC-c Accuracy](<https://www.sota2.com/research/sota/zero-shot-commonsense-reasoning-suite-piqa-hellaswag-winogrande-arc-socialiqa-boolq.md?metric=ARC-c%20Accuracy>): Higher is better; unit: PERCENT.

- [SocialIQA Accuracy](<https://www.sota2.com/research/sota/zero-shot-commonsense-reasoning-suite-piqa-hellaswag-winogrande-arc-socialiqa-boolq.md?metric=SocialIQA%20Accuracy>): Higher is better; unit: PERCENT.

- [BoolQ Accuracy](<https://www.sota2.com/research/sota/zero-shot-commonsense-reasoning-suite-piqa-hellaswag-winogrande-arc-socialiqa-boolq.md?metric=BoolQ%20Accuracy>): Higher is better; unit: PERCENT.

- [Average Accuracy](<https://www.sota2.com/research/sota/zero-shot-commonsense-reasoning-suite-piqa-hellaswag-winogrande-arc-socialiqa-boolq.md?metric=Average%20Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | PIQA Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| DNH-HOPE | 74.15 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| Titans (LMM) | 73.09 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| DeltaNet | 70.72 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| DNH-HOPE | 70.45 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| HOPE | 70.13 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| RetNet | 70.07 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| Transformer++ | 70.02 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| Titans (LMM) | 69.28 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| RetNet | 67.19 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| DeltaNet | 66.93 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| Transformer++ | 66.92 | Not recorded | 691ea0a0d6b0dd334292dfe0 |
| HOPE | 64.62 | Not recorded | 691ea0a0d6b0dd334292dfe0 |

### DNH-HOPE

PIQA Accuracy: 74.15. Result ID: 69ddfc3247f34ce924c56d37.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Titans (LMM)

PIQA Accuracy: 73.09. Result ID: 69ddfc3247f34ce924c56d35.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa766d1e209131cea6ce9b.md>)

Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### DeltaNet

PIQA Accuracy: 70.72. Result ID: 69ddfc3247f34ce924c56d34.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### DNH-HOPE

PIQA Accuracy: 70.45. Result ID: 69ddfc3247f34ce924c56d31.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

Recorded evaluation configuration: {"Model Parameters":"760M","Training Tokens":"30B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### HOPE

PIQA Accuracy: 70.13. Result ID: 69ddfc3247f34ce924c56d36.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6955ff9353fdd5c36173e0c4.md>)

Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### RetNet

PIQA Accuracy: 70.07. Result ID: 69ddfc3247f34ce924c56d33.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70471e209131cea4a24a.md>)

Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Transformer++

PIQA Accuracy: 70.02. Result ID: 69ddfc3247f34ce924c56d32.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa703d1e209131cea49db1.md>)

[Implementation](<https://github.com/tensorflow/tensor2tensor>)

Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Titans (LMM)

PIQA Accuracy: 69.28. Result ID: 69ddfc3247f34ce924c56d2f.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa766d1e209131cea6ce9b.md>)

Recorded evaluation configuration: {"Model Parameters":"760M","Training Tokens":"30B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### RetNet

PIQA Accuracy: 67.19. Result ID: 69ddfc3247f34ce924c56d2d.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70471e209131cea4a24a.md>)

Recorded evaluation configuration: {"Model Parameters":"760M","Training Tokens":"30B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### DeltaNet

PIQA Accuracy: 66.93. Result ID: 69ddfc3247f34ce924c56d2e.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

Recorded evaluation configuration: {"Model Parameters":"760M","Training Tokens":"30B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Transformer++

PIQA Accuracy: 66.92. Result ID: 69ddfc3247f34ce924c56d2c.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa703d1e209131cea49db1.md>)

[Implementation](<https://github.com/tensorflow/tensor2tensor>)

Recorded evaluation configuration: {"Model Parameters":"760M","Training Tokens":"30B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### HOPE

PIQA Accuracy: 64.62. Result ID: 69ddfc3247f34ce924c56d30.

[Reporting paper](<https://www.sota2.com/research/paper/691ea0a0d6b0dd334292dfe0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6955ff9353fdd5c36173e0c4.md>)

Recorded evaluation configuration: {"Model Parameters":"760M","Training Tokens":"30B","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
