Commonsense Reasoning on PIQA (Accuracy, Normalized Accuracy)
74AccuracyNemotron-2 Approx.
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Nemotron-2 Approx.Shot=0-shot2026.05 | 74 | 73.9 | |
| AIRAhybrid-AShot=0-shot, Architecture Variant=Stretched2026.05 | 73.9 | 73.4 | |
| AIRAhybrid-DShot=0-shot, Architecture Variant=Stretched2026.05 | 73.6 | 74.7 | |
| AIRAhybrid-BShot=0-shot, Architecture Variant=Stretched2026.05 | 73.4 | 74 | |
| Nemotron-H Approx.Shot=0-shot2026.05 | 73.3 | 73.7 | |
| AIRAhybrid-BShot=0-shot, Architecture Variant=Stacked2026.05 | 73.1 | 73.4 | |
| Mamba (Mb + M)Shot=0-shot2026.05 | 72.9 | 73.1 | |
| AIRAhybrid-CShot=0-shot, Architecture Variant=Stretched2026.05 | 72.8 | 73.6 | |
| AIRAhybrid-DShot=0-shot, Architecture Variant=Stacked2026.05 | 72.8 | 73.7 | |
| AIRAhybrid-EShot=0-shot, Architecture Variant=Stretched2026.05 | 72.3 | 72.9 | |
| AIRAhybrid-EShot=0-shot, Architecture Variant=Stacked2026.05 | 72.1 | 73.3 | |
| Composer (2Mb-M-3A)Shot=0-shot2026.05 | 71.9 | 73.9 |