Question Answering on PIQA (Normalized Accuracy)
73.56Normalized AccuracyComposite: Stacked
Evaluation results
| Method | Links | |
|---|---|---|
| Composite: StackedModel Architecture=Composer (Stacked), Model Parameter Scale=1B, Pre-training Dataset=DCLM, Pre-training Tokens=37.5B2025.10 | 73.56 | |
| 1:2 Striped Attn.Model Architecture=Striped Attention (1:2 ratio), Model Parameter Scale=1B, Pre-training Dataset=DCLM, Pre-training Tokens=37.5B2025.10 | 72.6 | |
| Composite: StretchedModel Architecture=Composer (Stretched), Model Parameter Scale=1B, Pre-training Dataset=DCLM, Pre-training Tokens=37.5B2025.10 | 72.3 | |
| 1:4 Striped Attn.Model Architecture=Striped Attention (1:4 ratio), Model Parameter Scale=1B, Pre-training Dataset=DCLM, Pre-training Tokens=37.5B2025.10 | 71.9 | |
| 1:8 Striped Attn.Model Architecture=Striped Attention (1:8 ratio), Model Parameter Scale=1B, Pre-training Dataset=DCLM, Pre-training Tokens=37.5B2025.10 | 71.9 | |
| Llama 3.2Model Architecture=Llama 3.2, Model Parameter Scale=1B, Pre-training Dataset=DCLM, Pre-training Tokens=37.5B2025.10 | 71.8 | |
| STAR*Model Architecture=STAR, Model Parameter Scale=1B, Pre-training Dataset=Original Paper Dataset2025.10 | 71.8 | |
| Sand. TransformerModel Architecture=Sandwich Transformer, Model Parameter Scale=1B, Pre-training Dataset=DCLM, Pre-training Tokens=37.5B2025.10 | 71.5 | |
| LSSARzero-shot=true, p=152025.01 | 65.34 | |
| Softmaxzero-shot=true2025.01 | 64.09 | |
| Regularized BaselineParameters=1.4B, Training Data=DCLM, Training Tokens=100M, Evaluation=Zero-shot2026.06 | 59.96 | |
| MIRParameters=1.4B, Training Data=DCLM, Training Tokens=100M, Evaluation=Zero-shot2026.06 | 59.85 | |
| Random GuessParameters=1.4B, Training Data=DCLM, Training Tokens=100M, Evaluation=Zero-shot2026.06 | 50 |