Multiple Choice Question Answering Suite (ARC-E, ARC-C, SciQ, PIQA, MMLU, CMMLU, CEVAL)
59.21Average AccuracyNextLat (d=2)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| NextLat (d=2)Horizon (d)=2, Parameters=1.3B, Training tokens=100B2025.11 | 59.21 | — | — | — | — | — | — | — | |
| JTP (d=1)Horizon (d)=1, Parameters=1.3B, Training tokens=100B2025.11 | 58.83 | — | — | — | — | — | — | — | |
| GPTParameters=1.3B, Training tokens=100B2025.11 | 58.82 | — | — | — | — | — | — | — | |
| NextLat (d=1)Horizon (d)=1, Parameters=1.3B, Training tokens=100B2025.11 | 58.79 | — | — | — | — | — | — | — | |
| MTP (d=1)Horizon (d)=1, Parameters=1.3B, Training tokens=100B2025.11 | 58.76 | — | — | — | — | — | — | — | |
| MTP (d=2)Horizon (d)=2, Parameters=1.3B, Training tokens=100B2025.11 | 58.69 | — | — | — | — | — | — | — | |
| JTP (d=2)Horizon (d)=2, Parameters=1.3B, Training tokens=100B2025.11 | 58.63 | — | — | — | — | — | — | — | |
| PDPCMetric=PD, Order=S-shape Function, Model Scale=3B, Training Tokens=1T2025.01 | 54.9 | 69.7 | 35.8 | 95.3 | 76.3 | 35.8 | 35.6 | 36.1 | |
| RandomModel Scale=3B, Training Tokens=1T2025.01 | 50.8 | 68.6 | 33.7 | 94.6 | 76 | 27.7 | 27.5 | 27.2 |