Question Answering on ARC (ARCe, ARCc)
63.4ARCe AccuracyMHA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MHAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/12025.10 | 63.4 | 37.2 | |
| MLAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/42025.10 | 63.3 | 35.4 | |
| CCGQAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/42025.10 | 62.6 | 36 | |
| CCAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/42025.10 | 62.4 | 34.7 | |
| GQAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/42025.10 | 62.3 | 34.7 | |
| Expert Divergence LearningModel Size=15B-A1.5B, Scheme=3-class2026.02 | 59.08 | 33.56 | |
| Expert Divergence LearningModel Size=8B-A0.8B, Scheme=49-class2026.02 | 58.55 | 30.85 | |
| Expert Divergence LearningModel Size=15B-A1.5B, Scheme=49-class2026.02 | 57.85 | 33.56 | |
| Expert Divergence LearningModel Size=8B-A0.8B, Scheme=3-class2026.02 | 56.79 | 32.2 | |
| MoEModel Size=15B-A1.5B, Training Strategy=Baseline2026.02 | 56.61 | 29.83 | |
| Expert Divergence LearningModel Size=3B-A0.3B, Scheme=49-class2026.02 | 54.5 | 31.53 | |
| MoEModel Size=8B-A0.8B, Training Strategy=Baseline2026.02 | 53.62 | 31.53 | |
| Expert Divergence LearningModel Size=3B-A0.3B, Scheme=3-class2026.02 | 51.15 | 30.85 | |
| MoEModel Size=3B-A0.3B, Training Strategy=Baseline2026.02 | 50.79 | 31.19 |