Code Generation and Understanding on Code Crux, MultiPL_E, MBPP
60.12Crux ScoreGatedNorm
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GatedNormArchitecture=24B-A3B, Precision=BF16, Training Tokens=500B2026.01 | 60.12 | 47.52 | 58.4 | |
| PreAffineArchitecture=24B-A3B, Precision=BF16, Training Tokens=500B2026.01 | 59.13 | 46.9 | 54.8 | |
| GatedNormArchitecture=24B-A3B, Precision=W4A4, Training Tokens=500B2026.01 | 59.13 | 46.97 | 58.8 | |
| GatedNormArchitecture=24B-A3B, Precision=W4A4+SQ, Training Tokens=500B2026.01 | 58.06 | 47.18 | 58.8 | |
| Gated AttentionArchitecture=24B-A3B, Precision=BF16, Training Tokens=500B2026.01 | 58 | 45.88 | 56.8 | |
| PreAffineArchitecture=24B-A3B, Precision=W4A4+SQ, Training Tokens=500B2026.01 | 57.56 | 42.76 | 55 | |
| PreAffineArchitecture=24B-A3B, Precision=W4A4, Training Tokens=500B2026.01 | 56.69 | 44.01 | 53 | |
| Gated AttentionArchitecture=24B-A3B, Precision=W4A4, Training Tokens=500B2026.01 | 56.5 | 44.41 | 56.6 | |
| Gated AttentionArchitecture=24B-A3B, Precision=W4A4+SQ, Training Tokens=500B2026.01 | 55.19 | 46.83 | 56.6 | |
| PreAffineArchitecture=MoE-7B-A2B, Precision=BF16, Training Tokens=1.2T2026.01 | 46.75 | 37.99 | 49.2 | |
| GatedNormArchitecture=MoE-7B-A2B, Precision=BF16, Training Tokens=1.2T2026.01 | 45.56 | 34.81 | 51 | |
| Gated AttentionArchitecture=MoE-7B-A2B, Precision=BF16, Training Tokens=1.2T2026.01 | 44.38 | 34.63 | 50.04 |