Commonsense Reasoning on ARC
93AccuracyOurs
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OursTraining=MATH2025.10 | 93 | — | — | |
| diffu-GRPOTraining=MATH2025.10 | 91.8 | — | — | |
| SFT Model2025.09 | 91.29 | — | — | |
| DPO2025.09 | 91.26 | — | — | |
| HT-MNPOReward Model=ArmoRM-Llama32025.09 | 91.23 | — | — | |
| HT-MNPOReward Model=Athene-RM-8B2025.09 | 91.18 | — | — | |
| SPPO2025.09 | 91.17 | — | — | |
| TD-MNPO2025.09 | 91.15 | — | — | |
| HT-MNPOReward Model=Skywork-Reward-V22025.09 | 91.12 | — | — | |
| SimPO2025.09 | 91.09 | — | — | |
| INPO2025.09 | 91.07 | — | — | |
| diffu-GRPOTraining=SUDOKU2025.10 | 91 | — | — | |
| diffu-GRPOTraining=COUNTDOWN2025.10 | 90.6 | — | — | |
| OursTraining=SUDOKU2025.10 | 90.6 | — | — | |
| LLaDATraining=-2025.10 | 90.2 | — | — | |
| OursTraining=GSM8K2025.10 | 90.2 | — | — | |
| diffu-GRPOTraining=GSM8K2025.10 | 89.8 | — | — | |
| Claude 3.5 Haiku (target)Model=Claude 3.5 Haiku2026.04 | 89.2 | — | — | |
| CoFiCot2026.03 | 88.2 | — | — | |
| OursTraining=COUNTDOWN2025.10 | 87.5 | — | — | |
| 120-way SCSampling strategy=120-way Self-Consistency2026.03 | 86 | — | — | |
| 40-way SCSampling strategy=40-way Self-Consistency2026.03 | 85.5 | — | — | |
| Zero-shotMode=Zero-shot2026.03 | 66.5 | — | — | |
| Llama 3.1 8B (baseline)Model=Llama 3.1 8B2026.04 | 56.66 | — | — | |
| Exp B: L*+LrOptimization Strategy=L*+Lr2026.04 | 56.23 | — | — | |
| GARFA (Exp A)Optimization Strategy=L*+L*2026.04 | 56.14 | — | — | |
| Exp C: Lr+L*Optimization Strategy=Lr+L*2026.04 | 53.16 | — | — | |
| Exp D: Lr+Lr (control)Optimization Strategy=Lr+Lr2026.04 | 52.65 | — | — | |
| NAMEX - Full (0 disagreement point)Routing Strategy=Stable-MoE, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.45 | — | — | |
| NAMEx-FullRouting Strategy=Stable-MoE Routing, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 51.45 | — | — | |
| NAMEX - Full (mean disagreement point)Routing Strategy=Cosine, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.37 | — | — | |
| NAMEX - Full (0 disagreement point)Routing Strategy=Linear, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.35 | — | — | |
| NAMEX - Full (mean disagreement point)Routing Strategy=Stable-MoE, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.35 | — | — | |
| NAMEx-FullRouting Strategy=Linear, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 51.35 | — | — | |
| NAMEX - Full (0 disagreement point)Routing Strategy=Cosine, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.3 | — | — | |
| EP - CAMEXRouting Strategy=Stable-MoE, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.3 | — | — | |
| NAMEx-FullRouting Strategy=Cosine, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 51.3 | — | — | |
| EP-CAMExRouting Strategy=Stable-MoE Routing, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 51.3 | — | — | |
| NAMEX - Full (mean disagreement point)Routing Strategy=Linear, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.25 | — | — | |
| EP - CAMEXRouting Strategy=Linear, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.23 | — | — | |
| NAMEX - Full (mean disagreement point)Routing Strategy=Linear, Evaluation Protocol=Zero-Shot2025.10 | 51.23 | — | — | |
| EP-CAMExRouting Strategy=Linear, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 51.23 | — | — | |
| Qwen1.5 - MoERouting Strategy=Stable-MoE, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.2 | — | — | |
| Qwen1.5-MoERouting Strategy=Stable-MoE Routing, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 51.2 | — | — | |
| Qwen1.5 - MoERouting Strategy=Linear, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.12 | — | — | |
| Qwen1.5-MoERouting Strategy=Linear, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 51.12 | — | — | |
| EP - CAMEXRouting Strategy=Cosine, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 51.1 | — | — | |
| EP-CAMExRouting Strategy=Cosine, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 51.1 | — | — | |
| NAMEX - Full (mean disagreement point)Routing Strategy=Stable-MoE, Evaluation Protocol=Zero-Shot2025.10 | 51.03 | — | — | |
| NAMEX - Full (0 disagreement point)Routing Strategy=Stable-MoE, Evaluation Protocol=Zero-Shot2025.10 | 50.96 | — | — | |
| NAMEX - Full (0 disagreement point)Routing Strategy=Linear, Evaluation Protocol=Zero-Shot2025.10 | 50.95 | — | — | |
| Qwen1.5 - MoERouting Strategy=Cosine, Evaluation Protocol=Fine-tuned (SmolTalk)2025.10 | 50.95 | — | — | |
| Qwen1.5-MoERouting Strategy=Cosine, Fine-tuned on=SmolTalk, Model Size=14B2025.10 | 50.95 | — | — | |
| NAMEX - Full (0 disagreement point)Routing Strategy=Cosine, Evaluation Protocol=Zero-Shot2025.10 | 50.93 | — | — | |
| EP - CAMEXRouting Strategy=Stable-MoE, Evaluation Protocol=Zero-Shot2025.10 | 50.89 | — | — | |
| EP - CAMEXRouting Strategy=Linear, Evaluation Protocol=Zero-Shot2025.10 | 50.83 | — | — | |
| Qwen1.5 - MoERouting Strategy=Stable-MoE, Evaluation Protocol=Zero-Shot2025.10 | 50.81 | — | — | |
| Qwen1.5 - MoERouting Strategy=Linear, Evaluation Protocol=Zero-Shot2025.10 | 50.77 | — | — | |
| NAMEX - Full (mean disagreement point)Routing Strategy=Cosine, Evaluation Protocol=Zero-Shot2025.10 | 50.77 | — | — | |
| EP - CAMEXRouting Strategy=Cosine, Evaluation Protocol=Zero-Shot2025.10 | 50.68 | — | — | |
| Qwen1.5 - MoERouting Strategy=Cosine, Evaluation Protocol=Zero-Shot2025.10 | 50.6 | — | — | |
| Full AttentionModel Group=Baselines, Parameter Count=4B2026.04 | — | 64.52 | 33.19 | |
| In-Place TTTArchitecture=Full Attention, Parameter Count=4B2026.04 | — | 64.98 | 32.34 | |
| In-Place TTTArchitecture=Sliding-Window Attention (SWA), Parameter Count=4B2026.04 | — | 64.6 | 33.7 | |
| Sliding-Window Attention (SWA)Model Group=Baselines, Parameter Count=4B2026.04 | — | 64.18 | 32.85 |