General Knowledge Question Answering on MMLU 1000
48Pass@1Agentica-24k
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Agentica-24kTraining Regime=Base Model2025.11 | 48 | 934 | 0.76 | |
| Agentica-24k + SLPO (Direct)Training Regime=SLPO (Direct)2025.11 | 48 | 390 | 1.82 | |
| ORION-AG-SLPOTraining Regime=SLPO, Base Model=Agentica-24k2025.11 | 48 | 282 | 2.52 | |
| ORION-AG-GRPOTraining Regime=GRPO, Base Model=Agentica-24k2025.11 | 47 | 157 | 4.52 | |
| ORION-DS-GRPOTraining Regime=GRPO, Base Model=DeepSeek R1 1.5b2025.11 | 46 | 166 | 4.28 | |
| L1 MaxTraining Regime=Baseline2025.11 | 45 | 288 | 2.46 | |
| DeepSeek R1 1.5bTraining Regime=Base Model2025.11 | 45 | 710 | 1 | |
| DeepSeek R1 1.5b + SLPO (Direct)Training Regime=SLPO (Direct)2025.11 | 45 | 490 | 1.45 | |
| ORION-DS-SLPOTraining Regime=SLPO, Base Model=DeepSeek R1 1.5b2025.11 | 43 | 49 | 14.49 | |
| ORION-DSTraining Regime=Mentalese SFT, Base Model=DeepSeek R1 1.5b2025.11 | 35 | 67 | 10.6 | |
| ORION-AGTraining Regime=Mentalese SFT, Base Model=Agentica-24k2025.11 | 35 | 62 | 11.45 |