Continual Learning on Math → Science → Code (Nemotron/KodCode)
59.2Math Test AccuracyMoLEM
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MoLEMTraining Stage=Stage 1 Math, Base Model=Qwen3-4B-Instruct-25072026.05 | 59.2 | 88.2 | 72 | 73.13 | — | — | — | |
| MoLEMTraining Stage=Stage 2 Science, Base Model=Qwen3-4B-Instruct-25072026.05 | 59.2 | 88.4 | 72 | 73.2 | 0 | 0 | 0 | |
| MoLEMTraining Stage=Stage 3 Code, Base Model=Qwen3-4B-Instruct-25072026.05 | 59.2 | 88.4 | 74.2 | 73.93 | 0 | 0 | 0 | |
| MemGenTraining Stage=Stage 1 Math, Base Model=Qwen3-4B-Instruct-25072026.05 | 58.2 | 87 | 40 | 61.73 | — | — | — | |
| SFTTraining Stage=Stage 1 Math, Base Model=Qwen3-4B-Instruct-25072026.05 | 57.2 | 84.2 | 32.8 | 58.07 | — | — | — | |
| MemGenTraining Stage=Stage 2 Science, Base Model=Qwen3-4B-Instruct-25072026.05 | 33.4 | 89 | 16 | 46.13 | 24.8 | -24.8 | -1.2 | |
| ExpeLTraining Stage=Stage 3 Code, Base Model=Qwen3-4B-Instruct-25072026.05 | 32 | 91.8 | 60.6 | 61.47 | -0.8 | 2.5 | -4.1 | |
| MemGenTraining Stage=Stage 3 Code, Base Model=Qwen3-4B-Instruct-25072026.05 | 32 | 84.8 | 73.6 | 63.47 | 15.2 | -15.2 | -28.6 | |
| ExpeLTraining Stage=Stage 2 Science, Base Model=Qwen3-4B-Instruct-25072026.05 | 31.4 | 90.8 | 65 | 62.4 | -3.4 | 3.4 | -1.2 | |
| VanillaTraining Stage=None, Base Model=Qwen3-4B-Instruct-25072026.05 | 30.4 | 88.2 | 72 | 63.53 | — | — | — | |
| SFTTraining Stage=Stage 2 Science, Base Model=Qwen3-4B-Instruct-25072026.05 | 29.6 | 86 | 15 | 43.53 | 27.6 | -27.6 | -4 | |
| ExpeLTraining Stage=Stage 1 Math, Base Model=Qwen3-4B-Instruct-25072026.05 | 28 | 87 | 64.2 | 59.73 | — | — | — | |
| SFTTraining Stage=Stage 3 Code, Base Model=Qwen3-4B-Instruct-25072026.05 | 12.8 | 27.2 | 63.2 | 34.4 | 51.6 | -51.6 | -30.5 |