Capability Retention on AIME-2024, MATH-500, GSM8K, HumanEval, MMLU (test)
63.5AIME-2024 ScoreMEMIT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MEMITModel=Qwen3-4B-It2026.05 | 63.5 | 94.4 | 92.1 | 85.4 | 72.8 | 81.6 | |
| BaseModel=Qwen3-4B-It2026.05 | 62.9 | 94.2 | 92.6 | 86 | 77.6 | 82.7 | |
| BaseModel=Qwen3-4B-It2026.05 | 62.9 | 94.2 | 92.6 | 86 | 77.6 | 82.6 | |
| MIXSDModel=Qwen3-4B-It, lambda=0.52026.05 | 61.9 | 93.2 | 90.5 | 84.8 | 49.3 | 75.9 | |
| MIXSDModel=Qwen3-4B-It, lambda=0.32026.05 | 55.4 | 92.6 | 90.1 | 89 | 53.9 | 76.2 | |
| MIXSDModel=Qwen3-4B-It, lambda=0.32026.05 | 50.4 | 92.8 | 92.3 | 88.4 | 56 | 76 | |
| MIXSDModel=Qwen3-4B-It, lambda=0.52026.05 | 47.9 | 93 | 92.3 | 87.8 | 51.8 | 74.6 | |
| OPSDModel=Qwen3-4B-It, n=1, T=02026.05 | 46 | 82.8 | 83.3 | 81.7 | 62.2 | 71.2 | |
| MIXSDModel=Qwen3-4B-It, lambda=02026.05 | 41.7 | 87.4 | 90.4 | 85.4 | 55.3 | 72 | |
| OPSDModel=Qwen3-4B-It2026.05 | 33.8 | 73.2 | 77 | 88.4 | 67.1 | 67.9 | |
| OPSD-NLLModel=Qwen3-4B-It, n=1, T=02026.05 | 32.5 | 62.2 | 55.4 | 79.3 | 49.6 | 55.8 | |
| MIXSDModel=Qwen3-8B, lambda=0.52026.05 | 29 | 83.2 | 93.2 | 82.9 | 60.4 | 69.7 | |
| MIXSDModel=Qwen3-8B, lambda=0.52026.05 | 27.1 | 80.8 | 92.3 | 82.9 | 66.8 | 70 | |
| BaseModel=Qwen3-8B2026.05 | 26 | 83.4 | 91.8 | 86.6 | 73.1 | 72.2 | |
| BaseModel=Qwen3-8B2026.05 | 26 | 83.4 | 91.8 | 86.6 | 73.1 | 72.2 | |
| MEMITModel=Qwen3-8B2026.05 | 24.6 | 83.6 | 91.6 | 85.4 | 72.1 | 71.4 | |
| MIXSDModel=Qwen3-8B, lambda=0.32026.05 | 24.6 | 81.2 | 91.9 | 81.7 | 70.6 | 70 | |
| MIXSDModel=Qwen3-8B, lambda=0.32026.05 | 21.9 | 79.4 | 87 | 81.1 | 65 | 66.9 | |
| OPSD-NLLModel=Qwen3-8B, n=1, T=02026.05 | 20.6 | 79 | 89.4 | 86.6 | 70.2 | 69.2 | |
| MIXSDModel=Qwen3-8B, lambda=02026.05 | 20.4 | 79.8 | 91.2 | 85.4 | 69.9 | 69.3 | |
| OPSDModel=Qwen3-8B, n=1, T=02026.05 | 19 | 80.2 | 89.3 | 85.4 | 62.2 | 67.2 | |
| OPSDModel=Qwen3-8B2026.05 | 14.2 | 75 | 80.8 | 79.3 | 59.4 | 61.7 | |
| BaseModel=Qwen3-1.7B2026.05 | 11 | 72.4 | 80.4 | 60.4 | 58.5 | 56.5 | |
| BaseModel=Qwen3-1.7B2026.05 | 11 | 72.4 | 80.4 | 60.4 | 58.5 | 56.5 | |
| MEMITModel=Qwen3-1.7B2026.05 | 10.6 | 70.6 | 81.1 | 56.1 | 57.1 | 55.1 | |
| OPSD-NLLModel=Qwen3-1.7B, n=1, T=02026.05 | 1.3 | 18 | 18.3 | 18.9 | 10.8 | 13.4 | |
| MIXSDModel=Qwen3-1.7B, lambda=0.52026.05 | 1 | 36.6 | 50.3 | 42.1 | 33 | 32.6 | |
| MIXSDModel=Qwen3-1.7B, lambda=0.52026.05 | 1 | 31.2 | 45.2 | 36 | 31.4 | 29 | |
| SFTModel=Qwen3-4B-It2026.05 | 0.8 | 24.4 | 22.1 | 67.7 | 61.3 | 35.3 | |
| SFTModel=Qwen3-8B2026.05 | 0.8 | 26.6 | 24.2 | 83.5 | 62 | 39.4 | |
| MIXSDModel=Qwen3-1.7B, lambda=0.32026.05 | 0.6 | 29.8 | 37.1 | 33.5 | 20.9 | 24.4 | |
| SFTModel=Qwen3-1.7B2026.05 | 0.4 | 11.8 | 7.4 | 9.2 | 10.3 | 7.8 | |
| OPSDModel=Qwen3-1.7B2026.05 | 0.4 | 19 | 24.3 | 31.1 | 4.7 | 15.9 | |
| MIXSDModel=Qwen3-1.7B, lambda=02026.05 | 0.2 | 12.8 | 18.7 | 9.8 | 23.4 | 13 | |
| SFTModel=Qwen3-4B-It2026.05 | 0.2 | 19.2 | 15 | 50 | 48.8 | 26.6 | |
| SFTModel=Qwen3-8B2026.05 | 0.2 | 18.8 | 14.1 | 67.1 | 52 | 30.4 | |
| MIXSDModel=Qwen3-1.7B, lambda=0.32026.05 | 0 | 18.6 | 22.6 | 21.3 | 25.4 | 17.6 | |
| SFTModel=Qwen3-1.7B2026.05 | 0 | 6.8 | 3.4 | 3 | 30.2 | 8.7 | |
| OPSDModel=Qwen3-1.7B, n=1, T=02026.05 | 0 | 6.4 | 7.2 | 8.5 | 6.9 | 5.8 |