Knowledge Retention on MMLU (full)
88.3MMLU AccuracyOriginal
Evaluation results
| Method | Links | |
|---|---|---|
| OriginalModel=o32026.04 | 88.3 | |
| CAPModel=o32026.04 | 86.3 | |
| OriginalModel=Gemini-2.5-Flash2026.04 | 80.9 | |
| OriginalModel=Claude-3.7-Sonnet2026.04 | 80.7 | |
| CAPModel=Gemini-2.5-Flash2026.04 | 78.2 | |
| CAPModel=Claude-3.7-Sonnet2026.04 | 77.4 | |
| OriginalModel=Claude-3.5-Sonnet2026.04 | 77.2 | |
| CAPModel=Claude-3.5-Sonnet2026.04 | 76.4 | |
| PromptingModel=Claude-3.7-Sonnet2026.04 | 76.2 | |
| OriginalModel=Qwen-2.5-max2026.04 | 76.2 | |
| PromptingModel=Gemini-2.5-Flash2026.04 | 76 | |
| PromptingModel=Claude-3.5-Sonnet2026.04 | 75.3 | |
| CAPModel=Qwen-2.5-max2026.04 | 74.9 | |
| OriginalModel=o1-mini2026.04 | 74.2 | |
| PromptingModel=o32026.04 | 73.8 | |
| CAPModel=o1-mini2026.04 | 73.7 | |
| PromptingModel=Qwen-2.5-max2026.04 | 73.6 | |
| OriginalModel=Doubao-1.5-Lite-32k2026.04 | 73.6 | |
| PromptingModel=o1-mini2026.04 | 73.4 | |
| OriginalModel=Yi-34B-Chat2026.04 | 72.8 | |
| OriginalModel=Doubao-Lite-32k2026.04 | 72.5 | |
| PromptingModel=Doubao-1.5-Lite-32k2026.04 | 71.9 | |
| CAPModel=Doubao-1.5-Lite-32k2026.04 | 71.9 | |
| CAPModel=Doubao-Lite-32k2026.04 | 71.3 | |
| PromptingModel=Doubao-Lite-32k2026.04 | 70.8 | |
| CAPModel=Yi-34B-Chat2026.04 | 70.6 | |
| ICULModel=Yi-34B-Chat2026.04 | 69.2 | |
| OriginalModel=GPT-4o mini2026.04 | 68.8 | |
| OriginalModel=Gemini-1.5-Flash2026.04 | 68 | |
| PromptingModel=GPT-4o mini2026.04 | 66.7 | |
| CAPModel=Gemini-1.5-Flash2026.04 | 65.6 | |
| OriginalModel=GPT-4 Turbo2026.04 | 65.4 | |
| PromptingModel=Gemini-1.5-Flash2026.04 | 64.5 | |
| CAPModel=GPT-4 Turbo2026.04 | 64.1 | |
| OriginalModel=Qwen-2.5-Turbo2026.04 | 63.3 | |
| CAPModel=Qwen-2.5-Turbo2026.04 | 62.7 | |
| PromptingModel=Qwen-2.5-Turbo2026.04 | 62.6 | |
| PromptingModel=GPT-4 Turbo2026.04 | 62.5 | |
| CAPModel=GPT-4o mini2026.04 | 61.9 | |
| PromptingModel=Yi-34B-Chat2026.04 | 61 | |
| OriginalBackbone=Zephyr-7B-β2025.10 | 58 | |
| RMUBackbone=Zephyr-7B-β2025.10 | 55 | |
| BS-SBackbone=Zephyr-7B-β2025.10 | 54 | |
| BS-TBackbone=Zephyr-7B-β2025.10 | 52 | |
| WGABackbone=Zephyr-7B-β2025.10 | 48 | |
| RetainBase Model=Llama-2-7B2026.03 | 47.8 | |
| OriginalBase Model=Llama-2-7B2026.03 | 46.3 | |
| Fine-tuneBase Model=Llama-2-7B2026.03 | 46 | |
| ELMBase Model=Llama-2-7B2026.03 | 44.6 | |
| OursBase Model=Llama-2-7B2026.03 | 44.6 | |
| WHPBase Model=Llama-2-7B2026.03 | 44.4 | |
| NPOBackbone=Zephyr-7B-β2025.10 | 44 | |
| SimNPOBackbone=Zephyr-7B-β2025.10 | 44 | |
| A-LoRABase Model=Llama-2-7B2026.03 | 43.8 | |
| GDBase Model=Llama-2-7B2026.03 | 43.6 | |
| GradDiffBackbone=Zephyr-7B-β2025.10 | 43 | |
| NPOBase Model=Llama-2-7B2026.03 | 42.7 | |
| KLBase Model=Llama-2-7B2026.03 | 41.1 | |
| LoRABase Model=Llama-2-7B2026.03 | 38.9 | |
| GABase Model=Llama-2-7B2026.03 | 26.9 |