Multi-task model alignment and mixing on Math, Chat, IF, and General QA tasks Llama-3.1-8B (test)
36Math AccuracyMod. Surgery
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Mod. SurgeryBackbone=Llama-3.1-8B, Alias=Modular Gradient Surgery (MGS)2026.02 | 36 | 30.5 | 30 | 33.1 | 32.6 | |
| Global SurgeryBackbone=Llama-3.1-8B, Alias=GGS2026.02 | 35.8 | 24.2 | 31.1 | 30.3 | 30.3 | |
| Naïve MixingBackbone=Llama-3.1-8B2026.02 | 35 | 22.9 | 25.4 | 25.2 | 27.3 |