Model Merging Evaluation on MergeBench
52.6MATH-500 ScoreFine-tuned (STL)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Fine-tuned (STL)Model=Llama-3.1-8B-Instruct2026.05 | 52.6 | 85.3 | 67.1 | 63.8 | 72.8 | 80.9 | 70.4 | |
| Task Arithmetic w/ ProbSurgeryModel=Llama-3.1-8B-Instruct, Task Arithmetic Scale=0.32026.05 | 49.6 | 85.3 | 62.8 | 61.9 | 48.8 | 77.8 | 64.4 | |
| Fine-tuned (STL)Model=Llama-3.2-3B-Instruct2026.05 | 49 | 80.6 | 54.9 | 57.1 | 69.7 | 73.1 | 64.1 | |
| Task ArithmeticModel=Llama-3.1-8B-Instruct, Task Arithmetic Scale=0.32026.05 | 49 | 85.6 | 62.8 | 61.6 | 45.3 | 76.9 | 63.5 | |
| BaseModel=Llama-3.1-8B-Instruct2026.05 | 48.6 | 83.5 | 62.2 | 63 | 72.8 | 80.9 | 68.5 | |
| BaseModel=Llama-3.2-3B-Instruct2026.05 | 47.8 | 72.6 | 49.4 | 56.6 | 67.3 | 73.1 | 61.1 | |
| Task Arithmetic w/ FEATCALModel=Llama-3.1-8B-Instruct, Task Arithmetic Scale=0.32026.05 | 47.6 | 84.5 | 59.8 | 62.7 | 60.8 | 79.5 | 65.8 | |
| Task Arithmetic w/ SurgeryModel=Llama-3.1-8B-Instruct, Task Arithmetic Scale=0.32026.05 | 47.2 | 85.2 | 64.6 | 62.2 | 47.1 | 77.6 | 64 | |
| Task Arithmetic w/ FEATCALModel=Llama-3.2-3B-Instruct, Task Arithmetic Scale=0.32026.05 | 47 | 78 | 53 | 55.3 | 66.5 | 72.5 | 62.1 | |
| Task Arithmetic w/ ProbSurgeryModel=Llama-3.2-3B-Instruct, Task Arithmetic Scale=0.32026.05 | 46 | 74.8 | 54.3 | 54.5 | 64.1 | 73 | 61.1 | |
| Task Arithmetic w/ SurgeryModel=Llama-3.2-3B-Instruct, Task Arithmetic Scale=0.32026.05 | 45.8 | 74.5 | 53 | 54.5 | 64.3 | 72.9 | 60.8 | |
| Task ArithmeticModel=Llama-3.2-3B-Instruct, Task Arithmetic Scale=0.32026.05 | 44 | 74.8 | 52.4 | 53.4 | 63.2 | 72.6 | 60.1 |