Model Merging on Multi-task Category-level Evaluation Suite
93.5Average ScoreMETIS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| METISMerging Paradigm=Many-Shot, Backbone=Llama-3.1-8B2026.06 | 93.5 | 58.5 | 1.136 | 0.972 | 0.977 | |
| TIESMerging Paradigm=Many-Shot, Backbone=Llama-3.1-8B2026.06 | 90.2 | 39 | 1.017 | 1.002 | 1 | |
| ConsensusTAMerging Paradigm=Many-Shot, Backbone=Llama-3.1-8B2026.06 | 89.8 | 51.2 | 1.085 | 0.943 | 0.965 | |
| TIESMerging Paradigm=Post-Hoc, Backbone=Llama-3.1-8B2026.06 | 85.2 | 39 | 1.017 | 1.002 | 0.698 | |
| DAREMerging Paradigm=Many-Shot, Backbone=Llama-3.1-8B2026.06 | 84.9 | 43.9 | 0.966 | 0.965 | 0.791 | |
| DAREMerging Paradigm=Post-Hoc, Backbone=Llama-3.1-8B2026.06 | 83.8 | 63.4 | 0.695 | 0.885 | 1.047 | |
| Task ArithmeticMerging Paradigm=Many-Shot, Backbone=Llama-3.1-8B2026.06 | 78.5 | 24.4 | 0.966 | 0.996 | 0.512 | |
| Task ArithmeticMerging Paradigm=Post-Hoc, Backbone=Llama-3.1-8B2026.06 | 70.4 | 39 | 0.525 | 0.796 | 0.919 | |
| ConsensusTAMerging Paradigm=Post-Hoc, Backbone=Llama-3.1-8B2026.06 | 69.4 | 29.3 | 0.458 | 0.812 | 0.977 |