Generative Language Model Merging on MergeBench Llama-3.2-3B (test) benchmark leaderboard · SOTA2 Research

Generative Language Model Merging on MergeBench Llama-3.2-3B (test)

53.52Instruction ScoreFine-tuned

Evaluation Results

Method	Links
Fine-tuned2026.06		53.52	60.27	44.62	41.64	42.23	48.46
ESM++2026.06		53.06	58.23	41.07	41.79	42.9	47.41
ESM2026.06		42.45	52.08	39.35	41.03	45.53	44.09
DARE2026.06		35.3	50.8	40.62	42.22	40.16	41.82
Consensus TA2026.06		33.12	48.07	42.05	42.39	36.87	40.5
Task Arithmetic2026.06		30.91	46.02	41.17	42.35	40.16	40.12
L&S2026.06		29.11	44.81	33.97	42.16	24.28	34.87
TSV-M2026.06		26.63	54.28	40.79	42.13	38.22	40.41
TIES-Merging2026.06		20.66	48.07	39.05	42.34	33.73	36.77
Model Soup2026.06		13.71	41.93	37.22	42.25	31.21	33.26
Pre-trained2026.06		10.45	30.25	27.17	40.73	19.87	25.69