# Navigating the Accuracy-Size Trade-Off with Flexible Model Merging Source: [Navigating the Accuracy-Size Trade-Off with Flexible Model Merging]() Collection index: [llms.txt]() Data updated: 2026-04-16T05:00:11.996Z Authors: Akash Dhasade, Divyansh Jhunjhunwala, Milos Vujasinovic, Gauri Joshi, Anne-Marie Kermarrec Published: 2025-05-29T07:50:32.000Z ## Abstract Model merging has emerged as an efficient method to combine multiple single-task fine-tuned models. The merged model can enjoy multi-task capabilities without expensive training. While promising, merging into a single model often suffers from an accuracy gap with respect to the fine-tuned models. On the other hand, deploying all individual fine-tuned models incurs high storage costs. We propose FlexMerge, a novel data-free model merging framework that: (a) flexibly generates merged models of varying sizes, spanning the full spectrum from a single merged model to retaining all fine-tuned models; and (b) supports multiple merging algorithms in a unified framework. Using FlexMerge, we systematically characterize the accuracy-size trade-off of different algorithms. Our study reveals two key findings: first, even modestly larger merged models can yield steep accuracy gains (up to 13.5% when just doubling the size); second, algorithm rankings are not consistent as size increases, with some methods overtaking others beyond the one-model regime. These results uncover a new design dimension for model merging: developing and comparing algorithms across the full spectrum of sizes rather than only at the single-model limit. Extensive experiments on vision and NLP benchmarks, with up to 30 tasks, confirm the generality and practicality of FlexMerge. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on OOD Benchmark Suite Subset (Cars, DTD, EuroSAT, GTSRB, MNIST, RESISC45, SVHN)](): Image Classification; OOD benchmark suite (SVHN, Cars, RESISC45, EuroSAT, SUN397, GTSRB, MNIST, DTD); Average OOD Accuracy: 74.88. ## Original sources and implementation [Original paper]() [PDF]()