# Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance

Source: [Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance](<https://www.sota2.com/research/paper/68fa74341e209131cea606c3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:54:51.804Z

Authors: Haiquan Lu, Xiaotian Liu, Yefan Zhou, Qunli Li, Kurt Keutzer, Michael W. Mahoney, Yujun Yan, Huanrui Yang, Yaoqing Yang

Published: 2024-07-17T20:31:26.000Z

## Abstract

Recent studies on deep ensembles have identified the sharpness of the local minima of individual learners and the diversity of the ensemble members as key factors in improving test-time performance. Building on this, our study investigates the interplay between sharpness and diversity within deep ensembles, illustrating their crucial role in robust generalization to both in-distribution (ID) and out-of-distribution (OOD) data. We discover a trade-off between sharpness and diversity: minimizing the sharpness in the loss landscape tends to diminish the diversity of individual members within the ensemble, adversely affecting the ensemble's improvement. The trade-off is justified through our theoretical analysis and verified empirically through extensive experiments. To address the issue of reduced diversity, we introduce SharpBalance, a novel training approach that balances sharpness and diversity within ensembles. Theoretically, we show that our training strategy achieves a better sharpness-diversity trade-off. Empirically, we conducted comprehensive evaluations in various data sets (CIFAR-10, CIFAR-100, TinyImageNet) and showed that SharpBalance not only effectively improves the sharpness-diversity trade-off, but also significantly improves ensemble performance in ID and OOD scenarios.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Classification on Tiny-ImageNet-C v1 (test)](<https://www.sota2.com/research/sota/object-classification-on-tiny-imagenet-c-v1-test.md>): Object Classification; Tiny-ImageNet-C v1 (test); Accuracy (Severity 1): 46.48.

- [Image Classification on CIFAR100-C (test)](<https://www.sota2.com/research/sota/image-classification-on-cifar100-c-test.md>): Image Classification; CIFAR100-C (test); Robustness Accuracy: 52.46.

- [Image Classification on CIFAR10 (test) (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-cifar10-test-accuracy.md>): Image Classification; CIFAR10 (test); Accuracy: 96.18.

- [Image Classification on CIFAR-100-C](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-c.md>): Image Classification; CIFAR-100-C; Accuracy (Corruption): not recorded.

- [Image Classification on CIFAR-10-C (test)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-c-test.md>): Image Classification; CIFAR-10-C (test); Accuracy (Clean): not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2407.12996>)

[PDF](<https://arxiv.org/pdf/2407.12996.pdf>)