# Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization Source: [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization]() Collection index: [llms.txt]() Data updated: 2025-10-28T13:56:36.114Z Authors: Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar Published: 2016-03-21T19:51:04.000Z ## Abstract Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian optimization to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation and early-stopping. We formulate hyperparameter optimization as a pure-exploration non-stochastic infinite-armed bandit problem where a predefined resource like iterations, data samples, or features is allocated to randomly sampled configurations. We introduce a novel algorithm, Hyperband, for this framework and analyze its theoretical properties, providing several desirable guarantees. Furthermore, we compare Hyperband with popular Bayesian optimization methods on a suite of hyperparameter optimization problems. We observe that Hyperband can provide over an order-of-magnitude speedup over our competitor set on a variety of deep-learning and kernel-based learning problems. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Federated Image Classification on FEMNIST i.i.d.](): Federated Image Classification; FEMNIST (i.i.d.); Test Error Rate: 14.79. - [Image Classification on FEMNIST (i.i.d.)](): Image Classification; FEMNIST (i.i.d.); Test Error: 14.79. - [Federated Image Classification on FEMNIST non-i.i.d. (test)](): Federated Image Classification; FEMNIST non-i.i.d.; Test Error: 14.78. - [Federated Image Classification on CIFAR-10 (i.i.d.)](): Federated Image Classification; CIFAR-10 (i.i.d.); Test Error: 21.62. - [Image Classification on FEMNIST non-i.i.d. (test)](): Image Classification; FEMNIST non-i.i.d.; Test Error Rate: 14.78. - [Federated Character Prediction on Shakespeare i.i.d.](): Federated Character Prediction; Shakespeare (i.i.d.); Test Error: 46.77. - [Federated Character Prediction on Shakespeare (non-i.i.d.)](): Federated Character Prediction; Shakespeare (non-i.i.d.); Test Error: 46.79. - [Language Modeling on Shakespeare (i.i.d.)](): Language Modeling; Shakespeare (i.i.d.); Test Error: 46.77. - [Language Modeling on Shakespeare (non-i.i.d.)](): Language Modeling; Shakespeare (non-i.i.d.); Test Error: 46.79. - [Task Generation on red-wine benchmark](): task generation; red-wine benchmark; Utility: 55.24. - [Hyperparameter Optimization on JAHS-Bench CH (val)](): Hyperparameter Optimization; JAHS-Bench CH (val); Validation Error: 5.704. - [Hyperparameter Optimization on JAHS-Bench FM (val)](): Hyperparameter Optimization; JAHS-Bench FM (val); Validation Error: 5.306. - [Hyperparameter Optimization on JAHS-C10 Bench (val)](): Hyperparameter Optimization; JAHS-C10 Bench (val); Validation Error: 10.077. - [Image Classification on CIFAR-10](): Image Classification; CIFAR-10; Accuracy: not recorded. ## Original sources and implementation [Original paper]() [PDF]()