# High Dimensional Bayesian Optimization Using Dropout

Source: [High Dimensional Bayesian Optimization Using Dropout](<https://www.sota2.com/research/paper/68fa7d851e209131cea9370a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:57:26.592Z

Authors: Cheng Li, Sunil Gupta, Santu Rana, Vu Nguyen, Svetha Venkatesh, Alistair Shilton

Published: 2018-02-15T04:30:59.000Z

## Abstract

Scaling Bayesian optimization to high dimensions is challenging task as the global optimization of high-dimensional acquisition function can be expensive and often infeasible. Existing methods depend either on limited active variables or the additive form of the objective function. We propose a new method for high-dimensional Bayesian optimization, that uses a dropout strategy to optimize only a subset of variables at each iteration. We derive theoretical bounds for the regret and show how it can inform the derivation of our algorithm. We demonstrate the efficacy of our algorithms for optimization on two benchmark functions and two real-world applications- training cascade classifiers and optimizing alloy composition.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Black-box Optimization on Hartmann-6D 300 evaluations](<https://www.sota2.com/research/sota/black-box-optimization-on-hartmann-6d-300-evaluations.md>): Black-box Optimization; Hartmann-6D 300 evaluations; Wall Clock Time (s): 3.237.

- [Black-box Optimization on Hartmann-6D 500 evaluations](<https://www.sota2.com/research/sota/black-box-optimization-on-hartmann-6d-500-evaluations.md>): Black-box Optimization; Hartmann-6D 500 evaluations; Wall Clock Time (s): 3.685.

- [Black-box Optimization on Levy-10D 300 evaluations](<https://www.sota2.com/research/sota/black-box-optimization-on-levy-10d-300-evaluations.md>): Black-box Optimization; Levy-10D 300 evaluations; Wall Clock Time (s): 3.225.

- [Black-box Optimization on Levy-10D 100 evaluations](<https://www.sota2.com/research/sota/black-box-optimization-on-levy-10d-100-evaluations.md>): Black-box Optimization; Levy-10D 100 evaluations; Wall Clock Time (s): 2.707.

- [Mathematical Reasoning on MATH (Accuracy)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-accuracy.md>): Mathematical Reasoning; MATH; Accuracy: 11.88.

- [Code Generation on HumanEval](<https://www.sota2.com/research/sota/code-generation-on-humaneval.md>): Code Generation; HumanEval; Pass@1: 26.83.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1802.05400>)

[PDF](<https://arxiv.org/pdf/1802.05400.pdf>)