# Learning Theory for Kernel Bilevel Optimization Source: [Learning Theory for Kernel Bilevel Optimization]() Collection index: [llms.txt]() Data updated: 2025-11-19T05:02:23.880Z Authors: Fares El Khoury, Edouard Pauwels, Samuel Vaiter, Michael Arbel Published: 2025-02-12T14:52:04.000Z ## Abstract Bilevel optimization has emerged as a technique for addressing a wide range of machine learning problems that involve an outer objective implicitly determined by the minimizer of an inner problem. While prior works have primarily focused on the parametric setting, a learning-theoretic foundation for bilevel optimization in the nonparametric case remains relatively unexplored. In this paper, we take a first step toward bridging this gap by studying Kernel Bilevel Optimization (KBO), where the inner objective is optimized over a reproducing kernel Hilbert space. This setting enables rich function approximation while providing a foundation for rigorous theoretical analysis. In this context, we derive novel finite-sample generalization bounds for KBO, leveraging tools from empirical process theory. These bounds further allow us to assess the statistical accuracy of gradient-based methods applied to the empirical discretization of KBO. We numerically illustrate our theoretical findings on a synthetic instrumental variable regression task. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Root-estimation on Fitted Q-regression Experiment 1 (scalar root experiment) (test)](): Root Estimation; Fitted Q-regression Experiment 1 (scalar root experiment) (test); RMSE: 0.0396. ## Original sources and implementation [Original paper]() [PDF]()