# Bilinear Classes: A Structural Framework for Provable Generalization in RL

Source: [Bilinear Classes: A Structural Framework for Provable Generalization in RL](<https://www.sota2.com/research/paper/68fa584b1e209131ce9acfde>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T15:14:33.507Z

Authors: Simon S. Du, Sham M. Kakade, Jason D. Lee, Shachar Lovett, Gaurav Mahajan, Wen Sun, Ruosong Wang

Published: 2021-03-19T16:34:20.000Z

## Abstract

This work introduces Bilinear Classes, a new structural framework, which permit generalization in reinforcement learning in a wide variety of settings through the use of function approximation. The framework incorporates nearly all existing models in which a polynomial sample complexity is achievable, and, notably, also includes new models, such as the Linear $Q^\*/V^\*$ model in which both the optimal $Q$-function and the optimal $V$-function are linear in some known feature space. Our main result provides an RL algorithm which has polynomial sample complexity for Bilinear Classes; notably, this sample complexity is stated in terms of a reduction to the generalization error of an underlying supervised learning sub-problem. These bounds nearly match the best known sample complexity bounds for existing models. Furthermore, this framework also extends to the infinite dimensional (RKHS) setting: for the the Linear $Q^\*/V^\*$ model, linear MDPs, and linear mixture MDPs, we provide sample complexities that have no explicit dependence on the explicit feature dimension (which could be infinite), but instead depends only on information theoretic quantities.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Model-free learning on Stochastic MDPs Bilinear or BE](<https://www.sota2.com/research/sota/model-free-learning-on-stochastic-mdps-bilinear-or-be.md>): Model-free learning; Stochastic MDPs Bilinear or BE; Regret T Exponent: 2.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2103.10897>)

[PDF](<https://arxiv.org/pdf/2103.10897.pdf>)
