# Efficient Exploration in Continuous-time Model-based Reinforcement Learning Source: [Efficient Exploration in Continuous-time Model-based Reinforcement Learning]() Collection index: [llms.txt]() Data updated: 2026-01-01T12:39:31.582Z Authors: Lenart Treven, Jonas H\\"ubotter, Bhavya Sukhija, Florian D\\"orfler, Andreas Krause Published: 2023-10-30T15:04:40.000Z ## Abstract Reinforcement learning algorithms typically consider discrete-time dynamics, even though the underlying systems are often continuous in time. In this paper, we introduce a model-based reinforcement learning algorithm that represents continuous-time dynamics using nonlinear ordinary differential equations (ODEs). We capture epistemic uncertainty using well-calibrated probabilistic models, and use the optimistic principle for exploration. Our regret bounds surface the importance of the measurement selection strategy(MSS), since in continuous time we not only must decide how to explore, but also when to observe the underlying system. Our analysis demonstrates that the regret is sublinear when modeling ODEs with Gaussian Processes (GP) for common choices of MSS, such as equidistant sampling. Additionally, we propose an adaptive, data-dependent, practical MSS that, when combined with GP dynamics, also achieves sublinear regret with significantly fewer samples. We showcase the benefits of continuous-time modeling over its discrete-time counterpart, as well as our proposed adaptive MSS over standard baselines, on several applications. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Optimal Control on Quadrotor 3D](): Optimal Control; Quadrotor 3D; Final Cost: not recorded. - [Optimal Control on Pendulum](): Optimal Control; Pendulum; Final Cost: not recorded. - [Optimal Control on Cancer Treatment](): Optimal Control; Cancer Treatment; Final Cost: not recorded. - [Optimal Control on Glucose in Blood](): Optimal Control; Glucose in Blood; Final Cost: not recorded. - [Optimal Control on Mountain Car](): Optimal Control; Mountain Car; Final Cost: not recorded. - [Optimal Control on Cart Pole](): Optimal Control; Cart Pole; Final Cost: not recorded. - [Optimal Control on Bicycle](): Optimal Control; Bicycle; Final Cost: not recorded. - [Optimal Control on Quadrotor 2D](): Optimal Control; Quadrotor 2D; Final Cost: not recorded. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()