# SafeAdapt: Provably Safe Policy Updates in Deep Reinforcement Learning Source: [SafeAdapt: Provably Safe Policy Updates in Deep Reinforcement Learning]() Collection index: [llms.txt]() Data updated: 2026-04-14T05:00:11.147Z Authors: Maksim Anisimov, Francesco Belardinelli, Matthew Wicker Published: 2026-04-10T16:09:39.000Z ## Abstract Safety guarantees are a prerequisite to the deployment of reinforcement learning (RL) agents in safety-critical tasks. Often, deployment environments exhibit non-stationary dynamics or are subject to changing performance goals, requiring updates to the learned policy. This leads to a fundamental challenge: how to update an RL policy while preserving its safety properties on previously encountered tasks? The majority of current approaches either do not provide formal guarantees or verify policy safety only a posteriori. We propose a novel a priori approach to safe policy updates in continual RL by introducing the Rashomon set: a region in policy parameter space certified to meet safety constraints within the demonstration data distribution. We then show that one can provide formal, provable guarantees for arbitrary RL algorithms used to update a policy by projecting their updates onto the Rashomon set. Empirically, we validate this approach across grid-world navigation environments (Frozen Lake and Poisoned Apple) where we guarantee an a priori provably deterministic safety on the source task during downstream adaptation. In contrast, we observe that regularisation-based baselines experience catastrophic forgetting of safety constraints while our approach enables strong adaptation with provable guarantees that safety is preserved. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Downstream Task on Poisoned Apple simple\_5x5](): Downstream Task; Poisoned Apple simple\_5x5; Total Reward: 0.96. - [Source Task Performance on Frozen Lake standard\_4x4](): Source Task Performance; Frozen Lake standard\_4x4; Critical State Safety Rate: 100. - [Source Task Performance on Poisoned Apple simple\_5x5](): Source Task Performance; Poisoned Apple simple\_5x5; Critical State Safety Rate: 100. - [Downstream Task on Frozen Lake standard\_4x4](): Downstream Task; Frozen Lake standard\_4x4; Total Reward: 1. ## Original sources and implementation [Original paper]() [PDF]()