Discovering What You Can Control: Interventional Boundary Discovery for Reinforcement Learning

About

When an RL agent's observations contain distractors driven by the same confounders as its true state, observational data alone cannot identify which dimensions the agent controls. In our benchmarks, even state-conditioned observational selectors can collapse when distractors mimic controllable state variables. We propose Interventional Boundary Discovery (IBD), which treats the agent's own action channel as a source of randomized interventions: randomizing actions implements an interventional contrast, and per-dimension two-sample tests with FDR correction produce a binary mask over observation dimensions. Across 12 continuous-control settings with up to 100 distractors, IBD matches oracle return in 11 of 12 settings, while observational baselines including mutual information, state-conditioned forward models, and gradient-based sensitivity often underperform simply passing the full observation to SAC.

Jiaxin Liu, Anzhe Cheng, Paul Bogdan · 2026

Related benchmarks

Task	Dataset	Result
Reinforcement Learning	DeepMind Control Suite walker_walk, hard	Episode Return842	6
Reinforcement Learning	DeepMind Control Suite cheetah_run medium	Episode Return479	6
Reinforcement Learning	DeepMind Control Suite reacher_hard, easy	Episode Return939	6
Reinforcement Learning	DeepMind Control Suite reacher_hard medium	Episode Return929	6
Reinforcement Learning	DeepMind Control Suite cartpole_swingup medium	Episode Return834	6
Reinforcement Learning	DeepMind Control Suite hopper_hop medium	Episode Return6	6
Reinforcement Learning	DeepMind Control Suite walker_walk medium	Episode Return792	6
Reinforcement Learning	DeepMind Control Suite cheetah_run hard	Episode Return436	6
Reinforcement Learning	DeepMind Control Suite reacher_hard	Episode Return907	6
Reinforcement Learning	DeepMind Control Suite finger_spin medium	Episode Return587	6