# RAMP: Hybrid DRL for Online Learning of Numeric Action Models

Source: [RAMP: Hybrid DRL for Online Learning of Numeric Action Models](<https://www.sota2.com/research/paper/69dc788f7d0172140ec78b02>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-14T05:00:04.234Z

Authors: Yarin Benyamin, Argaman Mordoch, Shahaf S. Shperberg, Roni Stern

Published: 2026-04-09T18:16:19.000Z

## Abstract

Automated planning algorithms require an action model specifying the preconditions and effects of each action, but obtaining such a model is often hard. Learning action models from observations is feasible, but existing algorithms for numeric domains are offline, requiring expert traces as input. We propose the Reinforcement learning, Action Model learning, and Planning (RAMP) strategy for learning numeric planning action models online via interactions with the environment. RAMP simultaneously trains a Deep Reinforcement Learning (DRL) policy, learns a numeric action model from past interactions, and uses that model to plan future actions when possible. These components form a positive feedback loop: the RL policy gathers data to refine the action model, while the planner generates plans to continue training the RL policy. To facilitate this integration of RL and numeric planning, we developed Numeric PDDLGym, an automated framework for converting numeric planning problems to Gym environments. Experimental results on standard IPC numeric domains show that RAMP significantly outperforms PPO, a well-known DRL algorithm, in terms of solvability and plan quality.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Action Model Learning on Action Model Learning Problem Properties](<https://www.sota2.com/research/sota/action-model-learning-on-action-model-learning-problem-properties.md>): Action Model Learning; Action Model Learning Problem Properties; Numeric Determinism: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.08685>)

[PDF](<https://arxiv.org/pdf/2604.08685.pdf>)
