# The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping

Source: [The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping](<https://www.sota2.com/research/paper/69ddca4a7d0172140ec79cbf>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-17T06:00:01.785Z

Authors: Yang Liu, Enxi Wang, Yufei Gao, Weixin Zhang, Bo Wang, Zhiyuan Zeng, Yikai Zhang, Yining Zheng, Xipeng Qiu

Published: 2026-04-13T10:59:28.000Z

## Abstract

Despite the success of reinforcement learning for large language models, a common failure mode is reduced sampling diversity, where the policy repeatedly generates similar erroneous behaviors. Classical entropy regularization encourages randomness under the current policy, but does not explicitly discourage recurrent failure patterns across rollouts. We propose MEDS, a Memory-Enhanced Dynamic reward Shaping framework that incorporates historical behavioral signals into reward design. By storing and leveraging intermediate model representations, we capture features of past rollouts and use density-based clustering to identify frequently recurring error patterns. Rollouts assigned to more prevalent error clusters are penalized more heavily, encouraging broader exploration while reducing repeated mistakes. Across five datasets and three base models, MEDS consistently improves average performance over existing baselines, achieving gains of up to 4.13 pass@1 points and 4.37 pass@128 points. Additional analyses using both LLM-based annotations and quantitative diversity metrics show that MEDS increases behavioral diversity during sampling.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Mathematical Reasoning on AMC 23 (pass@1, pass@128)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-amc-23-pass-at-1-pass-at-128.md>): Mathematical Reasoning; AMC 23; Pass@1: 82.62.

- [Mathematical Reasoning on MATH 500 (pass@1, pass@128)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-500-pass-at-1-pass-at-128.md>): Mathematical Reasoning; MATH 500; Pass@1 Rate: 92.51.

- [Mathematical Reasoning on Minerva (pass@1, pass@128)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-minerva-pass-at-1-pass-at-128.md>): Mathematical Reasoning; Minerva; Pass@1: 51.58.

- [Mathematical Reasoning on OlympiadBench (pass@1, pass@128)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-olympiadbench-pass-at-1-pass-at-128.md>): Mathematical Reasoning; OlympiadBench; Pass@1: 61.12.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.11297>)

[PDF](<https://arxiv.org/pdf/2604.11297.pdf>)

[Implementation](<https://github.com/Linxi000/MEDS>)
