# MemBuilder: Reinforcing LLMs for Long-Term Memory Construction via Attributed Dense Rewards

Source: [MemBuilder: Reinforcing LLMs for Long-Term Memory Construction via Attributed Dense Rewards](<https://www.sota2.com/research/paper/69647fe753fdd5c361744743>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-24T05:00:22.884Z

Authors: Zhiyu Shen, Ziming Wu, Fuming Lai, Shaobing Lian, Yanghui Rao

Published: 2026-01-09T02:44:37.000Z

## Abstract

Maintaining consistency in long-term dialogues remains a fundamental challenge for LLMs, as standard retrieval mechanisms often fail to capture the temporal evolution of historical states. While memory-augmented frameworks offer a structured alternative, current systems rely on static prompting of closed-source models or suffer from ineffective training paradigms with sparse rewards. We introduce MemBuilder, a reinforcement learning framework that trains models to orchestrate multi-dimensional memory construction with attributed dense rewards. MemBuilder addresses two key challenges: (1) Sparse Trajectory-Level Rewards: we employ synthetic session-level question generation to provide dense intermediate rewards across extended trajectories; and (2) Multi-Dimensional Memory Attribution: we introduce contribution-aware gradient weighting that scales policy updates based on each component's downstream impact. Experimental results show that MemBuilder enables a 4B-parameter model to outperform state-of-the-art closed-source baselines, exhibiting strong generalization across long-term dialogue benchmarks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Long-term dialogue memory on LoCoMo (test)](<https://www.sota2.com/research/sota/long-term-dialogue-memory-on-locomo-test.md>): Long-term dialogue memory; LoCoMo (test); Accuracy: 84.23.

- [Long-term dialogue memory on LongMemEval (test)](<https://www.sota2.com/research/sota/long-term-dialogue-memory-on-longmemeval-test.md>): Long-term dialogue memory; LongMemEval (test); Accuracy: 85.75.

- [Long-term dialogue memory on PerLTQA (test)](<https://www.sota2.com/research/sota/long-term-dialogue-memory-on-perltqa-test.md>): Long-term dialogue memory; PerLTQA (test); Accuracy: 93.14.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.05488>)

[PDF](<https://arxiv.org/pdf/2601.05488.pdf>)