# UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents

Source: [UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents](<https://www.sota2.com/research/paper/698575d4c97a0b3e3b5d5e5d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-07T05:00:14.331Z

Authors: Han Xiao, Guozhi Wang, Hao Wang, Shilong Liu, Yuxiang Chai, Yue Pan, Yufeng Zhou, Xiaoxin Chen, Yafei Wen, Hongsheng Li

Published: 2026-02-05T16:21:43.000Z

## Abstract

Online Reinforcement Learning (RL) offers a promising paradigm for enhancing GUI agents through direct environment interaction. However, its effectiveness is severely hindered by inefficient credit assignment in long-horizon tasks and repetitive errors across tasks due to the lack of experience transfer. To address these challenges, we propose UI-Mem, a novel framework that enhances GUI online RL with a Hierarchical Experience Memory. Unlike traditional replay buffers, our memory accumulates structured knowledge, including high-level workflows, subtask skills, and failure patterns. These experiences are stored as parameterized templates that enable cross-task and cross-application transfer. To effectively integrate memory guidance into online RL, we introduce Stratified Group Sampling, which injects varying levels of guidance across trajectories within each rollout group to maintain outcome diversity, driving the unguided policy toward internalizing guided behaviors. Furthermore, a Self-Evolving Loop continuously abstracts novel strategies and errors to keep the memory aligned with the agent's evolving policy. Experiments on online GUI benchmarks demonstrate that UI-Mem significantly outperforms traditional RL baselines and static reuse strategies, with strong generalization to unseen applications. Project page: https://ui-mem.github.io

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Mobile GUI Agent Task on AndroidLab](<https://www.sota2.com/research/sota/mobile-gui-agent-task-on-androidlab.md>): Mobile GUI Agent Task; AndroidLab; Success Rate: 44.9.

- [Mobile GUI Agents on AndroidWorld 138 tasks (test)](<https://www.sota2.com/research/sota/mobile-gui-agents-on-androidworld-138-tasks-test.md>): Mobile GUI Agents; AndroidWorld 138 tasks (test); Success Rate: 71.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2602.05832>)

[PDF](<https://arxiv.org/pdf/2602.05832.pdf>)