# SIRI: Self-Internalizing Reinforcement Learning with Intrinsic Skills for LLM Agent Training

Source: [SIRI: Self-Internalizing Reinforcement Learning with Intrinsic Skills for LLM Agent Training](<https://www.sota2.com/research/paper/6a1e6423dbd7150506f857c1>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-02T05:03:19.764Z

Authors: Zhongyu He, Yuanfan Li, Fei Huang, Tianyu Chen, Siyuan Chen, Xingyang Li, Meng Hsuan Yu, Xiangrong Liu, Leyi Wei, Lu Pan, Ke Zeng, Xunliang Cai

Published: 2026-06-01T15:02:59.000Z

## Abstract

Long-horizon LLM agents can benefit from reusable skills, yet existing skill-based methods often rely on external skill generators during training or persistent skill retrieval at inference, increasing engineering complexity, context length, and deployment latency. We propose Self-Internalizing Reinforcement learning with Intrinsic skills (SIRI), a three-phase framework that enables agents to discover, validate, and internalize skills without external skill generators or inference-time skill banks. SIRI first warms up the policy with GiGPO to acquire basic interaction ability and collect successful skill-free trajectories. It then performs self-skill mining, where the current policy summarizes compact skills from its own successful plain rollouts and validates them through paired skill-augmented and skill-free rollouts. Finally, SIRI distills only beneficial skill-guided action tokens into the plain policy using trajectory-level utility and action-level advantage. At inference, the agent runs with the original prompt only. On ALFWorld and WebShop with Qwen2.5-7B-Instruct, SIRI improves GiGPO from 0.908 to 0.930 on ALFWorld and from 0.728 to 0.813 on WebShop, outperforming prompt-based, RL-based, and memory-augmented baselines. Further analysis shows that our self-mining strategy can achieve performance comparable to distillation with closed-source large model. Our code is available at https://github.com/kirito618/SIRI.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [E-commerce product search and purchase on WebShop](<https://www.sota2.com/research/sota/e-commerce-product-search-and-purchase-on-webshop.md>): E-commerce product search and purchase; Webshop; Strict Success: 81.3.

- [Embodied Task on ALFWorld](<https://www.sota2.com/research/sota/embodied-task-on-alfworld.md>): Embodied Task; AlfWorld; Overall Success Rate: 93.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.02355>)

[PDF](<https://arxiv.org/pdf/2606.02355.pdf>)