# Self-Supervised Reinforcement Learning for Recommender Systems Source: [Self-Supervised Reinforcement Learning for Recommender Systems]() Collection index: [llms.txt]() Data updated: 2025-10-25T22:21:05.859Z Authors: Xin Xin, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M. Jose Published: 2020-06-10T11:18:57.000Z ## Abstract In session-based or sequential recommendation, it is important to consider a number of factors like long-term user engagement, multiple types of user-item interactions such as clicks, purchases etc. The current state-of-the-art supervised approaches fail to model them appropriately. Casting sequential recommendation task as a reinforcement learning (RL) problem is a promising direction. A major component of RL approaches is to train the agent through interactions with the environment. However, it is often problematic to train a recommender in an on-line fashion due to the requirement to expose users to irrelevant recommendations. As a result, learning the policy from logged implicit feedback is of vital importance, which is challenging due to the pure off-policy setting and lack of negative rewards (feedback). In this paper, we propose self-supervised reinforcement learning for sequential recommendation tasks. Our approach augments standard recommendation models with two output layers: one for self-supervised learning and the other for RL. The RL part acts as a regularizer to drive the supervised layer focusing on specific rewards(e.g., recommending items which may lead to purchases rather than clicks) while the self-supervised layer with cross-entropy loss provides strong gradient signals for parameter updates. Based on such an approach, we propose two frameworks namely Self-Supervised Q-learning(SQN) and Self-Supervised Actor-Critic(SAC). We integrate the proposed frameworks with four state-of-the-art recommendation models. Experimental results on two real-world datasets demonstrate the effectiveness of our approach. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Interactive Recommendation on KuaiRec Max Len 30](): Interactive Recommendation; KuaiRec Max Len 30; Rcum: 13.46. - [Interactive Recommendation on KuaiRec Max Len 50](): Interactive Recommendation; KuaiRec Max Len 50; Rcum: 13.23. - [Interactive Recommendation on KuaiRand Max Len 50](): Interactive Recommendation; KuaiRand Max Len 50; Rcum: 1.76. ## Original sources and implementation [Original paper]() [PDF]()