# Reinforcement Learning for Neural Model Editing

Source: [Reinforcement Learning for Neural Model Editing](<https://www.sota2.com/research/paper/6a2b929ed33dfb23e783b0ae>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-13T05:00:13.110Z

Authors: Shaivi Malik

Published: 2026-06-11T15:16:42.000Z

## Abstract

Editing pretrained neural networks requires specialized algorithms tailored to specific objectives. Designing such algorithms is often time-consuming and demands significant effort. We present an exploratory framework that formulates neural model editing as a reinforcement learning problem, where agents modify models using reward feedback. We introduce two environments: MaskWorld, where agents scale weights multiplicatively, and ShiftWorld, where agents apply additive weight updates. The reward function combines a utility-preservation objective with a task-specific editing objective, enabling agents to learn targeted modifications while maintaining overall model performance. We evaluate the framework on bias mitigation in text classification and machine unlearning in image classification, both of which traditionally rely on specialized algorithms. Our results show that the learned policies reduce forget set accuracy to nearly 0% while preserving over 90% retain set accuracy on the unlearning task. In the bias mitigation setting, the learned policies improve bias-related performance by more than 5% while maintaining general classification utility. Our findings show that neural model editing can be cast as a reinforcement learning problem, allowing editing policies to be learned from reward feedback rather than manually engineered for each task.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Text Classification on Jigsaw Toxic Comment Classification Unbiased Dataset synthetic (test)](<https://www.sota2.com/research/sota/text-classification-on-jigsaw-toxic-comment-classification-unbiased-dataset-synthetic-test.md>): Text Classification; Jigsaw Toxic Comment Classification Unbiased Dataset synthetic (test); Accuracy: 62.52.

- [Machine Unlearning on MNIST Retain Set (classes 1-6 and 8-9)](<https://www.sota2.com/research/sota/machine-unlearning-on-mnist-retain-set-classes-1-6-and-8-9.md>): Machine Unlearning; MNIST Retain Set (classes 1-6 and 8-9); Accuracy: 93.51.

- [Text Classification on Jigsaw Toxic Comment Classification General Dataset 2017 (test)](<https://www.sota2.com/research/sota/text-classification-on-jigsaw-toxic-comment-classification-general-dataset-2017-test.md>): Text Classification; Jigsaw Toxic Comment Classification General Dataset 2017 (test); Accuracy: 83.2.

- [Machine Unlearning Accuracy on MNIST Forget Set (class 7)](<https://www.sota2.com/research/sota/machine-unlearning-accuracy-on-mnist-forget-set-class-7.md>): Machine Unlearning; MNIST Forget Set (class 7); Accuracy (Forget Set): 0.06.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.13461>)

[PDF](<https://arxiv.org/pdf/2606.13461.pdf>)
