# Think, But Don't Overthink: Reproducing Recursive Language Models Source: [Think, But Don't Overthink: Reproducing Recursive Language Models]() Collection index: [llms.txt]() Data updated: 2026-03-05T05:00:04.307Z Authors: Daren Wang Published: 2026-03-03T05:36:44.000Z ## Abstract This project reproduces and extends the recently proposed \`\`Recursive Language Models'' (RLMs) framework by Zhang et al. (2026). This framework enables Large Language Models (LLMs) to process near-infinite contexts by offloading the prompt into an external REPL environment. While the original paper relies on a default recursion depth of 1 and suggests deeper recursion as a future direction, this study specifically investigates the impact of scaling the recursion depth. Using state-of-the-art open-source agentic models (DeepSeek v3.2 and Kimi K2), I evaluated pure LLM, RLM (depth=1), and RLM (depth=2) on the S-NIAH and OOLONG benchmarks. The findings reveal a compelling phenomenon: Deeper recursion causes models to \`\`overthink''. While depth-1 RLMs effectively boost accuracy on complex reasoning tasks, applying deeper recursion (depth=2) or using RLMs on simple retrieval tasks paradoxically degrades performance and exponentially inflates execution time (e.g., from 3.6s to 344.5s) and token costs. Code and data are available at: https://github.com/drbillwang/rlm-reproduction ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Simple Retrieval on Ruler S-NIAH](): Simple Retrieval; Ruler S-NIAH; Accuracy: 100. - [Long-context reasoning on OOLONG trec\_coarse](): Long-context Reasoning; OOLONG trec\_coarse; Score: 60. ## Original sources and implementation [Original paper]() [PDF]()