# Long-context reasoning on OOLONG trec\_coarse Source: [Long-context reasoning on OOLONG trec\_coarse]() Collection index: [llms.txt]() Data updated: 2026-03-18T06:38:19.234Z Evaluation of model performance on long-context reasoning tasks using the OOLONG trec\_coarse dataset, measured by Score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-context reasoning | | Dataset (including any stated split) | OOLONG trec\_coarse | | Metric | Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | Score | Result date | Report paper | | --- | --- | --- | --- | | Kimi K2 | 86.6 | Not recorded | 69a7bc982d603e6de3e3d675 | | SRLM | 65.5 | Not recorded | 69ba316b0636e5bf4afc05f4 | | SRLM (no sub-calls) | 60.7 | Not recorded | 69ba316b0636e5bf4afc05f4 | | RLM | 60 | Not recorded | 69a7bc982d603e6de3e3d675 | | RLM | 56.5 | Not recorded | 69a7bc982d603e6de3e3d675 | | RLM | 55 | Not recorded | 69a7bc982d603e6de3e3d675 | | RLM | 53 | Not recorded | 69ba316b0636e5bf4afc05f4 | | SRLM | 51.8 | Not recorded | 69ba316b0636e5bf4afc05f4 | | RLM (no sub-calls) | 50.5 | Not recorded | 69ba316b0636e5bf4afc05f4 | | RLM | 48 | Not recorded | 69a7bc982d603e6de3e3d675 | | Summary agent | 46 | Not recorded | 69ba316b0636e5bf4afc05f4 | | SRLM (no sub-calls) | 45.9 | Not recorded | 69ba316b0636e5bf4afc05f4 | | RLM | 45.7 | Not recorded | 69ba316b0636e5bf4afc05f4 | | Summary agent | 44.1 | Not recorded | 69ba316b0636e5bf4afc05f4 | | GPT-5 | 44 | Not recorded | 69a7bc982d603e6de3e3d675 | | Base Model | 44 | Not recorded | 69ba316b0636e5bf4afc05f4 | | RLM | 42.1 | Not recorded | 69a7bc982d603e6de3e3d675 | | CodeAct (+ sub-calls) | 40 | Not recorded | 69ba316b0636e5bf4afc05f4 | | RLM (no sub-calls) | 39.1 | Not recorded | 69ba316b0636e5bf4afc05f4 | | CodeAct (+ BM25) | 38 | Not recorded | 69ba316b0636e5bf4afc05f4 | | CodeAct (+ BM25) | 38 | Not recorded | 69ba316b0636e5bf4afc05f4 | | Qwen3-Coder-480B-A35B | 36 | Not recorded | 69a7bc982d603e6de3e3d675 | | Base Model | 36 | Not recorded | 69ba316b0636e5bf4afc05f4 | | RLM | 33.7 | Not recorded | 69a7bc982d603e6de3e3d675 | | CodeAct (+ sub-calls) | 32 | Not recorded | 69ba316b0636e5bf4afc05f4 | | RLM | 24 | Not recorded | 69a7bc982d603e6de3e3d675 | | Qwen3-8B | 0 | Not recorded | 69a7bc982d603e6de3e3d675 | | DeepSeek v3.2 | 0 | Not recorded | 69a7bc982d603e6de3e3d675 | ### Kimi K2 Score: 86.6. Result ID: 69a886df6dbb7075b950f8d7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"Base LLM"}. This record does not assert independent reproduction. ### SRLM Score: 65.5. Result ID: 69ba484447f34ce924a37563. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5"}. This record does not assert independent reproduction. ### SRLM (no sub-calls) Score: 60.7. Result ID: 69ba484447f34ce924a37564. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5"}. This record does not assert independent reproduction. ### RLM Score: 60. Result ID: 69a886df6dbb7075b950f8d8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Kimi K2","Recursion Depth":"1"}. This record does not assert independent reproduction. ### RLM Score: 56.5. Result ID: 69a886df6dbb7075b950f8cf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","Recursion Depth":"1"}. This record does not assert independent reproduction. ### RLM Score: 55. Result ID: 69a886df6dbb7075b950f8d9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Kimi K2","Recursion Depth":"2"}. This record does not assert independent reproduction. ### RLM Score: 53. Result ID: 69ba484447f34ce924a37561. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"GPT-5"}. This record does not assert independent reproduction. ### SRLM Score: 51.8. Result ID: 69ba484447f34ce924a3755b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-Coder-480B"}. This record does not assert independent reproduction. ### RLM (no sub-calls) Score: 50.5. Result ID: 69ba484447f34ce924a37562. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"GPT-5"}. This record does not assert independent reproduction. ### RLM Score: 48. Result ID: 69a886df6dbb7075b950f8d1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-Coder-480B-A35B","Recursion Depth":"1"}. This record does not assert independent reproduction. ### Summary agent Score: 46. Result ID: 69ba484447f34ce924a37560. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5"}. This record does not assert independent reproduction. ### SRLM (no sub-calls) Score: 45.9. Result ID: 69ba484447f34ce924a3755c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-Coder-480B"}. This record does not assert independent reproduction. ### RLM Score: 45.7. Result ID: 69ba484447f34ce924a37559. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-Coder-480B"}. This record does not assert independent reproduction. ### Summary agent Score: 44.1. Result ID: 69ba484447f34ce924a37558. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-Coder-480B"}. This record does not assert independent reproduction. ### GPT-5 Score: 44. Result ID: 69a886df6dbb7075b950f8ce. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Architecture":"Base LLM"}. This record does not assert independent reproduction. ### Base Model Score: 44. Result ID: 69ba484447f34ce924a3755d. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5"}. This record does not assert independent reproduction. ### RLM Score: 42.1. Result ID: 69a886df6dbb7075b950f8d5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek v3.2","Recursion Depth":"1"}. This record does not assert independent reproduction. ### CodeAct (+ sub-calls) Score: 40. Result ID: 69ba484447f34ce924a3755f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5"}. This record does not assert independent reproduction. ### RLM (no sub-calls) Score: 39.1. Result ID: 69ba484447f34ce924a3755a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-Coder-480B"}. This record does not assert independent reproduction. ### CodeAct (+ BM25) Score: 38. Result ID: 69ba484447f34ce924a37556. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-Coder-480B"}. This record does not assert independent reproduction. ### CodeAct (+ BM25) Score: 38. Result ID: 69ba484447f34ce924a3755e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5"}. This record does not assert independent reproduction. ### Qwen3-Coder-480B-A35B Score: 36. Result ID: 69a886df6dbb7075b950f8d0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Architecture":"Base LLM"}. This record does not assert independent reproduction. ### Base Model Score: 36. Result ID: 69ba484447f34ce924a37555. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-Coder-480B"}. This record does not assert independent reproduction. ### RLM Score: 33.7. Result ID: 69a886df6dbb7075b950f8d6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek v3.2","Recursion Depth":"2"}. This record does not assert independent reproduction. ### CodeAct (+ sub-calls) Score: 32. Result ID: 69ba484447f34ce924a37557. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-Coder-480B"}. This record does not assert independent reproduction. ### RLM Score: 24. Result ID: 69a886df6dbb7075b950f8d3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B","Recursion Depth":"1"}. This record does not assert independent reproduction. ### Qwen3-8B Score: 0. Result ID: 69a886df6dbb7075b950f8d2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Architecture":"Base LLM"}. This record does not assert independent reproduction. ### DeepSeek v3.2 Score: 0. Result ID: 69a886df6dbb7075b950f8d4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"Base LLM"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.