# Long-context Question Answering on RULER HotpotQA Source: [Long-context Question Answering on RULER HotpotQA]() Collection index: [llms.txt]() Data updated: 2026-05-29T11:13:53.558Z Evaluation of model performance on the RULER variant of the HotpotQA dataset, specifically testing Long-context Question Answering capabilities across a wide range of context lengths (from 7K tokens up to 3.5M tokens). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-context Question Answering | | Dataset (including any stated split) | RULER HotpotQA | | Metric | Accuracy @ 7K Context | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 10 of 10 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy @ 7K Context](): Higher is better; unit: PERCENT. - [Accuracy @ 14K Context](): Higher is better; unit: PERCENT. - [Accuracy @ 28K Context](): Higher is better; unit: PERCENT. - [Accuracy @ 56K Context](): Higher is better; unit: PERCENT. - [Accuracy @ 112K Context](): Higher is better; unit: PERCENT. - [Accuracy @ 224K Context](): Higher is better; unit: PERCENT. - [Accuracy @ 448K Context](): Higher is better; unit: PERCENT. - [Accuracy @ 896K Context](): Higher is better; unit: PERCENT. - [Accuracy @ 1.75M Context](): Higher is better; unit: PERCENT. - [Accuracy @ 3.5M Context](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy @ 7K Context | Result date | Report paper | | --- | --- | --- | --- | | MMPO-7B | 82.38 | Not recorded | 6a191db8dbd7150506f82996 | | MMPO-14B | 82.16 | Not recorded | 6a191db8dbd7150506f82996 | | RL-MemAgent-7B | 81.25 | Not recorded | 6a191db8dbd7150506f82996 | | RL-MemAgent-14B | 80.47 | Not recorded | 6a191db8dbd7150506f82996 | | QwenLong-L1-32B | 72.66 | Not recorded | 6a191db8dbd7150506f82996 | | DS-Qwen-32B | 70.31 | Not recorded | 6a191db8dbd7150506f82996 | | DS-Qwen-14B | 64.06 | Not recorded | 6a191db8dbd7150506f82996 | | Qwen2.5-7B-1M | 61.72 | Not recorded | 6a191db8dbd7150506f82996 | | Qwen2.5-14B-1M | 60.16 | Not recorded | 6a191db8dbd7150506f82996 | | DS-Qwen-7B | 30.47 | Not recorded | 6a191db8dbd7150506f82996 | ### MMPO-7B Accuracy @ 7K Context: 82.38. Result ID: 6a19732bfbc373592c14a599. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Memory Workflow":"MemAgent","Training Supervision":"Belief Entropy"}. This record does not assert independent reproduction. ### MMPO-14B Accuracy @ 7K Context: 82.16. Result ID: 6a19732bfbc373592c14a597. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"14B","Memory Workflow":"MemAgent","Training Supervision":"Belief Entropy"}. This record does not assert independent reproduction. ### RL-MemAgent-7B Accuracy @ 7K Context: 81.25. Result ID: 6a19732bfbc373592c14a598. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Memory Workflow":"MemAgent","Training Supervision":"RL"}. This record does not assert independent reproduction. ### RL-MemAgent-14B Accuracy @ 7K Context: 80.47. Result ID: 6a19732bfbc373592c14a596. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"14B","Memory Workflow":"MemAgent","Training Supervision":"RL"}. This record does not assert independent reproduction. ### QwenLong-L1-32B Accuracy @ 7K Context: 72.66. Result ID: 6a19732bfbc373592c14a590. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"32B","Backbone":"QwenLong-L1"}. This record does not assert independent reproduction. ### DS-Qwen-32B Accuracy @ 7K Context: 70.31. Result ID: 6a19732bfbc373592c14a593. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"32B","Backbone":"DS-Qwen"}. This record does not assert independent reproduction. ### DS-Qwen-14B Accuracy @ 7K Context: 64.06. Result ID: 6a19732bfbc373592c14a594. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"14B","Backbone":"DS-Qwen"}. This record does not assert independent reproduction. ### Qwen2.5-7B-1M Accuracy @ 7K Context: 61.72. Result ID: 6a19732bfbc373592c14a592. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Backbone":"Qwen2.5","Context Window":"1M"}. This record does not assert independent reproduction. ### Qwen2.5-14B-1M Accuracy @ 7K Context: 60.16. Result ID: 6a19732bfbc373592c14a591. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"14B","Backbone":"Qwen2.5","Context Window":"1M"}. This record does not assert independent reproduction. ### DS-Qwen-7B Accuracy @ 7K Context: 30.47. Result ID: 6a19732bfbc373592c14a595. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Backbone":"DS-Qwen"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.