# Long-context Reasoning on Long-context Benchmarks 16K (DocMath, Frames, LB-MQA, V2) (test) Source: [Long-context Reasoning on Long-context Benchmarks 16K (DocMath, Frames, LB-MQA, V2) (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:41:14.422Z Evaluation of Long-context Reasoning performance on a specific 16K context configuration of the Long-context Benchmarks suite, reporting scores for DocMath, Frames, and LB-MQA tasks (V2, test split). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-context Reasoning | | Dataset (including any stated split) | Long-context Benchmarks 16K context DocMath Frames LB-MQA V2 (test) | | Metric | DocMath | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 36 of 36 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [DocMath](): Higher is better; unit: PERCENT. - [Frames](): Higher is better; unit: PERCENT. - [LB-MQA](): Higher is better; unit: PERCENT. - [LB-V2](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | DocMath | Result date | Report paper | | --- | --- | --- | --- | | Qwen3-30B-A3B-Thinking + SPELL | 64.1 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen3-30B-A3B-Instruct + SPELL | 63 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen3-30B-A3B-Thinking | 62.9 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen3-30B-A3B-Thinking + RLVR | 62.7 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen3-30B-A3B-Instruct + RLVR | 62.5 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-32B-Instruct + SPELL | 62.3 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen3-30B-A3B-Instruct | 62.3 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen3-4B-Thinking + SPELL | 61.9 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-32B + SPELL | 61.8 | Not recorded | 68fa7bb41e209131cea8a096 | | R1-Distill-Qwen-14B + SPELL | 61.6 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen3-4B-Thinking + RLVR | 60.5 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-32B-Instruct | 60 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-32B-Instruct + RLVR | 59.9 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-14B-Instruct + SPELL | 59.6 | Not recorded | 68fa7bb41e209131cea8a096 | | R1-Distill-Qwen-14B + RLVR | 59.6 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen3-4B-Thinking | 58.6 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-32B + RLVR | 58.3 | Not recorded | 68fa7bb41e209131cea8a096 | | R1-Distill-Qwen-14B | 57.7 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-14B + SPELL | 57.6 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-14B-Instruct | 56.3 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-14B-Instruct + RLVR | 56.1 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-14B + RLVR | 52.2 | Not recorded | 68fa7bb41e209131cea8a096 | | R1-Distill-Llama-8B + SPELL | 48.9 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-32B | 46.8 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-7B-Instruct + SPELL | 45.8 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-7B-Instruct + RLVR | 45 | Not recorded | 68fa7bb41e209131cea8a096 | | R1-Distill-Llama-8B + RLVR | 43.4 | Not recorded | 68fa7bb41e209131cea8a096 | | R1-Distill-Llama-8B | 42 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-7B + RLVR | 41.8 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-7B + SPELL | 40 | Not recorded | 68fa7bb41e209131cea8a096 | | Llama3.1-8B-Instruct + SPELL | 39.2 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-7B-Instruct | 38.4 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-14B | 38 | Not recorded | 68fa7bb41e209131cea8a096 | | Llama3.1-8B-Instruct + RLVR | 37.9 | Not recorded | 68fa7bb41e209131cea8a096 | | Llama3.1-8B-Instruct | 33.2 | Not recorded | 68fa7bb41e209131cea8a096 | | Qwen2.5-7B | 10.9 | Not recorded | 68fa7bb41e209131cea8a096 | ### Qwen3-30B-A3B-Thinking + SPELL DocMath: 64.1. Result ID: 699ae4cb5698665ec474eb23. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + SPELL DocMath: 63. Result ID: 699ae4cb5698665ec474eb14. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Thinking DocMath: 62.9. Result ID: 699ae4cb5698665ec474eb21. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Thinking + RLVR DocMath: 62.7. Result ID: 699ae4cb5698665ec474eb22. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + RLVR DocMath: 62.5. Result ID: 699ae4cb5698665ec474eb13. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-32B-Instruct + SPELL DocMath: 62.3. Result ID: 699ae4cb5698665ec474eb11. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct DocMath: 62.3. Result ID: 699ae4cb5698665ec474eb12. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking + SPELL DocMath: 61.9. Result ID: 699ae4cb5698665ec474eb20. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-32B + SPELL DocMath: 61.8. Result ID: 699ae4cb5698665ec474eb08. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Base Models","Input Length":"16K"}. This record does not assert independent reproduction. ### R1-Distill-Qwen-14B + SPELL DocMath: 61.6. Result ID: 699ae4cb5698665ec474eb1d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking + RLVR DocMath: 60.5. Result ID: 699ae4cb5698665ec474eb1f. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-32B-Instruct DocMath: 60. Result ID: 699ae4cb5698665ec474eb0f. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-32B-Instruct + RLVR DocMath: 59.9. Result ID: 699ae4cb5698665ec474eb10. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-14B-Instruct + SPELL DocMath: 59.6. Result ID: 699ae4cb5698665ec474eb0e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### R1-Distill-Qwen-14B + RLVR DocMath: 59.6. Result ID: 699ae4cb5698665ec474eb1c. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking DocMath: 58.6. Result ID: 699ae4cb5698665ec474eb1e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-32B + RLVR DocMath: 58.3. Result ID: 699ae4cb5698665ec474eb07. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Base Models","Input Length":"16K"}. This record does not assert independent reproduction. ### R1-Distill-Qwen-14B DocMath: 57.7. Result ID: 699ae4cb5698665ec474eb1b. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-14B + SPELL DocMath: 57.6. Result ID: 699ae4cb5698665ec474eb05. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Base Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-14B-Instruct DocMath: 56.3. Result ID: 699ae4cb5698665ec474eb0c. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-14B-Instruct + RLVR DocMath: 56.1. Result ID: 699ae4cb5698665ec474eb0d. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-14B + RLVR DocMath: 52.2. Result ID: 699ae4cb5698665ec474eb04. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Base Models","Input Length":"16K"}. This record does not assert independent reproduction. ### R1-Distill-Llama-8B + SPELL DocMath: 48.9. Result ID: 699ae4cb5698665ec474eb1a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-32B DocMath: 46.8. Result ID: 699ae4cb5698665ec474eb06. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Base Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + SPELL DocMath: 45.8. Result ID: 699ae4cb5698665ec474eb0b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + RLVR DocMath: 45. Result ID: 699ae4cb5698665ec474eb0a. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### R1-Distill-Llama-8B + RLVR DocMath: 43.4. Result ID: 699ae4cb5698665ec474eb19. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### R1-Distill-Llama-8B DocMath: 42. Result ID: 699ae4cb5698665ec474eb18. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Reasoning Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-7B + RLVR DocMath: 41.8. Result ID: 699ae4cb5698665ec474eb01. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Base Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-7B + SPELL DocMath: 40. Result ID: 699ae4cb5698665ec474eb02. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Base Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Llama3.1-8B-Instruct + SPELL DocMath: 39.2. Result ID: 699ae4cb5698665ec474eb17. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct DocMath: 38.4. Result ID: 699ae4cb5698665ec474eb09. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-14B DocMath: 38. Result ID: 699ae4cb5698665ec474eb03. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Base Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Llama3.1-8B-Instruct + RLVR DocMath: 37.9. Result ID: 699ae4cb5698665ec474eb16. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Llama3.1-8B-Instruct DocMath: 33.2. Result ID: 699ae4cb5698665ec474eb15. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Instruct Models","Input Length":"16K"}. This record does not assert independent reproduction. ### Qwen2.5-7B DocMath: 10.9. Result ID: 699ae4cb5698665ec474eb00. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Base Models","Input Length":"16K"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.