# Multi-needle retrieval on NIAH (M) Source: [Multi-needle retrieval on NIAH (M)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:34:20.519Z Evaluation of model performance on the NIAH (M) dataset for Multi-needle retrieval tasks, measured using Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-needle retrieval | | Dataset (including any stated split) | NIAH (M) | | Metric | Accuracy (NIAH M) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 35 of 35 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy (NIAH M)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy (NIAH M) | Result date | Report paper | | --- | --- | --- | --- | | GLM4-9B-Chat-1M | 90.2 | Not recorded | 68fa74c31e209131cea63a45 | | Gradient-Llama3-8B | 89.6 | Not recorded | 68fa74c31e209131cea63a45 | | Llama3.1-8B-Instruct | 89 | Not recorded | 68fa74c31e209131cea63a45 | | Yi-34B-200k | 84.9 | Not recorded | 68fa74c31e209131cea63a45 | | Qwen2-72B-Instruct | 83.4 | Not recorded | 68fa74c31e209131cea63a45 | | LongRecipe | 82.6 | Not recorded | 68fa74c31e209131cea63a45 | | FLT | 82.3 | Not recorded | 68fa74c31e209131cea63a45 | | Gemini-1.5-Pro | 82 | Not recorded | 68fa74c31e209131cea63a45 | | POSE | 80.1 | Not recorded | 68fa74c31e209131cea63a45 | | LongRecipe | 79.5 | Not recorded | 68fa74c31e209131cea63a45 | | Gradient-Llama3-70B | 79.2 | Not recorded | 68fa74c31e209131cea63a45 | | LongRecipe | 78.9 | Not recorded | 68fa74c31e209131cea63a45 | | GPT-4 | 76.2 | Not recorded | 68fa74c31e209131cea63a45 | | RPES | 73.7 | Not recorded | 68fa74c31e209131cea63a45 | | FLT | 73.2 | Not recorded | 68fa74c31e209131cea63a45 | | RPES | 72.7 | Not recorded | 68fa74c31e209131cea63a45 | | Llama3.1-8B | 72 | Not recorded | 68fa74c31e209131cea63a45 | | RPES | 71.8 | Not recorded | 68fa74c31e209131cea63a45 | | POSE | 70 | Not recorded | 68fa74c31e209131cea63a45 | | POSE | 68.8 | Not recorded | 68fa74c31e209131cea63a45 | | Llama3.1-70B-Instruct | 68.3 | Not recorded | 68fa74c31e209131cea63a45 | | LongRecipe | 65.8 | Not recorded | 68fa74c31e209131cea63a45 | | Yi-9B-200k | 65.7 | Not recorded | 68fa74c31e209131cea63a45 | | POSE | 64.7 | Not recorded | 68fa74c31e209131cea63a45 | | LongRecipe | 64.7 | Not recorded | 68fa74c31e209131cea63a45 | | FLT\* | 64.7 | Not recorded | 68fa74c31e209131cea63a45 | | RPES | 64.6 | Not recorded | 68fa74c31e209131cea63a45 | | RPES | 60.4 | Not recorded | 68fa74c31e209131cea63a45 | | POSE | 58.5 | Not recorded | 68fa74c31e209131cea63a45 | | LongRecipe | 53.4 | Not recorded | 68fa74c31e209131cea63a45 | | FLT\* | 52.7 | Not recorded | 68fa74c31e209131cea63a45 | | FLT | 43 | Not recorded | 68fa74c31e209131cea63a45 | | RPES | 41.9 | Not recorded | 68fa74c31e209131cea63a45 | | Qwen2-7B-Instruct | 38.8 | Not recorded | 68fa74c31e209131cea63a45 | | POSE | 35.9 | Not recorded | 68fa74c31e209131cea63a45 | ### GLM4-9B-Chat-1M Accuracy (NIAH M): 90.2. Result ID: 699366313a6c31484915a80f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### Gradient-Llama3-8B Accuracy (NIAH M): 89.6. Result ID: 699366313a6c31484915a80d. [Reporting paper]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### Llama3.1-8B-Instruct Accuracy (NIAH M): 89. Result ID: 699366313a6c31484915a80e. [Reporting paper]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### Yi-34B-200k Accuracy (NIAH M): 84.9. Result ID: 699366313a6c31484915a80b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### Qwen2-72B-Instruct Accuracy (NIAH M): 83.4. Result ID: 699366313a6c31484915a811. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### LongRecipe Accuracy (NIAH M): 82.6. Result ID: 699366313a6c31484915a7f9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama3-8B-I","Length":"128k"}. This record does not assert independent reproduction. ### FLT Accuracy (NIAH M): 82.3. Result ID: 699366313a6c31484915a7f2. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama3-8B-I","Length":"80k"}. This record does not assert independent reproduction. ### Gemini-1.5-Pro Accuracy (NIAH M): 82. Result ID: 699366313a6c31484915a814. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### POSE Accuracy (NIAH M): 80.1. Result ID: 699366313a6c31484915a7f8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3-8B-I","Length":"128k"}. This record does not assert independent reproduction. ### LongRecipe Accuracy (NIAH M): 79.5. Result ID: 699366313a6c31484915a804. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2-7B","Length":"80k"}. This record does not assert independent reproduction. ### Gradient-Llama3-70B Accuracy (NIAH M): 79.2. Result ID: 699366313a6c31484915a812. [Reporting paper]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### LongRecipe Accuracy (NIAH M): 78.9. Result ID: 699366313a6c31484915a7f5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama3-8B-I","Length":"80k"}. This record does not assert independent reproduction. ### GPT-4 Accuracy (NIAH M): 76.2. Result ID: 699366313a6c31484915a813. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### RPES Accuracy (NIAH M): 73.7. Result ID: 699366313a6c31484915a802. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2-7B","Length":"80k"}. This record does not assert independent reproduction. ### FLT Accuracy (NIAH M): 73.2. Result ID: 699366313a6c31484915a7f6. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama3-8B-I","Length":"128k"}. This record does not assert independent reproduction. ### RPES Accuracy (NIAH M): 72.7. Result ID: 699366313a6c31484915a7f7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama3-8B-I","Length":"128k"}. This record does not assert independent reproduction. ### Llama3.1-8B Accuracy (NIAH M): 72. Result ID: 699366313a6c31484915a809. [Reporting paper]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### RPES Accuracy (NIAH M): 71.8. Result ID: 699366313a6c31484915a7f3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama3-8B-I","Length":"80k"}. This record does not assert independent reproduction. ### POSE Accuracy (NIAH M): 70. Result ID: 699366313a6c31484915a803. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2-7B","Length":"80k"}. This record does not assert independent reproduction. ### POSE Accuracy (NIAH M): 68.8. Result ID: 699366313a6c31484915a7f4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3-8B-I","Length":"80k"}. This record does not assert independent reproduction. ### Llama3.1-70B-Instruct Accuracy (NIAH M): 68.3. Result ID: 699366313a6c31484915a810. [Reporting paper]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### LongRecipe Accuracy (NIAH M): 65.8. Result ID: 699366313a6c31484915a808. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2-7B","Length":"128k"}. This record does not assert independent reproduction. ### Yi-9B-200k Accuracy (NIAH M): 65.7. Result ID: 699366313a6c31484915a80a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### POSE Accuracy (NIAH M): 64.7. Result ID: 699366313a6c31484915a7fc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Mistral-7B","Length":"80k"}. This record does not assert independent reproduction. ### LongRecipe Accuracy (NIAH M): 64.7. Result ID: 699366313a6c31484915a7fd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Mistral-7B","Length":"80k"}. This record does not assert independent reproduction. ### FLT\* Accuracy (NIAH M): 64.7. Result ID: 699366313a6c31484915a801. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2-7B","Length":"80k","Strategy":"YARN and DCA"}. This record does not assert independent reproduction. ### RPES Accuracy (NIAH M): 64.6. Result ID: 699366313a6c31484915a806. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2-7B","Length":"128k"}. This record does not assert independent reproduction. ### RPES Accuracy (NIAH M): 60.4. Result ID: 699366313a6c31484915a7fb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Mistral-7B","Length":"80k"}. This record does not assert independent reproduction. ### POSE Accuracy (NIAH M): 58.5. Result ID: 699366313a6c31484915a807. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2-7B","Length":"128k"}. This record does not assert independent reproduction. ### LongRecipe Accuracy (NIAH M): 53.4. Result ID: 699366313a6c31484915a800. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Mistral-7B","Length":"128k"}. This record does not assert independent reproduction. ### FLT\* Accuracy (NIAH M): 52.7. Result ID: 699366313a6c31484915a805. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2-7B","Length":"128k","Strategy":"YARN and DCA"}. This record does not assert independent reproduction. ### FLT Accuracy (NIAH M): 43. Result ID: 699366313a6c31484915a7fa. [Reporting paper]() Recorded evaluation configuration: {"Model":"Mistral-7B","Length":"80k"}. This record does not assert independent reproduction. ### RPES Accuracy (NIAH M): 41.9. Result ID: 699366313a6c31484915a7fe. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Mistral-7B","Length":"128k"}. This record does not assert independent reproduction. ### Qwen2-7B-Instruct Accuracy (NIAH M): 38.8. Result ID: 699366313a6c31484915a80c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Length":"128k"}. This record does not assert independent reproduction. ### POSE Accuracy (NIAH M): 35.9. Result ID: 699366313a6c31484915a7ff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Mistral-7B","Length":"128k"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.