Needle-in-a-Haystack
Benchmarks
TaskDatasetSOTA resultTrendResults
Needle-in-a-Haystack (test)
100Accuracy
56
Needle-in-a-Haystack
100Accuracy
44
Needle In A Haystack
67Exact-match Accuracy
40
Needle-in-a-haystack 4x original context
100Accuracy
35
Needle-in-a-Haystack 32K context (test)
76Accuracy
30
Needle-in-a-Haystack 8K context (test)
100Accuracy
30
Needle-in-a-Haystack
100Retrieval Accuracy
29
Needle-in-a-Haystack L=8k
100Accuracy
24
Needle-in-A-Haystack RULER
100Success Rate (4K Context)
22
Needle-in-a-Haystack 32K context
98.2Accuracy
19
Needle-in-a-Haystack Induction
98Accuracy
18
Multi-key needle-in-a-haystack 16k context length
100Recall
16
Needle-in-a-Haystack 64K-token context
98.7Success Rate
12
Needle-in-a-Haystack (NIAH)
99.5Mean Recall
9
Needle-In-A-Haystack (NIAH)
99.8NIAH-1 Success Rate
7
Needle-In-a-Haystack Verbatim prompt (test)
0.996Accuracy (Depth 0%)
7
Needle-In-a-Haystack Gemini prompt (test)
57.2Success Rate @ 0% Insertion
7
Needle In A Haystack
90Recall@1K
6
Needle-in-a-Haystack (NIAH) 64K
97.6Average Accuracy
5
Needle-in-a-Haystack (NIAH) 32K
100Average Accuracy
5
Needle-in-a-Haystack (NIAH) 16K
100Average Accuracy
5
Needle-in-a-Haystack 1.0 (test)
99.9Score
5
Needle-in-a-haystack 8x original context
52.2Accuracy
4
Needle-in-a-haystack 2x original context
74.92Needle-in-a-haystack Accuracy (2x Context)
4
Needle-in-a-Haystack
98.2Average Score
3