NIAH
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
NIAH (test)
99.6Average Score
59
NIAH
100Accuracy
50
NIAH seq=4096
100Accuracy
44
NIAH (M)
90.2Accuracy (NIAH M)
35
NIAH-Single-3 256K context
94Accuracy
23
NIAH-Single-3 128K context
98.8Accuracy
23
NIAH-Single-3 64K context
100Accuracy
23
NIAH-Single-2 32K context
100Accuracy
23
NIAH-Single-1 256K context
100Accuracy
23
NIAH-Single-1 128K context
100Accuracy
23
NIAH-Single-1 64K context
100Accuracy
23
NIAH Single 3
100Accuracy (1024)
22
NIAH 128k
24.4Single Score
20
NIAH 64k
49.3Single Score
20
NIAH multivalue
4.1Speedup
20
NIAH
100NIAH Score
14
NIAH Single 64K context 2
100Accuracy
13
NIAH-Multi
100Accuracy
13
NIAH-Single-3 128K context
90.6Accuracy
12
NIAH-Single-3 64K context
97Accuracy
12
NIAH Single 3 32K context
98.6Accuracy
12
NIAH-Single-2 256K context
93.4Accuracy
12
NIAH-Single-1 256K
100Accuracy
12
NIAH-Single-1 128K
100Accuracy
12
NIAH-Single-1 32K
100Accuracy
12