RULER
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
RULER 16k context length
0FWE Score
21
RULER 64K context
84.3Accuracy
19
RULER 16K
89.28CWE Score
18
RULER 16K 1.0 (test)
89.28CWE Score
18
RULER 128K context
64Accuracy
18
RULER
92.88Performance @ 8K Context
18
RULER 64K
97.33Score
17
RULER QA
95.4Accuracy (32K Context)
17
RULER
96Accuracy
17
Ruler
93.2Accuracy
16
Ruler
2Average Rank
16
RULER 32k context (test)
91.7Accuracy
16
RULER 4k context (test)
95Accuracy
16
RULER 4k context length
0FWE Rate
16
RULER 1.0 (test)
0.977Accuracy (4K Context)
16
RULER 32K-HARD
88.74Average Accuracy
15
RULER 128K
88.94S-N Score
15
RULER 32K
89.4S-N Score
15
RULER 4K
99.54S-N Score
15
RULER 8K
99.67Score
15
RULER 32K
86.65RULER 32K Retrieval Score
15
RULER-Bench
68.2Average Score
14
RULER 1000 tokens
94.5NIAH
14
RULER 8K
70.98Accuracy
14
RULER
80.1Accuracy
14