LongReason
Benchmarks
TaskDatasetSOTA resultTrendResults
LongReason Average across 16k-64k
54.2Accuracy
36
LongReason 64k prefill length
53.02Accuracy
36
LongReason 32k prefill length
53.9Accuracy
36
LongReason 16k prefill length 1.0 (test)
55.67Accuracy
36
LongReason 64K-input 70K context
71.25Accuracy
34
LongReason
86.6Accuracy (32K Context)
21
LongReason
86.9Score
18
LongReason
84.13Accuracy (32k)
17
LongReason
72.3Acc
15
LongReason
84.89Accuracy
12
LongReason
80Peak Memory (GB)
6
LongReason (val)
79.3Accuracy (val)
4