Long-context Reasoning on LongGenBench 4K
53.18GSM8K ScoreFull Cache
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Full CacheBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=Full Cache2026.05 | 53.18 | 54.27 | 71.54 | 59.63 | |
| Moment-KVBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 52.86 | 55.12 | 72.83 | 60.27 | |
| SCOPEBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 52.01 | 52.82 | 71.91 | 58.91 | |
| StreamingLLMBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 50.93 | 54.84 | 71.91 | 59.23 | |
| H2OBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 50.38 | 53.81 | 69.66 | 57.95 | |
| Moment-KVBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 50.38 | 55.63 | 73.66 | 59.89 | |
| SCOPEBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 48.68 | 54.78 | 72.41 | 58.62 | |
| PyramidInferBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 46.2 | 53.6 | 69.58 | 56.46 | |
| StreamingLLMBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 44.49 | 55.57 | 71.83 | 57.3 | |
| H2OBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 38.75 | 51.42 | 64.58 | 51.58 | |
| PyramidInferBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 37.1 | 51.8 | 63.5 | 50.8 | |
| Moment-KVBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 13.41 | 28.45 | 66.04 | 35.97 | |
| Moment-KVBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 12.63 | 28.57 | 65.84 | 35.68 | |
| Full CacheBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=Full Cache2026.05 | 11.94 | 28.17 | 64.21 | 34.77 | |
| StreamingLLMBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 11.93 | 27.06 | 64.33 | 34.44 | |
| SCOPEBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 11.39 | 25.09 | 64.75 | 33.74 | |
| H2OBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 11.24 | 27.81 | 63.91 | 34.32 | |
| PyramidInferBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 11.16 | 28.24 | 64 | 34.47 | |
| SCOPEBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 11.16 | 24.03 | 65.08 | 33.42 | |
| H2OBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 9.92 | 26.87 | 61.2 | 32.66 | |
| PyramidInferBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 9.92 | 26.87 | 61.25 | 32.68 | |
| StreamingLLMBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 9.22 | 28.51 | 65.41 | 34.38 |