Long-context reasoning on LongGenBench 8K
44.51GSM8K ScoreFull Cache
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Full CacheBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=Full Cache2026.05 | 44.51 | 50.92 | 64.81 | 53.41 | |
| SCOPEBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 44.2 | 49 | 66.16 | 53.12 | |
| Moment-KVBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 40.31 | 50 | 66.33 | 52.21 | |
| StreamingLLMBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 39.36 | 48.29 | 63.83 | 50.49 | |
| H2OBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 38.29 | 48.02 | 64.9 | 50.4 | |
| Moment-KVBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 35.63 | 49.92 | 63.33 | 49.63 | |
| PyramidInferBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=1024 Tokens2026.05 | 34.28 | 45 | 57.16 | 45.48 | |
| StreamingLLMBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 33.88 | 48.53 | 64.25 | 48.89 | |
| SCOPEBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 27.85 | 47.39 | 63.25 | 46.16 | |
| H2OBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 25.07 | 44.38 | 47.16 | 38.87 | |
| PyramidInferBase Model=LLaMA-3.1-8B-Instruct, Decoding Budget=512 Tokens2026.05 | 24.68 | 42.72 | 52 | 39.8 | |
| Full CacheBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=Full Cache2026.05 | 9.45 | 20.28 | 51.63 | 27.12 | |
| Moment-KVBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 8.8 | 19.63 | 53.33 | 27.25 | |
| Moment-KVBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 7.61 | 19.62 | 50.08 | 25.77 | |
| PyramidInferBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 7.22 | 19.51 | 42.91 | 23.21 | |
| StreamingLLMBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 6.58 | 21.08 | 46.5 | 24.72 | |
| SCOPEBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 6.58 | 16.44 | 51.75 | 24.92 | |
| H2OBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 6.03 | 20.22 | 48.41 | 24.89 | |
| PyramidInferBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 5.87 | 17.86 | 39.83 | 21.19 | |
| SCOPEBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=1024 Tokens2026.05 | 5.79 | 18 | 53.25 | 25.68 | |
| H2OBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 5.63 | 18.31 | 43.25 | 22.4 | |
| StreamingLLMBase Model=Mistral-7B-Instruct-v0.3, Decoding Budget=512 Tokens2026.05 | 4.68 | 18.64 | 53.58 | 25.63 |