Long-context Reasoning on LongReason 64k prefill length
53.02AccuracyFull
Evaluation Results
| Method | Links | |
|---|---|---|
| FullBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=100%, Prompting Strategy=w. CoT2026.06 | 53.02 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 49.87 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 48.99 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 48.74 | |
| FullBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=100%, Prompting Strategy=w/o. CoT2026.06 | 48.61 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 48.11 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 47.73 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 47.61 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 47.48 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 46.47 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 46.22 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 45.72 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 45.72 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 45.59 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 45.34 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 45.04 | |
| FullBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=100%, Prompting Strategy=w. CoT2026.06 | 44.96 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 44.28 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 43.53 | |
| FullBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=100%, Prompting Strategy=w/o. CoT2026.06 | 42.7 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 41.56 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 41.31 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 41.18 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 41.06 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 40.93 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 40.81 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 39.42 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 39.29 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 39.17 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 38.79 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 38.53 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 38.41 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 38.28 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 38.04 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 37.66 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 37.28 |