Long-context Reasoning on LongReason Average across 16k-64k
54.2AccuracyFull
Evaluation Results
| Method | Links | |
|---|---|---|
| FullBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=100%, Prompting Strategy=w. CoT2026.06 | 54.2 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 52.53 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 51.13 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 51.09 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 50.76 | |
| FullBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=100%, Prompting Strategy=w/o. CoT2026.06 | 50.15 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 49.79 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 49.37 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 48.88 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 48.7 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 48.57 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 47.15 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 46.89 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 46.81 | |
| FullBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=100%, Prompting Strategy=w. CoT2026.06 | 46.55 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 46.25 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 45.83 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 45.7 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 45.68 | |
| FullBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=100%, Prompting Strategy=w/o. CoT2026.06 | 44.42 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 43.95 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 43.11 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 43.03 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 42.86 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 42.23 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 41.81 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 41.65 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 41.56 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 41.39 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 40.75 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 40.72 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 40.67 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 39.67 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 39.42 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 39.42 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 39.38 |