Long-context Reasoning on LongReason 32k prefill length
53.9AccuracyFull
Evaluation Results
| Method | Links | |
|---|---|---|
| FullBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=100%, Prompting Strategy=w. CoT2026.06 | 53.9 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 52.39 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 51.13 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 51.01 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 50.5 | |
| FullBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=100%, Prompting Strategy=w/o. CoT2026.06 | 49.75 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 49.5 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 48.61 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 48.36 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 47.98 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 47.74 | |
| FullBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=100%, Prompting Strategy=w. CoT2026.06 | 46.47 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 46.35 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 46.35 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 46.22 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 45.97 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 45.72 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 45.59 | |
| FullBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=100%, Prompting Strategy=w/o. CoT2026.06 | 44.96 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 44.84 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 43.82 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 43.07 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 43.07 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 42.81 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 42.57 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 42.44 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 42.07 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 41.69 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 41.69 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 40.93 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 40.43 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 40.42 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 40.3 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 40.05 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 39.92 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 39.67 |