Long-context Reasoning on LongReason 16k prefill length 1.0 (test)
55.67AccuracyFull
Evaluation Results
| Method | Links | |
|---|---|---|
| FullBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=100%, Prompting Strategy=w. CoT2026.06 | 55.67 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 55.32 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 54.16 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 53.67 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 53.15 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 52.39 | |
| FullBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=100%, Prompting Strategy=w/o. CoT2026.06 | 52.08 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 51.8 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 51.77 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 50.88 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 50.88 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 50.13 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 49.12 | |
| InfoKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 48.74 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 48.36 | |
| FullBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=100%, Prompting Strategy=w. CoT2026.06 | 48.23 | |
| PyramidKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 47.36 | |
| SnapKVBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 47.23 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 46.47 | |
| ExpectedBackbone Model=Llama-3.1-8B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 45.72 | |
| FullBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=100%, Prompting Strategy=w/o. CoT2026.06 | 45.59 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 45.34 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 45.09 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w. CoT2026.06 | 44.71 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 43.83 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 43.42 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 43.2 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 43.19 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 42.95 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 42.82 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=40%, Prompting Strategy=w/o. CoT2026.06 | 42.44 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w. CoT2026.06 | 42.44 | |
| SnapKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 41.06 | |
| ExpectedBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 40.55 | |
| InfoKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 40.55 | |
| PyramidKVBackbone Model=Llama-3.2-3B-Instruct, KV Cache Rate=20%, Prompting Strategy=w/o. CoT2026.06 | 40.18 |