Zero-shot Task Completion on HellaSwag 0-shot (N=500)
47.5AccuracyFull cache
Evaluation results
| Method | Links | |
|---|---|---|
| Full cacheModel=Qwen2.5-1.5B2026.03 | 47.5 | |
| KV-DirectModel=Qwen2.5-1.5B2026.03 | 47.5 | |
| Window-64Model=Qwen2.5-1.5B2026.03 | 47.5 | |
| Full cacheModel=Qwen3-0.6B2026.03 | 44 | |
| KV-DirectModel=Qwen3-0.6B2026.03 | 44 | |
| Window-64Model=Qwen3-0.6B2026.03 | 44 | |
| Full cacheModel=DS-R1-1.5B2026.03 | 42 | |
| KV-DirectModel=DS-R1-1.5B2026.03 | 42 | |
| Window-64Model=DS-R1-1.5B2026.03 | 42 | |
| Full cacheModel=Qwen2.5-0.5B2026.03 | 41 | |
| KV-DirectModel=Qwen2.5-0.5B2026.03 | 41 | |
| Window-64Model=Qwen2.5-0.5B2026.03 | 41 | |
| Full cacheModel=SmolLM2-135M2026.03 | 39.4 | |
| KV-DirectModel=SmolLM2-135M2026.03 | 39.4 | |
| Window-64Model=SmolLM2-135M2026.03 | 39.4 |