KV-Cache Footprint Analysis on Sequence length T=2048 BF16 (inference)
67KV Cache Size (MB)Wiola-360m
Evaluation Results
| Method | Links | |
|---|---|---|
| Wiola-360mParams=361M2026.07 | 67 | |
| OPT-350MParams=350M2026.07 | 168 | |
| Pythia-410MParams=410M2026.07 | 192 | |
| GPT-2 XLParams=1.5B2026.07 | 421 |