Memory Footprint Analysis on Qwen2.5-3B-Instruct K3V3 deployment
155.7KV Cache Size (MB)TQ-MSE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TQ-MSET (Context Length)=16K2026.06 | 155.7 | 3.88 | 7.94 | 1.77 | |
| Block-GTQT (Context Length)=16K2026.06 | 186.4 | 3.24 | 7.97 | 1.8 | |
| FP16 FA-2T (Context Length)=16K2026.06 | 604 | 1 | 12.53 | 6.36 | |
| TQ-MSET (Context Length)=64K2026.06 | 622.9 | 3.88 | 12.94 | 6.77 | |
| Block-GTQT (Context Length)=64K2026.06 | 745.5 | 3.24 | 13.06 | 6.89 | |
| TQ-MSET (Context Length)=128K2026.06 | 1,245.7 | 3.88 | 19.6 | 13.43 | |
| Block-GTQT (Context Length)=128K2026.06 | 1,491.1 | 3.24 | 19.85 | 13.67 | |
| FP16 FA-2T (Context Length)=64K2026.06 | 2,415.9 | 1 | 31.29 | 25.12 | |
| TQ-MSET (Context Length)=256K2026.06 | 2,491.4 | 3.88 | 32.93 | 26.76 | |
| Block-GTQT (Context Length)=256K2026.06 | 2,982.2 | 3.24 | 33.42 | 27.25 | |
| FP16 FA-2T (Context Length)=128K2026.06 | 4,831.8 | 1 | 56.31 | 50.13 | |
| TQ-MSET (Context Length)=512K2026.06 | 4,982.8 | 3.88 | 59.58 | 53.41 | |
| Block-GTQT (Context Length)=512K2026.06 | 5,964.3 | 3.24 | 60.56 | 54.39 |