Large Language Model Inference on Qwen3-0.6B (inference)
0.255Storage (GB)EDGERAZOR
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| EDGERAZORW-A-KV=1.58-8-8, Weight Types=TQ1_0, KV Types=Q8_02026.04 | 0.255 | 0.49 | 665.92 | 292.05 | |
| EDGERAZORW-A-KV=1.58-8-8, Weight Types=TQ2_0, KV Types=Q8_02026.04 | 0.275 | 0.509 | 659.63 | 325.19 | |
| Llama.cpp PTQW-A-KV=2-8-8, Weight Types=Q2_K, KV Types=Q8_02026.04 | 0.323 | 0.639 | 704.7 | 224.8 | |
| EDGERAZORW-A-KV=4-8-8, Weight Types=Q4_0, KV Types=Q8_02026.04 | 0.437 | 0.751 | 1,275.25 | 270.25 | |
| Llama.cpp PTQW-A-KV=4-8-8, Weight Types=Q4_K, KV Types=Q8_02026.04 | 0.451 | 0.767 | 717.05 | 233.47 | |
| Qwen3-0.6BW-A-KV=16-16-16, Weight Types=BF16, KV Types=BF162026.04 | 1.406 | 1.747 | 335.91 | 21.6 |