Quantization Distribution Evaluation on C4 (calibration set)
0KL Divergence (Top 10)BF16 (baseline)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| BF16 (baseline)Precision=BF162026.05 | 0 | 100 | 0 | |
| PO2(Split87)Quantization=W4A42026.05 | 0.0364 | 92.35 | 0.0464 | |
| PO2(NF4)Quantization=W4A42026.05 | 0.0403 | 91.96 | 0.0514 | |
| Split87Quantization=W4A42026.05 | 0.0413 | 91.86 | 0.0524 | |
| BOF4Quantization=W4A42026.05 | 0.0446 | 91.55 | 0.0568 | |
| IF4Quantization=W4A42026.05 | 0.0456 | 91.42 | 0.0585 | |
| NF4Quantization=W4A42026.05 | 0.046 | 91.41 | 0.0589 | |
| SFP4Quantization=W4A42026.05 | 0.0488 | 91.18 | 0.0621 | |
| 4over6Quantization=W4A42026.05 | 0.0491 | 91.16 | 0.0628 | |
| NVFP4Quantization=W4A42026.05 | 0.0543 | 90.67 | 0.0696 | |
| NVINT4Quantization=W4A42026.05 | 0.0702 | 89.25 | 0.0919 |