Attention Quantization Error Analysis on Flash Attention (seq=16384, head_dim=64)
0.49L2 Relative ErrorFlash+MSD
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Flash+MSDSequence length (seq)=16384, Head dimension (head_dim)=64, KV Quantization type=INT8 KV with per-channel scale2026.05 | 0.49 | 89.4 | 45.9 | 22.1 | 4.1 | |
| FlashSequence length (seq)=16384, Head dimension (head_dim)=64, KV Quantization type=INT8 KV with per-channel scale, mode=BF162026.05 | 1.38 | 97.7 | 87.2 | 66.7 | 8.6 | |
| DequantSequence length (seq)=16384, Head dimension (head_dim)=64, KV Quantization type=INT8 KV with per-channel scale, mode=BF162026.05 | 1.41 | 97.7 | 87.4 | 67.5 | 8.9 |