Linear Kernel Performance on 256x256x1 Matrix Multiplication NVIDIA H800 GPU (test)
5.54Latency (us)BWTA_QK
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BWTA_QKWbit/Abit=1.5/1.52026.04 | 5.54 | 5,915 | |
| BWTA_AttnWbit/Abit=1/1.52026.04 | 6.71 | 4,883 | |
| Binary A x V (Supported)Wbit/Abit=1/12026.04 | 6.97 | 4,701 | |
| Binary Linear (Supported)Wbit/Abit=1/12026.04 | 7.29 | 4,495 | |
| BWTAWbit/Abit=1/1.52026.04 | 8.56 | 3,828 | |
| bitlinear_int8xint2(Bitnet)Wbit/Abit=2/82026.04 | 9.08 | 3,609 | |
| torch.nn.functionalWbit/Abit=16/162026.04 | 10.74 | 3,050 | |
| bnb.nn.Linear4bitWbit/Abit=4/162026.04 | 60.33 | 543 |