Weight Compression Latency on 4096x4096 Weight Tensor
0.69Encoding Latency (s)GPTQ
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPTQHardware=NVIDIA RTX 6000 Ada, Target Rate=4 bits per parameter, Tensor Size=4096 x 40962025.10 | 0.69 | 0.08 | — | — | |
| NWCHardware=NVIDIA RTX 6000 Ada, Target Rate=4 bits per parameter, Tensor Size=4096 x 40962025.10 | 1.64 | 1.17 | 1.07 | 0.1 | |
| QTIPHardware=NVIDIA RTX 6000 Ada, Target Rate=4 bits per parameter, Tensor Size=4096 x 40962025.10 | 19.84 | 0.52 | — | — | |
| QuIP#Hardware=NVIDIA RTX 6000 Ada, Target Rate=4 bits per parameter, Tensor Size=4096 x 40962025.10 | 21.71 | 1.33 | — | — |