ResearchBenchmarksMatrix Multiplication Latency on Llama-3 8BMatrix Multiplication Latency on Llama-3 8BFollow152.69Kernel-level latency (µs)CodeGEMM132.9772266.0386399.1532.1614Dec 19, 2025Evaluation resultsMethodMethodLinksKernel-level latency (µs)CodeGEMMconfig=m1v4g128config=m1v4g1282025.12152.69LUTGEMMquantization=q2-g128quantization=q2-g1282025.12160.1QuIP#config=e8pconfig=e8p2025.12162.63CodeGEMMconfig=m2v8g128config=m2v8g1282025.12172.18QTIPconfig=r2config=r22025.12189.94AQLMconfig=2x8config=2x82025.12250.12cuBLASprecision=fp16precision=fp162025.12332.45AQLMconfig=1x16config=1x162025.12645.51