ResearchBenchmarksInference Throughput on Llama-3BFollow215.6Throughput (TOK/s)GPTQ114.512140.756167193.244Jan 29, 2026Evaluation ResultsMethodMethodLinksThroughput (TOK/s)GPTQQuantization=W4A16, Ha...Quantization=W4A16, Hardware=NVIDIA A8002026.01215.6HeRo-QQuantization=W4A16, Ha...Quantization=W4A16, Hardware=NVIDIA A8002026.01210SpinQuantQuantization=W4A16, Ha...Quantization=W4A16, Hardware=NVIDIA A8002026.01209.5FP16Precision=FP16, Hardwa...Precision=FP16, Hardware=NVIDIA A8002026.01118.4