ResearchBenchmarksInference Throughput on Llama-8BFollow115.2Throughput (Tokens/s)GPTQ45.83263.84181.8599.859Jan 29, 2026Evaluation ResultsMethodMethodLinksThroughput (Tokens/s)GPTQQuantization=W4A16, Ha...Quantization=W4A16, Hardware=NVIDIA A8002026.01115.2HeRo-QQuantization=W4A16, Ha...Quantization=W4A16, Hardware=NVIDIA A8002026.01113.1SpinQuantQuantization=W4A16, Ha...Quantization=W4A16, Hardware=NVIDIA A8002026.01112.8FP16Precision=FP16, Hardwa...Precision=FP16, Hardware=NVIDIA A8002026.0148.5