ResearchBenchmarksInference Throughput on Llama-1BFollow310.5Throughput (Tokens/sec)GPTQ200.988229.419257.85286.281Jan 29, 2026Evaluation ResultsMethodMethodLinksThroughput (Tokens/sec)GPTQQuantization=W4A16, Ha...Quantization=W4A16, Hardware=NVIDIA A8002026.01310.5HeRo-QQuantization=W4A16, Ha...Quantization=W4A16, Hardware=NVIDIA A8002026.01303.1SpinQuantQuantization=W4A16, Ha...Quantization=W4A16, Hardware=NVIDIA A8002026.01302.8FP16Precision=FP16, Hardwa...Precision=FP16, Hardware=NVIDIA A8002026.01205.2