ResearchBenchmarksLatency profiling on Llama-3.1-8BLatency profiling on Llama-3.1-8BFollow7.3TPOTUniQL6.59211.37116.1520.929Dec 3, 2025Evaluation resultsMethodMethodLinksTPOTTTLTUniQLW-bit=4, Prun. p%=35%W-bit=4, Prun. p%=35%2025.127.38,105.4UniQLW-bit=4, Prun. p%=0%W-bit=4, Prun. p%=0%2025.1299,944.6TAO-HQQW-bit=4*, Prun. p%=0%W-bit=4*, Prun. p%=0%2025.1210.211,639.5TRT-AWQW-bit=4*, Prun. p%=0%W-bit=4*, Prun. p%=0%2025.1211.211,665.7FP16W-bit=16, Prun. p%=0%W-bit=16, Prun. p%=0%2025.122526,653.8