List your company
ResearchFormerly WizWand

Decoding on DGX A100 platform

359.3Throughput (tokens/sec)LAQuant
61.34138.695216.05293.405May 9, 2026
Updated 5mo ago

Evaluation results

MethodLinks
LAQuantModel=Qwen3-1.7B, Bits=3, Batch size=12026.05359.31.73
LAQuantModel=Llama3.1-8B, Bits=3, Batch size=12026.05294.373.93
ParoQuantModel=Qwen3-1.7B, Bits=3, Batch size=12026.05285.41.37
LAQuantModel=Qwen3-4B, Bits=3, Batch size=12026.05274.92.42
LAQuantModel=Qwen3-1.7B, Bits=4, Batch size=12026.05260.81.25
LAQuantModel=Qwen3-8B, Bits=3, Batch size=12026.052563.52
ParoQuantModel=Llama3.1-8B, Bits=3, Batch size=12026.05233.733.12
ParoQuantModel=Qwen3-1.7B, Bits=4, Batch size=12026.05217.41.05
ParoQuantModel=Qwen3-4B, Bits=3, Batch size=12026.05214.91.89
FP16Model=Qwen3-1.7B, Bits=16, Batch size=12026.05207.9—
ParoQuantModel=Qwen3-8B, Bits=3, Batch size=12026.05203.82.8
LAQuantModel=Qwen3-4B, Bits=4, Batch size=12026.05163.11.43
LAQuantModel=Llama3.1-8B, Bits=4, Batch size=12026.05143.211.91
ParoQuantModel=Qwen3-4B, Bits=4, Batch size=12026.05140.21.23
LAQuantModel=Qwen3-8B, Bits=4, Batch size=12026.05131.41.81
ParoQuantModel=Llama3.1-8B, Bits=4, Batch size=12026.05126.591.69
ParoQuantModel=Qwen3-8B, Bits=4, Batch size=12026.05116.41.6
FP16Model=Qwen3-4B, Bits=16, Batch size=12026.05113.8—
FP16Model=Llama3.1-8B, Bits=16, Batch size=12026.0574.93—
FP16Model=Qwen3-8B, Bits=16, Batch size=12026.0572.8—