List your company
ResearchFormerly WizWand

Decoding Performance on RTX 4090 platform

426Throughput (tokens/sec)LAQuant
40.16140.33240.5340.67May 9, 2026
Updated 5mo ago

Evaluation results

MethodLinks
LAQuantModel=Qwen3-1.7B, Bits=3, Batch size=12026.054262.27
ParoQuantModel=Qwen3-1.7B, Bits=3, Batch size=12026.05365.81.95
LAQuantModel=Llama3.1-8B, Bits=3, Batch size=12026.05322.115.72
LAQuantModel=Qwen3-4B, Bits=3, Batch size=12026.05322.13.48
LAQuantModel=Qwen3-1.7B, Bits=4, Batch size=12026.05302.11.61
ParoQuantModel=Llama3.1-8B, Bits=3, Batch size=12026.05282.945.3
ParoQuantModel=Qwen3-4B, Bits=3, Batch size=12026.05279.63.02
LAQuantModel=Qwen3-8B, Bits=3, Batch size=12026.05276.65.03
ParoQuantModel=Qwen3-1.7B, Bits=4, Batch size=12026.05271.61.45
ParoQuantModel=Qwen3-8B, Bits=3, Batch size=12026.05243.24.42
FP16Model=Qwen3-1.7B, Bits=16, Batch size=12026.05187.3—
LAQuantModel=Qwen3-4B, Bits=4, Batch size=12026.05183.51.99
ParoQuantModel=Qwen3-4B, Bits=4, Batch size=12026.05168.41.82
LAQuantModel=Llama3.1-8B, Bits=4, Batch size=12026.05129.712.3
ParoQuantModel=Llama3.1-8B, Bits=4, Batch size=12026.05122.662.18
LAQuantModel=Qwen3-8B, Bits=4, Batch size=12026.05121.82.21
ParoQuantModel=Qwen3-8B, Bits=4, Batch size=12026.05114.62.08
FP16Model=Qwen3-4B, Bits=16, Batch size=12026.0592.5—
FP16Model=Llama3.1-8B, Bits=16, Batch size=12026.0556.3—
FP16Model=Qwen3-8B, Bits=16, Batch size=12026.0555—