List your company
ResearchFormerly WizWand

Decoding on RTX 6000 Ada platform

468.7Throughput (tokens/sec)LAQuant
35.852148.226260.6372.974May 9, 2026
Updated 5mo ago

Evaluation results

MethodLinks
LAQuantModel=Qwen3-1.7B, Bits=3, Batch size=12026.05468.72.5
ParoQuantModel=Qwen3-1.7B, Bits=3, Batch size=12026.05392.92.1
LAQuantModel=Llama3.1-8B, Bits=3, Batch size=12026.05345.476.45
LAQuantModel=Qwen3-4B, Bits=3, Batch size=12026.053253.58
LAQuantModel=Qwen3-1.7B, Bits=4, Batch size=12026.05313.91.68
ParoQuantModel=Llama3.1-8B, Bits=3, Batch size=12026.05298.155.57
ParoQuantModel=Qwen3-1.7B, Bits=4, Batch size=12026.05279.71.49
ParoQuantModel=Qwen3-4B, Bits=3, Batch size=12026.05278.23.06
LAQuantModel=Qwen3-8B, Bits=3, Batch size=12026.05276.95.27
ParoQuantModel=Qwen3-8B, Bits=3, Batch size=12026.05241.34.59
LAQuantModel=Qwen3-4B, Bits=4, Batch size=12026.05191.42.11
FP16Model=Qwen3-1.7B, Bits=16, Batch size=12026.05187.1—
ParoQuantModel=Qwen3-4B, Bits=4, Batch size=12026.05174.41.92
LAQuantModel=Llama3.1-8B, Bits=4, Batch size=12026.05129.262.41
LAQuantModel=Qwen3-8B, Bits=4, Batch size=12026.05122.52.33
ParoQuantModel=Llama3.1-8B, Bits=4, Batch size=12026.05120.932.26
ParoQuantModel=Qwen3-8B, Bits=4, Batch size=12026.05112.62.14
FP16Model=Qwen3-4B, Bits=16, Batch size=12026.0590.8—
FP16Model=Llama3.1-8B, Bits=16, Batch size=12026.0553.55—
FP16Model=Qwen3-8B, Bits=16, Batch size=12026.0552.5—