List your company
ResearchFormerly WizWand

Text Generation on Decoding Throughput

320Decoding Throughput (tokens/s)AWQ
13.292.85172.5252.15Nov 13, 2025
Updated 8mo ago

Evaluation results

MethodLinks
AWQModel=Qwen3-1.7B, Batch size=12025.11320
PAROQModel=Qwen3-1.7B, Batch size=12025.11278
QTIPModel=Qwen3-1.7B, Batch size=12025.11209
AWQModel=Qwen3-4B, Batch size=12025.11176
FP16Model=Qwen3-1.7B, Batch size=12025.11170
PAROQModel=Qwen3-4B, Batch size=12025.11160
AWQModel=LLaMA-3-8B, Batch size=12025.11120
QTIPModel=Qwen3-4B, Batch size=12025.11117
PAROQModel=LLaMA-3-8B, Batch size=12025.11112
QTIPModel=LLaMA-3-8B, Batch size=12025.1195
FP16Model=Qwen3-4B, Batch size=12025.1178
AWQModel=Qwen3-14B, Batch size=12025.1170
PAROQModel=Qwen3-14B, Batch size=12025.1165
QTIPModel=Qwen3-14B, Batch size=12025.1155
FP16Model=LLaMA-3-8B, Batch size=12025.1145
FP16Model=Qwen3-14B, Batch size=12025.1125