List your company
ResearchFormerly WizWand

Large Language Model Inference on Decode Phase BS=1

0.152Latency (s)BWTA (Bitnet-b1.58-2B)
0.01780.923651.82952.73535Apr 5, 2026
Updated 6mo ago

Evaluation results

MethodLinks
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.35/6.05 (30% layers), Storage (GB)=1.796, G. Len.=502026.040.152375.4
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.45/7.35 (10% layers), Storage (GB)=1.819, G. Len.=502026.040.156365.9
Bitnet-b1.58-2BWbit/Abit=1.5/8, Storage (GB)=1.835, G. Len.=502026.040.158360.2
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.35/6.05 (30% layers), Storage (GB)=1.796, G. Len.=1002026.040.285344.4
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.45/7.35 (10% layers), Storage (GB)=1.819, G. Len.=1002026.040.29338
Bitnet-b1.58-2BWbit/Abit=1.5/8, Storage (GB)=1.835, G. Len.=1002026.040.295331
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.35/6.05 (30% layers), Storage (GB)=1.796, G. Len.=1502026.040.443345.8
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.45/7.35 (10% layers), Storage (GB)=1.819, G. Len.=1502026.040.449338.4
Bitnet-b1.58-2BWbit/Abit=1.5/8, Storage (GB)=1.835, G. Len.=1502026.040.458333.9
Gemma-2BWbit/Abit=16/16, Storage (GB)=5.012, G. Len.=502026.040.512111.6
Gemma-2BWbit/Abit=4/16 (gptq), Storage (GB)=3.130, G. Len.=502026.040.98950.8
Gemma-2BWbit/Abit=16/16, Storage (GB)=5.012, G. Len.=1002026.041.14687.3
Llama-3.2-3BWbit/Abit=4/16 (bnb), Storage (GB)=2.240, G. Len.=502026.041.31743.4
Gemma-2BWbit/Abit=16/16, Storage (GB)=5.012, G. Len.=1502026.041.8683.3
Gemma-2BWbit/Abit=4/16 (gptq), Storage (GB)=3.130, G. Len.=1002026.041.95951
Llama-3.2-3BWbit/Abit=4/16 (bnb), Storage (GB)=2.240, G. Len.=1002026.042.56339
Gemma-2BWbit/Abit=4/16 (gptq), Storage (GB)=3.130, G. Len.=1502026.042.93252.9
Llama-3.2-3BWbit/Abit=4/16 (bnb), Storage (GB)=2.240, G. Len.=1502026.043.50744.2