List your company
ResearchFormerly WizWand

Large Language Model Inference on Prefill Phase SeqLen=2k

0.025Prefill Time (s)BWTA (Bitnet-b1.58-2B)
0.010880.106190.20150.29681Apr 5, 2026
Updated 6mo ago

Evaluation results

MethodLinks
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.35/6.05 (30% layers), Storage (GB)=1.796, BS=12026.040.02581,52910.1
Bitnet-b1.58-2BWbit/Abit=1.5/8, Storage (GB)=1.835, BS=12026.040.02677,25310.56
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.45/7.35 (10% layers), Storage (GB)=1.819, BS=12026.040.02677,22510.28
Gemma-2BWbit/Abit=16/16, Storage (GB)=5.012, BS=12026.040.06332,49220.75
Llama-3.2-3BWbit/Abit=4/16 (bnb), Storage (GB)=2.240, BS=12026.040.07228,26715.59
Gemma-2BWbit/Abit=4/16 (gptq), Storage (GB)=3.130, BS=12026.040.10719,13322.53
Gemma-2BWbit/Abit=16/16, Storage (GB)=5.012, BS=42026.040.11617,69464.51
Llama-3.2-3BWbit/Abit=4/16 (bnb), Storage (GB)=2.240, BS=42026.040.17211,92454.94
Gemma-2BWbit/Abit=4/16 (gptq), Storage (GB)=3.130, BS=42026.040.17311,85857.71
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.35/6.05 (30% layers), Storage (GB)=1.796, BS=82026.040.17911,46632.02
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.45/7.35 (10% layers), Storage (GB)=1.819, BS=82026.040.18511,09631.48
Bitnet-b1.58-2BWbit/Abit=1.5/8, Storage (GB)=1.835, BS=82026.040.18910,81431.67
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.35/6.05 (30% layers), Storage (GB)=1.796, BS=162026.040.3565,74556.94
BWTA (Bitnet-b1.58-2B)Wbit/Abit=1.45/7.35 (10% layers), Storage (GB)=1.819, BS=162026.040.3695,54455.41
Bitnet-b1.58-2BWbit/Abit=1.5/8, Storage (GB)=1.835, BS=162026.040.3785,41555.6