List your company
ResearchFormerly WizWand

Evaluation Benchmarks Zero-shot

71.76Average AccuracyDense
40.164848.367456.5764.7726May 23, 2025Jul 24, 2025Sep 25, 2025Nov 26, 2025Jan 28, 2026Mar 31, 2026Jun 2, 2026
Updated 4mo ago

Evaluation results

MethodLinks
DenseModel=LLaMA2-13B, PR=0%, Tokens/s=1066, TI=1.00×, Latency=396.9, Speedup=1.00×2025.0571.76
ParetoQTime (hours)=1777, GPU Mem. (GB)=82.4, Base Model=LLaMA-3.1-8B, Quantization=approx 2-bit2026.0570.9
WINDQuantTime (hours)=49, GPU Mem. (GB)=17.1, Base Model=LLaMA-3.1-8B, Quantization=approx 2-bit2026.0570.2
BF16 (dense)Model=Qwen3-8B2026.0569.08
DenseModel=LLaMA-3-8B, Pruning depth (Lp)=7-layer2026.0567.51
ADMM-QQuantization Scheme=SmoothQuant, Weight Bit-width=4, Activation Bit-width=8, Weight Quantization Granularity=per-channel2026.0566.1
GPTQQuantization Scheme=SmoothQuant, Weight Bit-width=4, Activation Bit-width=8, Weight Quantization Granularity=per-channel2026.0565.74
TRSPModel=LLaMA2-13B, PR=25%, Tokens/s=1386, TI=1.30×, Latency=298.4, Speedup=1.33×2025.0565.11
DenseModel=OPT-13B, PR=0%, Tokens/s=1029, TI=1.00×, Latency=386.5, Speedup=1.00×2025.0561.79
TRSPModel=OPT-13B, PR=25%, Tokens/s=1348, TI=1.31×, Latency=286.3, Speedup=1.35×2025.0560.84
LLM-Streamline + Ghost LayerModel=LLaMA-3-8B, Pruning depth (Lp)=7-layer2026.0560.1
FloatModel=SmolLM2 1.7B, Average Bit-Width=BF162026.0659.52
ShortGPT + Ghost LayerModel=LLaMA-3-8B, Pruning depth (Lp)=7-layer2026.0559.33
MXFP8Model=SmolLM2 1.7B, Average Bit-Width=8.02026.0658.69
ADMM-QQuantization Scheme=SpinQuant, Weight Bit-width=4, Activation Bit-width=4, KV Cache Bit-width=4, Weight Quantization Granularity=per-channel2026.0557.96
dMXModel=SmolLM2 1.7B, Average Bit-Width=7.292026.0657.9
GPTQQuantization Scheme=SpinQuant, Weight Bit-width=4, Activation Bit-width=4, KV Cache Bit-width=4, Weight Quantization Granularity=per-channel2026.0557.67
TRSPModel=LLaMA2-13B, PR=50%, Tokens/s=1823, TI=1.71×, Latency=216.9, Speedup=1.83×2025.0557.57
dMXModel=SmolLM2 1.7B, Average Bit-Width=6.062026.0657.28
NeUQITokens=∼ 0.5M×1, Backbone=LLaMA 2 7B, Quantization bits=2-bit, Group size=128, Method Variant=lightweight distillation2025.0556.77
dMXModel=SmolLM2 1.7B, Average Bit-Width=5.092026.0655.36
GPTQQuantization Scheme=SmoothQuant, Weight Bit-width=3, Activation Bit-width=8, Weight Quantization Granularity=per-channel2026.0554.91
ADMM-QQuantization Scheme=SpinQuant, Weight Bit-width=3, Activation Bit-width=4, KV Cache Bit-width=4, Weight Quantization Granularity=per-channel2026.0554.7
ADMM-QQuantization Scheme=SmoothQuant, Weight Bit-width=3, Activation Bit-width=8, Weight Quantization Granularity=per-channel2026.0554.64
dMXModel=SmolLM2 1.7B, Average Bit-Width=4.532026.0654.41
FloatModel=Qwen3 1.7B, Average Bit-Width=BF162026.0654.12
dMXModel=Qwen3 1.7B, Average Bit-Width=7.652026.0654.09
MXFP8Model=Qwen3 1.7B, Average Bit-Width=8.02026.0654.08
ShortGPT + Ghost LayerModel=LLaMA-3.1-8B, Pruning depth (Lp)=11-layer2026.0554.02
GPTQQuantization Scheme=SpinQuant, Weight Bit-width=3, Activation Bit-width=4, KV Cache Bit-width=4, Weight Quantization Granularity=per-channel2026.0553.97
LLM-Streamline + Ghost LayerModel=LLaMA-3.1-8B, Pruning depth (Lp)=11-layer2026.0553.92
dMXModel=Qwen3 1.7B, Average Bit-Width=5.902026.0653.74
Shortened LLaMA + Ghost LayerModel=LLaMA-3-8B, Pruning depth (Lp)=7-layer2026.0553.44
MXFP4Model=SmolLM2 1.7B, Average Bit-Width=4.02026.0653.42
dMXModel=Qwen3 1.7B, Average Bit-Width=4.532026.0652.2
PV-tuningTokens=∼ 1B×1, Backbone=LLaMA 2 7B, Quantization bits=2-bit, Group size=1282025.0552.17
dMXModel=Qwen3 1.7B, Average Bit-Width=5.022026.0651.62
FloatModel=Llama 3.2 1B, Average Bit-Width=BF162026.0651.53
dMXModel=Llama 3.2 1B, Average Bit-Width=8.02026.0651.46
MXFP8Model=Llama 3.2 1B, Average Bit-Width=8.02026.0651.39
MXFP4Model=Qwen3 1.7B, Average Bit-Width=4.02026.0651.15
TRSPModel=OPT-13B, PR=50%, Tokens/s=1801, TI=1.75×, Latency=208.9, Speedup=1.85×2025.0550.83
dMXModel=Llama 3.2 1B, Average Bit-Width=6.042026.0649.61
dMXModel=Llama 3.2 1B, Average Bit-Width=5.112026.0649.27
dMXModel=Llama 3.2 1B, Average Bit-Width=4.572026.0648.11
GPTQTokens=∼ 0.5M×1, Backbone=LLaMA 2 7B, Quantization bits=2-bit, Group size=128, Method Variant=lightweight distillation2025.0548.05
MXFP4Model=Llama 3.2 1B, Average Bit-Width=4.02026.0647.82
OmniQuantTokens=∼0.25M×20, Backbone=LLaMA 2 7B, Quantization bits=2-bit, Group size=1282025.0547.59
ADMM-QQuantization Scheme=SpinQuant, Weight Bit-width=2, Activation Bit-width=4, KV Cache Bit-width=4, Weight Quantization Granularity=per-channel2026.0546.29
ADMM-QQuantization Scheme=SmoothQuant, Weight Bit-width=2, Activation Bit-width=8, Weight Quantization Granularity=per-channel2026.0546.13
EfficientQATTime (hours)=14, GPU Mem. (GB)=8.1, Base Model=LLaMA-3.1-8B, Quantization=approx 2-bit2026.0544.8
GPTQQuantization Scheme=SpinQuant, Weight Bit-width=2, Activation Bit-width=4, KV Cache Bit-width=4, Weight Quantization Granularity=per-channel2026.0543.28
Shortened LLaMA + Ghost LayerModel=LLaMA-3.1-8B, Pruning depth (Lp)=11-layer2026.0542.31
LLM-QATTime (hours)=0.5, GPU Mem. (GB)=126.9, Base Model=LLaMA-3.1-8B, Quantization=approx 2-bit2026.0542
GPTQQuantization Scheme=SmoothQuant, Weight Bit-width=2, Activation Bit-width=8, Weight Quantization Granularity=per-channel2026.0541.38