List your company
ResearchFormerly WizWand

Science Question Answering on SciQ

97.7Normalized AccuracyHPTQ
84.90888.22991.5594.871Dec 13, 2024Feb 24, 2025May 8, 2025Jul 20, 2025Oct 1, 2025Dec 13, 2025Feb 25, 2026
Updated 7mo ago

Evaluation results

MethodLinks
HPTQAvg. Bitwidth=4.125, Zero-shot=true2025.0797.7
BF16 BaselineAvg. Bitwidth=16.000, Zero-shot=true2025.0797.3
GPTQAvg. Bitwidth=4.125, Zero-shot=true2025.0797.2
SSQR-2%Avg. Bitwidth=4.765, Zero-shot=true2025.0797.2
SSQR-5%Avg. Bitwidth=5.725, Zero-shot=true2025.0797.2
Llama 3-8B E8T2Shots=0-shot2024.1297
SSQR-5%Avg. Bitwidth=4.725, Zero-shot=true2025.0797
HPTQAvg. Bitwidth=3.125, Zero-shot=true2025.0796.9
SSQR-1%Avg. Bitwidth=4.445, Zero-shot=true2025.0796.9
SSQR-3%Avg. Bitwidth=5.085, Zero-shot=true2025.0796.9
SSQR-4%Avg. Bitwidth=5.405, Zero-shot=true2025.0796.9
SSQR-3%Avg. Bitwidth=4.085, Zero-shot=true2025.0796.6
SSQR-4%Avg. Bitwidth=4.405, Zero-shot=true2025.0796.5
SSQR-2%Avg. Bitwidth=3.765, Zero-shot=true2025.0796.4
SSQR-1%Avg. Bitwidth=3.445, Zero-shot=true2025.0796.3
BF16 BaselineAvg. Bitwidth=16.000, Backbone=Qwen3-8B2025.0795.7
SSQR-1%Avg. Bitwidth=4.445, Zero-shot=true2025.0795.7
HPTQAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.0795.6
SSQR-3%Avg. Bitwidth=5.085, Backbone=Qwen3-8B2025.0795.6
SSQR-1%Avg. Bitwidth=3.445, Zero-shot=true2025.0795.6
GPTQAvg. Bitwidth=3.125, Zero-shot=true2025.0795.5
SSQR-1%Avg. Bitwidth=3.445, Backbone=Qwen3-8B2025.0795.5
SSQR-2%Avg. Bitwidth=4.765, Backbone=Qwen3-8B2025.0795.5
SSQR-5%Avg. Bitwidth=4.725, Backbone=Qwen3-8B2025.0795.5
HPTQAvg. Bitwidth=3.125, Zero-shot=true2025.0795.5
GPTQAvg. Bitwidth=4.125, Zero-shot=true2025.0795.5
SSQR-4%Avg. Bitwidth=5.405, Zero-shot=true2025.0795.5
HPTQAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.0795.4
BF16 BaselineAvg. Bitwidth=16.000, Zero-shot=true2025.0795.4
SSQR-3%Avg. Bitwidth=5.085, Zero-shot=true2025.0795.4
GPTQAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.0795.3
SSQR-4%Avg. Bitwidth=4.405, Zero-shot=true2025.0795.3
SSQR-5%Avg. Bitwidth=5.725, Zero-shot=true2025.0795.3
SSQR-5%Avg. Bitwidth=4.725, Zero-shot=true2025.0795.3
Qwen2.5-7BModel Type=Dense, Active Parameters=7B, Total Parameters=7B2026.0295.2
Qwen2.5-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B2026.0295.2
SSQR-1%Avg. Bitwidth=4.445, Backbone=Qwen3-8B2025.0795.2
SSQR-5%Avg. Bitwidth=5.725, Backbone=Qwen3-8B2025.0795.2
SSQR-2%Avg. Bitwidth=4.765, Zero-shot=true2025.0795.2
SSQR-2%Avg. Bitwidth=3.765, Backbone=Qwen3-8B2025.0795.1
SSQR-4%Avg. Bitwidth=5.405, Backbone=Qwen3-8B2025.0795.1
HPTQAvg. Bitwidth=4.125, Zero-shot=true2025.0795
SSQR-3%Avg. Bitwidth=4.085, Zero-shot=true2025.0795
OLMoE-1B-7BModel Type=MoE, Active Parameters=1B, Total Parameters=7B2026.0294.9
SSQR-4%Avg. Bitwidth=4.405, Backbone=Qwen3-8B2025.0794.9
RTNAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.0794.5
SSQR-2%Avg. Bitwidth=3.765, Zero-shot=true2025.0794.4
SSQR-3%Avg. Bitwidth=4.085, Backbone=Qwen3-8B2025.0794.3
HRTNAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.0794.2
GPTQAvg. Bitwidth=3.125, Zero-shot=true2025.0794
Llama 3-8BShots=0-shot2024.1293.9
TSD-KDselection=Adaptive c%2026.0293.8
HRTNAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.0793.7
Qwen2.5-1.5BModel Type=Dense, Active Parameters=1.5B, Total Parameters=1.5B2026.0293.4
TSD-KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.0293
Llama-3.2-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B2026.0292.7
ExpertWeaver-E64-A14-S2(OLMo-7B)Model Type=MoE, Active Parameters=1B, Total Parameters=7B, Training Budget=200B tokens2026.0291.8
SSQR-5%Avg. Bitwidth=3.725, Zero-shot=true2025.0791.5
HPTQAvg. Bitwidth=2.125, Zero-shot=true2025.0790.8
SSQR-5%Avg. Bitwidth=3.725, Backbone=Qwen3-8B2025.0790.8
SSQR-3%Avg. Bitwidth=3.085, Zero-shot=true2025.0790.5
SSQR-4%Avg. Bitwidth=3.405, Zero-shot=true2025.0790.1
HPTQAvg. Bitwidth=2.125, Zero-shot=true2025.0789.9
SSQR-4%Avg. Bitwidth=3.405, Backbone=Qwen3-8B2025.0789.5
LLaMA-MoE-v2-3.5BModel Type=MoE, Active Parameters=3.5B2026.0288.8
SSQR-5%Avg. Bitwidth=3.725, Zero-shot=true2025.0788.7
OLMo-7BModel Type=Dense, Active Parameters=7B, Total Parameters=7B2026.0288.6
OLMoE-1B-7B*Model Type=MoE, Active Parameters=1B, Total Parameters=7B, Training Budget=500B tokens2026.0288.5
Open-LLaMA-3B-v2Model Type=Dense, Active Parameters=3B, Total Parameters=3B2026.0288
PDPCSelection Metric=PD, Order=S-shape Function, Model Scale=1.3B, Training Tokens=100B2025.0187.9
Sequence-Level KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.0287.9
Sequence-Level KD2026.0287.9
Preference CLSelection Metric=PPL, Order=S-shape Reverse Function, Model Scale=1.3B, Training Tokens=100B2025.0187.8
Preference CLSelection Metric=Qu.Edu, Order=S-shape Function, Model Scale=1.3B, Training Tokens=100B2025.0187.8
ExpertWeaver-E64-A14-S2(Qwen2.5-7B)Model Type=MoE, Active Parameters=3.5B, Total Parameters=7B, Training Budget=200B tokens2026.0287.7
LLaMA-MoE-v1-3.5BModel Type=MoE, Active Parameters=3.5B2026.0287.6
Sheared-LLaMA-2.7BModel Type=Dense, Active Parameters=2.7B, Total Parameters=2.7B2026.0287.5
SequentialSelection Metric=PD, Order=High-to-Low, Model Scale=1.3B, Training Tokens=100B2025.0187.1
RTNAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.0787.1
HPTQAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.0786.8
Speculative KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.0286.7
Speculative KD2026.0286.7
Supervised-KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.0286.6
GKDbeta=0.9, Distillation Setup=Qwen2.5 (14B -> 1.5B)2026.0286.6
Supervised-KD2026.0286.6
GKDbeta=0.92026.0286.6
SSQR-3%Avg. Bitwidth=3.085, Backbone=Qwen3-8B2025.0786.4
Qwen2.5-14B (Teacher)Model Scale=14B2026.0286.4
Qwen2.5-14BRole=Teacher2026.0286.4
SequentialSelection Metric=PD, Order=Low-to-High, Model Scale=1.3B, Training Tokens=100B2025.0186.2
Preference CLSelection Metric=Qu.Edu, Order=S-shape Reverse Function, Model Scale=1.3B, Training Tokens=100B2025.0186.2
Preference CLSelection Metric=PD, Order=S-shape Reverse Function, Model Scale=1.3B, Training Tokens=100B2025.0186.2
RandomOrder=Random, Model Scale=1.3B, Training Tokens=100B2025.0185.8
DistiLLMDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.0285.7
DistiLLM2026.0285.7
INCITE-Base-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B2026.0285.6
Preference CLSelection Metric=PPL, Order=S-shape Function, Model Scale=1.3B, Training Tokens=100B2025.0185.5
Qwen2.5-1.5B (Student)Model Scale=1.5B2026.0285.5
Qwen2.5-1.5BRole=Student2026.0285.5
SequentialSelection Metric=Qu.Edu, Order=High-to-Low, Model Scale=1.3B, Training Tokens=100B2025.0185.4