Science Question Answering on SciQ
97.7Normalized AccuracyHPTQ
Evaluation Results
| Method | Links | |
|---|---|---|
| HPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 97.7 | |
| BF16 BaselineAvg. Bitwidth=16.000, Zero-shot=true2025.07 | 97.3 | |
| GPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 97.2 | |
| SSQR-2%Avg. Bitwidth=4.765, Zero-shot=true2025.07 | 97.2 | |
| SSQR-5%Avg. Bitwidth=5.725, Zero-shot=true2025.07 | 97.2 | |
| Llama 3-8B E8T2Shots=0-shot2024.12 | 97 | |
| SSQR-5%Avg. Bitwidth=4.725, Zero-shot=true2025.07 | 97 | |
| HPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 96.9 | |
| SSQR-1%Avg. Bitwidth=4.445, Zero-shot=true2025.07 | 96.9 | |
| SSQR-3%Avg. Bitwidth=5.085, Zero-shot=true2025.07 | 96.9 | |
| SSQR-4%Avg. Bitwidth=5.405, Zero-shot=true2025.07 | 96.9 | |
| SSQR-3%Avg. Bitwidth=4.085, Zero-shot=true2025.07 | 96.6 | |
| SSQR-4%Avg. Bitwidth=4.405, Zero-shot=true2025.07 | 96.5 | |
| SSQR-2%Avg. Bitwidth=3.765, Zero-shot=true2025.07 | 96.4 | |
| SSQR-1%Avg. Bitwidth=3.445, Zero-shot=true2025.07 | 96.3 | |
| BF16 BaselineAvg. Bitwidth=16.000, Backbone=Qwen3-8B2025.07 | 95.7 | |
| SSQR-1%Avg. Bitwidth=4.445, Zero-shot=true2025.07 | 95.7 | |
| HPTQAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.07 | 95.6 | |
| SSQR-3%Avg. Bitwidth=5.085, Backbone=Qwen3-8B2025.07 | 95.6 | |
| SSQR-1%Avg. Bitwidth=3.445, Zero-shot=true2025.07 | 95.6 | |
| GPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 95.5 | |
| SSQR-1%Avg. Bitwidth=3.445, Backbone=Qwen3-8B2025.07 | 95.5 | |
| SSQR-2%Avg. Bitwidth=4.765, Backbone=Qwen3-8B2025.07 | 95.5 | |
| SSQR-5%Avg. Bitwidth=4.725, Backbone=Qwen3-8B2025.07 | 95.5 | |
| HPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 95.5 | |
| GPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 95.5 | |
| SSQR-4%Avg. Bitwidth=5.405, Zero-shot=true2025.07 | 95.5 | |
| HPTQAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.07 | 95.4 | |
| BF16 BaselineAvg. Bitwidth=16.000, Zero-shot=true2025.07 | 95.4 | |
| SSQR-3%Avg. Bitwidth=5.085, Zero-shot=true2025.07 | 95.4 | |
| GPTQAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.07 | 95.3 | |
| SSQR-4%Avg. Bitwidth=4.405, Zero-shot=true2025.07 | 95.3 | |
| SSQR-5%Avg. Bitwidth=5.725, Zero-shot=true2025.07 | 95.3 | |
| SSQR-5%Avg. Bitwidth=4.725, Zero-shot=true2025.07 | 95.3 | |
| Qwen2.5-7BModel Type=Dense, Active Parameters=7B, Total Parameters=7B2026.02 | 95.2 | |
| Qwen2.5-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B2026.02 | 95.2 | |
| SSQR-1%Avg. Bitwidth=4.445, Backbone=Qwen3-8B2025.07 | 95.2 | |
| SSQR-5%Avg. Bitwidth=5.725, Backbone=Qwen3-8B2025.07 | 95.2 | |
| SSQR-2%Avg. Bitwidth=4.765, Zero-shot=true2025.07 | 95.2 | |
| SSQR-2%Avg. Bitwidth=3.765, Backbone=Qwen3-8B2025.07 | 95.1 | |
| SSQR-4%Avg. Bitwidth=5.405, Backbone=Qwen3-8B2025.07 | 95.1 | |
| HPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 95 | |
| SSQR-3%Avg. Bitwidth=4.085, Zero-shot=true2025.07 | 95 | |
| OLMoE-1B-7BModel Type=MoE, Active Parameters=1B, Total Parameters=7B2026.02 | 94.9 | |
| SSQR-4%Avg. Bitwidth=4.405, Backbone=Qwen3-8B2025.07 | 94.9 | |
| RTNAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.07 | 94.5 | |
| SSQR-2%Avg. Bitwidth=3.765, Zero-shot=true2025.07 | 94.4 | |
| SSQR-3%Avg. Bitwidth=4.085, Backbone=Qwen3-8B2025.07 | 94.3 | |
| HRTNAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.07 | 94.2 | |
| GPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 94 | |
| Llama 3-8BShots=0-shot2024.12 | 93.9 | |
| TSD-KDselection=Adaptive c%2026.02 | 93.8 | |
| HRTNAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.07 | 93.7 | |
| Qwen2.5-1.5BModel Type=Dense, Active Parameters=1.5B, Total Parameters=1.5B2026.02 | 93.4 | |
| TSD-KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 93 | |
| Llama-3.2-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B2026.02 | 92.7 | |
| ExpertWeaver-E64-A14-S2(OLMo-7B)Model Type=MoE, Active Parameters=1B, Total Parameters=7B, Training Budget=200B tokens2026.02 | 91.8 | |
| SSQR-5%Avg. Bitwidth=3.725, Zero-shot=true2025.07 | 91.5 | |
| HPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 90.8 | |
| SSQR-5%Avg. Bitwidth=3.725, Backbone=Qwen3-8B2025.07 | 90.8 | |
| SSQR-3%Avg. Bitwidth=3.085, Zero-shot=true2025.07 | 90.5 | |
| SSQR-4%Avg. Bitwidth=3.405, Zero-shot=true2025.07 | 90.1 | |
| HPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 89.9 | |
| SSQR-4%Avg. Bitwidth=3.405, Backbone=Qwen3-8B2025.07 | 89.5 | |
| LLaMA-MoE-v2-3.5BModel Type=MoE, Active Parameters=3.5B2026.02 | 88.8 | |
| SSQR-5%Avg. Bitwidth=3.725, Zero-shot=true2025.07 | 88.7 | |
| OLMo-7BModel Type=Dense, Active Parameters=7B, Total Parameters=7B2026.02 | 88.6 | |
| OLMoE-1B-7B*Model Type=MoE, Active Parameters=1B, Total Parameters=7B, Training Budget=500B tokens2026.02 | 88.5 | |
| Open-LLaMA-3B-v2Model Type=Dense, Active Parameters=3B, Total Parameters=3B2026.02 | 88 | |
| PDPCSelection Metric=PD, Order=S-shape Function, Model Scale=1.3B, Training Tokens=100B2025.01 | 87.9 | |
| Sequence-Level KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 87.9 | |
| Sequence-Level KD2026.02 | 87.9 | |
| Preference CLSelection Metric=PPL, Order=S-shape Reverse Function, Model Scale=1.3B, Training Tokens=100B2025.01 | 87.8 | |
| Preference CLSelection Metric=Qu.Edu, Order=S-shape Function, Model Scale=1.3B, Training Tokens=100B2025.01 | 87.8 | |
| ExpertWeaver-E64-A14-S2(Qwen2.5-7B)Model Type=MoE, Active Parameters=3.5B, Total Parameters=7B, Training Budget=200B tokens2026.02 | 87.7 | |
| LLaMA-MoE-v1-3.5BModel Type=MoE, Active Parameters=3.5B2026.02 | 87.6 | |
| Sheared-LLaMA-2.7BModel Type=Dense, Active Parameters=2.7B, Total Parameters=2.7B2026.02 | 87.5 | |
| SequentialSelection Metric=PD, Order=High-to-Low, Model Scale=1.3B, Training Tokens=100B2025.01 | 87.1 | |
| RTNAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.07 | 87.1 | |
| HPTQAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.07 | 86.8 | |
| Speculative KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 86.7 | |
| Speculative KD2026.02 | 86.7 | |
| Supervised-KDDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 86.6 | |
| GKDbeta=0.9, Distillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 86.6 | |
| Supervised-KD2026.02 | 86.6 | |
| GKDbeta=0.92026.02 | 86.6 | |
| SSQR-3%Avg. Bitwidth=3.085, Backbone=Qwen3-8B2025.07 | 86.4 | |
| Qwen2.5-14B (Teacher)Model Scale=14B2026.02 | 86.4 | |
| Qwen2.5-14BRole=Teacher2026.02 | 86.4 | |
| SequentialSelection Metric=PD, Order=Low-to-High, Model Scale=1.3B, Training Tokens=100B2025.01 | 86.2 | |
| Preference CLSelection Metric=Qu.Edu, Order=S-shape Reverse Function, Model Scale=1.3B, Training Tokens=100B2025.01 | 86.2 | |
| Preference CLSelection Metric=PD, Order=S-shape Reverse Function, Model Scale=1.3B, Training Tokens=100B2025.01 | 86.2 | |
| RandomOrder=Random, Model Scale=1.3B, Training Tokens=100B2025.01 | 85.8 | |
| DistiLLMDistillation Setup=Qwen2.5 (14B -> 1.5B)2026.02 | 85.7 | |
| DistiLLM2026.02 | 85.7 | |
| INCITE-Base-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B2026.02 | 85.6 | |
| Preference CLSelection Metric=PPL, Order=S-shape Function, Model Scale=1.3B, Training Tokens=100B2025.01 | 85.5 | |
| Qwen2.5-1.5B (Student)Model Scale=1.5B2026.02 | 85.5 | |
| Qwen2.5-1.5BRole=Student2026.02 | 85.5 | |
| SequentialSelection Metric=Qu.Edu, Order=High-to-Low, Model Scale=1.3B, Training Tokens=100B2025.01 | 85.4 |