List your company
ResearchFormerly WizWand

Boolean Question Answering on BoolQ

91.26AccuracyDirect Fine-tuning
76.148880.071983.99587.9181May 11, 2023Nov 6, 2023May 3, 2024Oct 30, 2024Apr 27, 2025Oct 23, 2025Apr 21, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Direct Fine-tuningrank=1, training_steps=+0 steps2026.0291.26——
Direct Fine-tuningrank=1, training_steps=+200 steps2026.0291——
In-Squeezereduction=128 ... -> 1, strategy=Min steps2026.0290.92——
Cont-Squeezereduction=128 -> 1, training_steps=+200 steps2026.0290.81——
Direct Fine-tuningrank=1, training_steps=+700 steps2026.0290.79——
Cont-Squeezereduction=128 -> 1, training_steps=+700 steps2026.0290.56——
In-Squeezereduction=128 ... -> 1, strategy=Standard2026.0290.29——
Qwen3-30B-A3BPruning ratio=0%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0788.69——
UnifiedQA-3bparameters=3B, Protocol=Transfer-learning, External Knowledge=Yes2023.0787.8——
FP16Base Model=Qwen3-8B2026.0186.61——
RSPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0786.61——
Fullw_eff=∞2026.0486.6——
Stochasticw_eff=2562026.0486.6——
Qwen2-57B-A14BPruning ratio=0%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0786.45——
HiF4+HiGPTQModel=Qwen2.5-14B, A-W Quant Type=HiF4+HiGPTQ2026.0286.27—0.73
MoBA (k=2)w_eff=2562026.0486.1——
HiF4Model=Qwen2.5-14B, A-W Quant Type=HiF42026.0285.86—0.32
MC-SMoEPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0785.84——
NVFP4+PTSModel=Qwen2.5-14B, A-W Quant Type=NVFP4+PTS2026.0285.72—0.18
BF16Model=Qwen2.5-14B, A-W Quant Type=BF162026.0285.54——
MoNEPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0785.41——
FAQBase Model=Qwen3-8B2026.0185.29——
FP16Base Model=Qwen3-4B2026.0185.11——
Stochasticw_eff=1282026.0485.1——
FP16Base Model=Qwen2.5-7B2026.0184.71——
SWAw_eff=2562026.0484.7——
UnifiedQA-largeparameters=770M, Protocol=Transfer-learning, External Knowledge=Yes2023.0784.5——
IgnoringBase model=LLaMA-2-13B2025.0884.5——
AWQBase Model=Qwen3-8B2026.0184.22——
ForgettingBase model=LLaMA-2-13B2025.0884.13——
NVFP4Model=Qwen2.5-14B, A-W Quant Type=NVFP42026.0283.49—-2.05
FAQBase Model=Qwen2.5-7B2026.0183.3——
MoBA (k=2)w_eff=1282026.0482.6——
Full Tokens (standard SFT)Base model=LLaMA-2-13B2025.0882.24——
HiF4+HiGPTQModel=Mistral-7B, A-W Quant Type=HiF4+HiGPTQ2026.0282.22—0.05
BaselineBit width (b)=16.002025.0582.2——
BF16Model=Mistral-7B, A-W Quant Type=BF162026.0282.17——
OriginalModel=LLaMA-3.1-8B, Ratio=1.0, Evaluation Protocol=0-shot2026.0482.1——
HiF4Model=Mistral-7B, A-W Quant Type=HiF42026.0281.35—-0.82
BaselineParam Ratio=1.00, Base Model=LLaMA-3-8b, Fine-tuning Protocol=None2025.1281.3——
BF16Model=Llama3-8B, A-W Quant Type=BF162026.0281.16——
MoNEPruning ratio=50%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0780.99——
NVFP4+PTSModel=Mistral-7B, A-W Quant Type=NVFP4+PTS2026.0280.86—-1.31
UnifiedQA-baseparameters=220M, Protocol=Transfer-learning, External Knowledge=Yes2023.0780.8——
LLaMA2-7BNumber of Parameters=7B, Backbone=LLaMA2, Shots=02024.0780.7——
BaseBase model=LLaMA-2-13B2025.0880.67——
AWQBase Model=Qwen2.5-7B2026.0180.49——
HiF4+HiGPTQModel=Llama3-8B, A-W Quant Type=HiF4+HiGPTQ2026.0280.4—-0.76
MoonlightPruning ratio=0%, Base Model=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0780.4——
Llama 2 7Bshot=zero-shot, normalization=none2024.0280.2——
AWQBase Model=Qwen3-4B2026.0180——
MC-SMoEPruning ratio=50%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0779.91——
Deepseek-V2-LitePruning ratio=0%, Base Model=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0779.88——
HiF4Model=Llama3-8B, A-W Quant Type=HiF42026.0279.85—-1.31
SFTVocab size=64k, Number of shots=4-shot2025.1279.8——
ESAMBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.0279.79——
RTNBase Model=Qwen3-8B2026.0179.48——
CD-MoE-SR#L=9, ACT.=2.8B, MEM.=72.5%, Speedup=1.26x, SFT Protocol=w/ lightweight SFT2024.1179.4——
SL-SAMBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.0279.39——
Qwen3-1.7B#Params=1.7B/1.7B, #Tokens=36T, Zero-shot=true2026.0279.3——
ALM*Vocab size=64k, Number of shots=4-shot, Combined with SFT=true2025.1279.3——
RSTBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.0279.24——
AdaSAMBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.0279.2——
AdamWBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.0279.14——
FKL*Vocab size=64k, Number of shots=4-shot, Combined with SFT=true2025.1278.9——
DSKD*Vocab size=64k, Number of shots=4-shot, Combined with SFT=true2025.1278.9——
FKL*Vocab size=16k, Number of shots=4-shot, Combined with SFT=true2025.1278.9——
FKLVocab size=32k, Number of shots=4-shot2025.1278.8——
SFTVocab size=16k, Number of shots=4-shot2025.1278.8——
OLTQAConfiguration=Full2023.0578.78——
ALM*Vocab size=32k, Number of shots=4-shot, Combined with SFT=true2025.1278.6——
DSKD*Vocab size=32k, Number of shots=4-shot, Combined with SFT=true2025.1278.6——
ALMVocab size=16k, Number of shots=4-shot2025.1278.6——
ALM*Vocab size=16k, Number of shots=4-shot, Combined with SFT=true2025.1278.6——
FKL*Vocab size=32k, Number of shots=4-shot, Combined with SFT=true2025.1278.5——
FAQBase Model=Qwen3-4B2026.0178.41——
ULDVocab size=32k, Number of shots=4-shot2025.1278.4——
NVFP4Model=Llama3-8B, A-W Quant Type=NVFP42026.0278.35—-2.81
SFTVocab size=32k, Number of shots=4-shot2025.1278.3——
ALMVocab size=32k, Number of shots=4-shot2025.1278.3——
ALMVocab size=64k, Number of shots=4-shot2025.1278.2——
ULD*Vocab size=32k, Number of shots=4-shot, Combined with SFT=true2025.1278.2——
FKLVocab size=16k, Number of shots=4-shot2025.1278.1——
OriginalModel=LLaMA-2-7B, Ratio=1.0, Evaluation Protocol=0-shot2026.0477.9——
BaselineParam Ratio=1.00, Base Model=LLaMA-2-7b, Fine-tuning Protocol=None2025.1277.8——
BF16Model=Llama2-7B, A-W Quant Type=BF162026.0277.74——
RTNBase Model=Qwen2.5-7B2026.0177.71——
G-LionShots=3, Optimizer=Global Lion, Model=LLaMA 7B, Instruction Finetuning=true2024.0377.5——
FKLVocab size=64k, Number of shots=4-shot2025.1277.5——
NVFP4+PTSModel=Llama3-8B, A-W Quant Type=NVFP4+PTS2026.0277.46—-3.7
SPES-9B#Params=3.1B/9B, #Tokens=400B, Zero-shot=true, initialized from=pretrained dense model2026.0277.3——
LLRCParam Ratio=0.90, Base Model=LLaMA-2-7b, Fine-tuning Protocol=None2025.1277.3——
G-AdamWShots=3, Optimizer=Global AdamW, Model=LLaMA 7B, Instruction Finetuning=true2024.0377.23——
DSKDVocab size=64k, Number of shots=4-shot2025.1277.2——
DSKDVocab size=32k, Number of shots=4-shot2025.1277.2——
Distributed Lion-MaVoShots=3, Optimizer=D-Lion (MaVo), Model=LLaMA 7B, Instruction Finetuning=true2024.0377.14——
Stochasticw_eff=642026.0477.1——
OLTQAKnowledge Distillation=Static MKD2023.0577.03——
Distributed Lion-AvgShots=3, Optimizer=D-Lion (Avg), Model=LLaMA 7B, Instruction Finetuning=true2024.0376.9——
HiF4Model=Llama2-7B, A-W Quant Type=HiF42026.0276.73—-1.01