List your company
ResearchFormerly WizWand

Question Answering on OpenbookQA (Accuracy and Normalized accuracy)

87.6AccuracyQwen3-8B + SFT + WeMask(TF)
16.8835.2453.671.96Mar 6, 2026Mar 17, 2026Mar 28, 2026Apr 9, 2026Apr 20, 2026May 1, 2026May 13, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Qwen3-8B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0587.6—
Qwen3-8B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0587.2—
Qwen3-8B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0587—
AdaFuseBackbone=Mistral-7B, Evaluation Protocol=domain-specific fine-tuning2026.0386.6—
PESC (block-wise)Backbone=Mistral-7B, Evaluation Protocol=domain-specific fine-tuning2026.0386.4—
MoRAL (layer-wise)Backbone=Mistral-7B, Evaluation Protocol=domain-specific fine-tuning2026.0385.8—
LoRABackbone=Mistral-7B, Evaluation Protocol=domain-specific fine-tuning2026.0384.2—
Qwen3-4B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0582.2—
Qwen3-4B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0581.4—
Qwen3-4B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0581—
Qwen3-8B + Gated Attentionshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0579.4—
Qwen3-4B + Gated Attentionshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0577.8—
Mistral-7B (base)Backbone=Mistral-7B, Evaluation Protocol=Base2026.0357.8—
Nemotron-H-8BParameters=8B, FLOPs / token=1.51263 × 10^102026.0335.447.4
Swimba-14BParameters=14B, Experts=4, FLOPs / token=1.51282 × 10^102026.0334.946.7
TokAlign++Backbone=LLaMA38B, Initialization Method=TokAlign++, #GPU Hour=197.21, Shots=52026.0533.8—
TokAlign++Backbone=LLaMA38B, Initialization Method=TokAlign++, #GPU Hour=197.21, Shots=02026.0531.2—
Qwen3-8Bshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0527.6—
TokAlign++Backbone=Pythia2.8B, Initialization Method=TokAlign++, #GPU Hour=38.96, Shots=52026.0525.8—
TokAlign++Backbone=Pythia2.8B, Initialization Method=TokAlign++, #GPU Hour=38.96, Shots=02026.0524.8—
TokAlign++Backbone=Pythia1B, Initialization Method=TokAlign++, #GPU Hour=19.94, Shots=52026.0521.4—
Qwen3-4Bshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0519.6—
TokAlign++Backbone=Pythia1B, Initialization Method=TokAlign++, #GPU Hour=19.94, Shots=02026.0519.6—