List your company
ResearchFormerly WizWand

Question Answering on OpenBookQA (Normalized Accuracy)

55.6Normalized AccuracyLlama-Instruct
-5.34410.47826.342.122Dec 13, 2024Mar 3, 2025May 22, 2025Aug 10, 2025Oct 29, 2025Jan 17, 2026Apr 8, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Llama-InstructShots=10-shot2026.0355.6
HATified-SFTShots=10-shot2026.0352.6
BaseModel=Llama-70B2026.0449
BaseModel=Qwen2.5-7B2026.0447.4
BaseModel=Mistral-7B2026.0446.2
BaseModel=Llama-13B2026.0445.6
Llama 3-8BShots=0-shot2024.1245
Llama 3-8B E8T2Shots=0-shot2024.1244.8
GlobalGCBackbone=Llama-2 7B, Zero-Shot=true2025.0233.6
ClipByValueBackbone=Llama-2 7B, Zero-Shot=true2025.0233.2
GlobalGCBackbone=Mixtral 8x1B, Zero-Shot=true2025.0233
AGCBackbone=Llama-2 7B, Zero-Shot=true2025.0232.8
AdaGCBackbone=Llama-2 7B, Zero-Shot=true2025.0232.8
Mamba-2Model Scale=440M, Evaluation Protocol=Zero-shot2026.0432.8
Mamba-2-1.5BTraining Tokens=100B, Context Length=2K, Scale=1.5B2026.0332.6
Mamba-3-MIMO-1.5BTraining Tokens=100B, Context Length=2K, Scale=1.5B, MIMO Rank=42026.0332.6
Mamba-2 + PoSTModel Scale=440M, Evaluation Protocol=Zero-shot2026.0432.6
ClippyBackbone=Llama-2 7B, Zero-Shot=true2025.0232.4
AdaGCBackbone=Mixtral 8x1B, Zero-Shot=true2025.0232.2
Mamba-3-SISO-1.5BTraining Tokens=100B, Context Length=2K, Scale=1.5B2026.0332
RWKV-7 + PoSTModel Scale=180M, Evaluation Protocol=Zero-shot2026.0432
Gated DeltaNet + PoSTModel Scale=180M, Evaluation Protocol=Zero-shot2026.0431.8
EFLAParameters=1.3B, Zero-shot=true2025.1231.6
GDN-1.5BTraining Tokens=100B, Context Length=2K, Scale=1.5B2026.0331.6
BaseModel=GPT-2 (774M)2026.0431.2
EFLA + Loose βParameters=340M, Zero-shot=true2025.1230.8
GatedFWA-NSAClass=Attention, Implementation=Triton, Linear=true2025.1230.8
Mamba-2Model Scale=180M, Evaluation Protocol=Zero-shot2026.0430.6
Gated DeltaNetModel Scale=180M, Evaluation Protocol=Zero-shot2026.0430.6
ClipByValueBackbone=Llama-2 1.3B, Zero-Shot=true2025.0230.4
AdaGCBackbone=Llama-2 1.3B, Zero-Shot=true2025.0230.4
GlobalGCBackbone=Llama-2 1.3B, Zero-Shot=true2025.0230.2
Mamba-2-880MTraining Tokens=100B, Context Length=2K, Scale=880M2026.0330.2
Transformer (LLaMA) + SWA + NSAClass=Attention, Implementation=Triton, Linear=true2025.1230.18
HGRN2Class=RNN-Like, Implementation=Triton, Linear=true2025.1230
ClippyBackbone=Llama-2 1.3B, Zero-Shot=true2025.0230
Mamba-3-SISO-880MTraining Tokens=100B, Context Length=2K, Scale=880M2026.0330
Mamba-2 + PoSTModel Scale=180M, Evaluation Protocol=Zero-shot2026.0430
GatedFWAClass=Attention, Implementation=Triton, Linear=true2025.1229.93
DeltaNetParameters=1.3B, Zero-shot=true2025.1229.8
PLDRv51-SOC-110M-5Zero-shot=true2026.0329.8
Transformer-1.5BTraining Tokens=100B, Context Length=2K, Scale=1.5B2026.0329.6
Transformer (LLaMA)Class=Attention, Implementation=CUDA, Linear=false2025.1229.4
Transformer (LLaMA) + SWAClass=Attention, Implementation=CUDA, Linear=true2025.1229.22
GLAClass=RNN-Like, Implementation=Triton, Linear=true2025.1229.2
PLDRv51-SOC-110M-4Zero-shot=true2026.0329.2
DeltaNetClass=RNN-Like, Implementation=Triton, Linear=true2025.1229
PolychromaticLMstage=Base2026.0329
RWKV-7Model Scale=180M, Evaluation Protocol=Zero-shot2026.0429
GDN-880MTraining Tokens=100B, Context Length=2K, Scale=880M2026.0328.6
Mamba-3-MIMO-880MTraining Tokens=100B, Context Length=2K, Scale=880M, MIMO Rank=42026.0328.6
RetNetClass=RNN-Like, Implementation=CUDA, Linear=true2025.1228.4
Mamba-3-MIMO-440MTraining Tokens=100B, Context Length=2K, Scale=440M, MIMO Rank=42026.0328.4
ABL-SOC-110M-3Zero-shot=true2026.0328.2
MambaClass=RNN-Like, Implementation=CUDA, Linear=true2025.1228
GDN-440MTraining Tokens=100B, Context Length=2K, Scale=440M2026.0327.6
ABL-SOC-110M-1Zero-shot=true2026.0327.6
PLDRv51-SOC-110M-3Zero-shot=true2026.0327.2
DeltaNetParameters=340M, Zero-shot=true2025.1227
EFLA + Adaptive DecayParameters=340M, Zero-shot=true2025.1227
PLDRv51-SOC-110M-1Zero-shot=true2026.0327
PolychromaticLMstage=SFT2026.0326.8
Transformer-880MTraining Tokens=100B, Context Length=2K, Scale=880M2026.0326.8
EFLAParameters=340M, Zero-shot=true2025.1226.6
ABL-SOC-110M-2Zero-shot=true2026.0326.6
RFMoEScale=L, Size=870.6M, FLOPs=613.2M, Zero-shot=true2026.0426.6
RFMoEScale=M, Size=307.3M, FLOPs=249.2M, Zero-shot=true2026.0426.4
PLDRv51-SOC-110M-2Zero-shot=true2026.0326.2
GPT-Neo-125MZero-shot=true2026.0326.2
Transformer-440MTraining Tokens=100B, Context Length=2K, Scale=440M2026.0326
Mamba-2-440MTraining Tokens=100B, Context Length=2K, Scale=440M2026.0326
Mamba-3-SISO-440MTraining Tokens=100B, Context Length=2K, Scale=440M2026.0326
SUB-SOC-110M-1Zero-shot=true2026.0326
ReMoEScale=S, Size=92.44M, FLOPs=90.93M, Zero-shot=true2026.0426
SUB-SOC-110M-2Zero-shot=true2026.0325.2
Random2026.0325
AoEScale=S, Size=93.85M, FLOPs=88.57M, Zero-shot=true2026.0425
MoEScale=L, Size=808.4M, FLOPs=608.4M, Zero-shot=true2026.0425
MoEScale=S, Size=92.44M, FLOPs=90.93M, Zero-shot=true2026.0424.6
MoEScale=M, Size=289.9M, FLOPs=248.0M, Zero-shot=true2026.0424.6
RFMoEScale=S, Size=95.32M, FLOPs=91.08M, Zero-shot=true2026.0424.4
Mamba-2-180MTraining Tokens=100B, Context Length=2K, Scale=180M2026.0323.2
Mamba-3-SISO-180MTraining Tokens=100B, Context Length=2K, Scale=180M2026.0322.8
GDN-180MTraining Tokens=100B, Context Length=2K, Scale=180M2026.0322
Mamba-3-MIMO-180MTraining Tokens=100B, Context Length=2K, Scale=180M, MIMO Rank=42026.0322
Transformer-180MTraining Tokens=100B, Context Length=2K, Scale=180M2026.0321.8
DualAlpha (α)=63/64, Data repetitions=12025.1217.6
DualAlpha (α)=3/4, Data repetitions=322025.1214.4
AutoregressiveAlpha (α)=1, Data repetitions=12025.1213.6
AutoregressiveAlpha (α)=1, Data repetitions=322025.129.9
DualAlpha (α)=1/8, Data repetitions=1282025.128.5
GAINModel=Llama-13B2026.040.4
GAINModel=Mistral-7B2026.040
GAINModel=Llama-70B2026.04-0.2
AutoregressiveAlpha (α)=1, Data repetitions=1282025.12-0.5
GAINModel=Qwen2.5-7B2026.04-1
GAINModel=GPT-2 (774M)2026.04-1.8
LoRAModel=GPT-2 (774M)2026.04-2.2
LoRAModel=Qwen2.5-7B2026.04-2.8
LoRAModel=Mistral-7B2026.04-3