List your company
ResearchFormerly WizWand

Open-domain Question Answering on Natural Questions (Accuracy)

30.6AccuracyMixtral 8x7B
7.61613.58319.5525.517Jan 8, 2024Jun 3, 2024Oct 28, 2024Mar 25, 2025Aug 19, 2025Jan 13, 2026Jun 10, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Mixtral 8x7BActive Params=13B2024.0130.6
MistralSize=7B, Tokens=?, Shot(s)=5, Training Strategy=trained from scratch, Attention Type=softmax-attention2024.0929.7
GSASize=7B, Tokens=+100B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.0926.9
Llama2Size=7B, Tokens=2T, Shot(s)=5, Training Strategy=trained from scratch, Attention Type=softmax-attention2024.0926
LLaMA 2 70BActive Params=70B2024.0125.4
MambaSize=7B, Tokens=1.2T, Shot(s)=5, Training Strategy=trained from scratch2024.0925.4
MoE with MPIModel Scale=11B, Setup=5-shot2026.0625.36
MoEModel Scale=11B, Setup=5-shot2026.0625.3
SUPRASize=7B, Tokens=+100B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.0924.7
GemmaSize=7B, Tokens=6T, Shot(s)=5, Training Strategy=trained from scratch, Attention Type=softmax-attention2024.0924.3
LLaMA 1 33BActive Params=33B2024.0124.1
GSASize=7B, Tokens=+20B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.0923.4
Mistral 7BActive Params=7B2024.0123.2
GLASize=7B, Tokens=+20B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.0922.2
RWKV6Size=7B, Tokens=1.4T, Shot(s)=5, Training Strategy=trained from scratch2024.0920.9
MoE with MPIModel Scale=3B, Setup=5-shot2026.0620.13
MoEModel Scale=3B, Setup=5-shot2026.0617.87
LLaMA 2 7BActive Params=7B2024.0117.5
LLaMA 2 13BActive Params=13B2024.0116.7
RetNetSize=7B, Tokens=+20B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.0916.2
BaseModel=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.0415.12
POPModel=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.0414.65
BaseModel=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.0414.36
POPModel=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.0414.01
Train on DModel=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.0412.54
Train on DModel=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.048.5