List your company
ResearchFormerly WizWand

Science Question Answering on ARC

98.8ARC AccuracyLlama-4-Maverick
19.999240.457160.91581.3729Aug 27, 2024Dec 12, 2024Mar 30, 2025Jul 15, 2025Oct 31, 2025Feb 15, 2026Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Llama-4-MaverickCategory=Candidate Models2026.0398.8
Claude-Haiku-4.5Category=Candidate Models2026.0398.8
DeepSeek-R1Category=Candidate Models2026.0398.8
GraphRouterCategory=Routers2026.0398.8
FineRouterCategory=Routers2026.0398.8
IPRCategory=Routers2026.0397.7
Claude-Sonnet-4.5Category=Candidate Models2026.0397.6
GPT-OSS-120BCategory=Candidate Models2026.0397.6
kNNCategory=Routers2026.0397.6
RouteLLMCategory=Routers2026.0397.6
Qwen3-32BCategory=Candidate Models2026.0396.4
Mistral-LargeCategory=Candidate Models2026.0392.9
DeepSeek-v3Category=Candidate Models2026.0391.7
Qwen3-235B-A22BCategory=Candidate Models2026.0391.7
RouterDCCategory=Routers2026.0391.7
Llama-3.3-70BCategory=Candidate Models2026.0390.5
FPModel=LLADA-1.5, Precision=Full Precision2026.0688.5
STaR-QuantModel=LLADA-1.5, Quantization=W8A82026.0686.5
AutoAdaptSetting=Template Aware (TA), Technique=SFT2026.0383.79
HF DefaultsSetting=Template Aware (TA), Technique=SFT2026.0378.24
MLCopilotSetting=Template Aware (TA), Technique=SFT2026.0377.56
DS-AgentSetting=Template Aware (TA), Technique=SFT2026.0377.47
AutoMLAgentSetting=Template Aware (TA), Technique=SFT2026.0377.39
MLPCategory=Routers2026.0370.2
Mistral-SmallCategory=Candidate Models2026.0369.1
FullBackbone=Mixtral-8x7B-Instruct, Expert Sparsity=0%2025.1262.54
FullModel=Mixtral-8x7B-Instruct, Expert Sparsity=0%2025.1262.54
STaR-QuantModel=DREAM, Quantization=W8A82026.0660.28
FPModel=DREAM, Precision=Full Precision2026.0659.8
FullModel=Mixtral-8x7B, Expert Sparsity=0%2025.1256.48
L3-8B (Base)Model=L3-8B, dqk=128, dvo=128, Method=Base2025.0755.5
Mistral-7B-INSTBase Model=Mistral-7B-INST, Alignment Strategy=Base, Alignment Data Prompts=HelpSteer22024.0855.29
RLHFBase Model=Mistral-7B-INST, Alignment Strategy=RLHF, Alignment Data Prompts=HelpSteer22024.0855.2
UNABase Model=Mistral-7B-INST, Alignment Strategy=UNA, Alignment Data Prompts=HelpSteer22024.0855.2
KV-Latent Train (L3-8B)Model=L3-8B, dqk=64, dvo=64, Method=Train2025.0753.8
MoE PathfinderBackbone=Mixtral-8x7B-Instruct, Expert Sparsity=50%2025.1253.18
MoE PathfinderModel=Mixtral-8x7B-Instruct, Expert Sparsity=50%2025.1253.18
NAMEx-FullRouting Strategy=Stable-MoE, Evaluation Protocol=Fine-tuned (SmolTalk), Disagreement Point=02025.1050.64
NAMEx-FullRouting Strategy=Stable-MoE, Evaluation Protocol=Fine-tuned (SmolTalk), Disagreement Point=mean2025.1050.63
EP-CAMExRouting Strategy=Stable-MoE, Evaluation Protocol=Fine-tuned (SmolTalk)2025.1050.45
Deepseek-MoERouting Strategy=Stable-MoE, Evaluation Protocol=Fine-tuned (SmolTalk)2025.1050.28
NAMEx-FullRouting Strategy=Stable-MoE, Evaluation Protocol=Zero-Shot, Disagreement Point=mean2025.1050.19
NAMEx-FullRouting Strategy=Stable-MoE, Evaluation Protocol=Zero-Shot, Disagreement Point=02025.1050.15
EP-CAMExRouting Strategy=Stable-MoE, Evaluation Protocol=Zero-Shot2025.1050
NAMEx-FullRouting Strategy=Cosine, Evaluation Protocol=Fine-tuned (SmolTalk), Disagreement Point=02025.1049.92
NAMEx-FullRouting Strategy=Cosine, Evaluation Protocol=Fine-tuned (SmolTalk), Disagreement Point=mean2025.1049.92
Deepseek-MoERouting Strategy=Stable-MoE, Evaluation Protocol=Zero-Shot2025.1049.9
NAMEx-FullRouting Strategy=Linear, Evaluation Protocol=Fine-tuned (SmolTalk), Disagreement Point=02025.1049.85
NAMEx-FullRouting Strategy=Linear, Evaluation Protocol=Fine-tuned (SmolTalk), Disagreement Point=mean2025.1049.84
EP-CAMExRouting Strategy=Cosine, Evaluation Protocol=Fine-tuned (SmolTalk)2025.1049.73
MoE PathfinderModel=Mixtral-8x7B, Expert Sparsity=50%2025.1249.66
EP-CAMExRouting Strategy=Linear, Evaluation Protocol=Fine-tuned (SmolTalk)2025.1049.62
NAMEx-FullRouting Strategy=Cosine, Evaluation Protocol=Zero-Shot, Disagreement Point=02025.1049.6
Deepseek-MoERouting Strategy=Cosine, Evaluation Protocol=Fine-tuned (SmolTalk)2025.1049.6
NAMEx-FullRouting Strategy=Cosine, Evaluation Protocol=Zero-Shot, Disagreement Point=mean2025.1049.58
NAMEx-FullRouting Strategy=Linear, Evaluation Protocol=Zero-Shot, Disagreement Point=mean2025.1049.52
NAMEx-FullRouting Strategy=Linear, Evaluation Protocol=Zero-Shot, Disagreement Point=02025.1049.51
Deepseek-MoERouting Strategy=Linear, Evaluation Protocol=Fine-tuned (SmolTalk)2025.1049.5
EP-CAMExRouting Strategy=Cosine, Evaluation Protocol=Zero-Shot2025.1049.4
Deepseek-MoERouting Strategy=Cosine, Evaluation Protocol=Zero-Shot2025.1049.3
EP-CAMExRouting Strategy=Linear, Evaluation Protocol=Zero-Shot2025.1049.26
Deepseek-MoERouting Strategy=Linear, Evaluation Protocol=Zero-Shot2025.1049.15
UNABase Model=Qwen2-1.5B-INST, Alignment Strategy=UNA, Alignment Data Prompts=HelpSteer22024.0844.28
STaR-QuantModel=LLADA, Quantization=W8A82026.0644.25
FPModel=LLADA, Precision=Full Precision2026.0644.03
Qwen2-1.5B-INSTBase Model=Qwen2-1.5B-INST, Alignment Strategy=Base, Alignment Data Prompts=HelpSteer22024.0843.94
LLaMA-3.2-3B-InstructNumber of shots=0-shot2025.1243.6
RLHFBase Model=Qwen2-1.5B-INST, Alignment Strategy=RLHF, Alignment Data Prompts=HelpSteer22024.0842.83
LLaMA-3.2-3B-Instruct + SFT-v3.4 + DPONumber of shots=0-shot2025.1241.81
LLaMA-3.2-3B-Instruct + SFT-v3.4 + DPO + DTFT-v3Number of shots=0-shot2025.1241.13
LLaMA-3.2-3B-Instruct + SFT-v3.4 + DPO + DTFT-v5Number of shots=0-shot2025.1240.78
LLaMA-3.2-3B-Instruct + SFT-v3.4 + DPO + DTFT-v4Number of shots=0-shot2025.1240.7
StatLLaMA (DTFT-v2)Number of shots=0-shot2025.1240.61
LLaMA-3.2-3B-Instruct + SFT-v3.4 + DPO + DTFT-v1Number of shots=0-shot2025.1240.27
KV-Latent Distill (L3-8B)Model=L3-8B, dqk=64, dvo=64, Method=Distill2025.0739.1
KV-Latent Train (L3-8B, d=16)Model=L3-8B, dqk=16, dvo=16, Method=Train2025.0738.5
L2-7B (Base)Model=L2-7B, dqk=128, dvo=128, Method=Base2025.0730.7
RandomBackbone=Mixtral-8x7B-Instruct, Expert Sparsity=50%2025.1229.61
RandomModel=Mixtral-8x7B-Instruct, Expert Sparsity=50%2025.1229.61
KV-Latent Train (L2-7B)Model=L2-7B, dqk=64, dvo=64, Method=Train2025.0727.5
KV-Latent Distill (L2-7B)Model=L2-7B, dqk=64, dvo=64, Method=Distill2025.0727
RandomModel=Mixtral-8x7B, Expert Sparsity=50%2025.1223.03