List your company
ResearchFormerly WizWand

Language Understanding on MMLU-Pro (Accuracy)

72.8MMLU-Pro AccuracyFOREVER
7.824.67541.5558.425Apr 21, 2026Apr 26, 2026May 1, 2026May 6, 2026May 11, 2026May 16, 2026May 22, 2026
Updated 5mo ago

Evaluation results

MethodLinks
FOREVERModel Scale=14B, Training Protocol=Capability-continual2026.0572.8
MTLModel Scale=14B, Training Protocol=Joint-training2026.0569.8
EWCModel Scale=14B, Training Protocol=Capability-continual2026.0569.4
GCWMModel Scale=14B, Training Protocol=Capability-continual2026.0568.8
FOREVERModel Scale=8B, Training Protocol=Capability-continual2026.0568.1
Seq. SFTModel Scale=14B, Training Protocol=Capability-continual2026.0567.2
AIMMergingModel Scale=8B, Training Protocol=Capability-continual2026.0566.7
GCWMModel Scale=8B, Training Protocol=Capability-continual2026.0566.5
OPCMModel Scale=14B, Training Protocol=Capability-continual2026.0566.3
L&SModel Scale=8B, Training Protocol=Capability-continual2026.0566.1
MTLModel Scale=8B, Training Protocol=Joint-training2026.0565.4
MTLModel Scale=4B, Training Protocol=Joint-training2026.0565.3
OPCMModel Scale=8B, Training Protocol=Capability-continual2026.0565.2
OPCMModel Scale=4B, Training Protocol=Capability-continual2026.0564.2
EWCModel Scale=8B, Training Protocol=Capability-continual2026.0564.2
GCWMModel Scale=4B, Training Protocol=Capability-continual2026.0563.9
Seq. SFTModel Scale=4B, Training Protocol=Capability-continual2026.0563.6
FOREVERModel Scale=4B, Training Protocol=Capability-continual2026.0562.8
Seq. SFTModel Scale=8B, Training Protocol=Capability-continual2026.0560.4
EWCModel Scale=4B, Training Protocol=Capability-continual2026.0560
MTLModel Scale=1.7B, Training Protocol=Joint-training2026.0557.5
AIMMergingModel Scale=14B, Training Protocol=Capability-continual2026.0557.5
L&SModel Scale=14B, Training Protocol=Capability-continual2026.0556.8
BaseBackbone=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.0456.57
POPBackbone=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.0456.44
AIMMergingModel Scale=4B, Training Protocol=Capability-continual2026.0556.4
L&SModel Scale=4B, Training Protocol=Capability-continual2026.0555.6
Train on DBackbone=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.0453.86
GCWMModel Scale=1.7B, Training Protocol=Capability-continual2026.0552
OPCMModel Scale=1.7B, Training Protocol=Capability-continual2026.0551.9
ARES-SFT2026.0550.56
Webscale2026.0549.5
ARES-RL2026.0549.36
Natural Reasoning2026.0547.96
FOREVERModel Scale=1.7B, Training Protocol=Capability-continual2026.0547.3
NSAModel Size=8B, Context=short-context2026.0546.3
DashAttentionModel Size=8B, Context=short-context2026.0546.2
FullAttnModel Size=8B, Context=short-context2026.0546.1
CPT2026.0546.02
InfLLMv2Model Size=8B, Context=short-context2026.0545.6
MTLModel Scale=0.6B, Training Protocol=Joint-training2026.0545.1
AIMMergingModel Scale=1.7B, Training Protocol=Capability-continual2026.0544.3
POPBackbone=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.0443.82
L&SModel Scale=1.7B, Training Protocol=Capability-continual2026.0543.5
EWCModel Scale=1.7B, Training Protocol=Capability-continual2026.0540.5
Train on DBackbone=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.0438.07
Seq. SFTModel Scale=1.7B, Training Protocol=Capability-continual2026.0535.3
BaseBackbone=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.0430.1
AIMMergingModel Scale=0.6B, Training Protocol=Capability-continual2026.0529.4
FOREVERModel Scale=0.6B, Training Protocol=Capability-continual2026.0529.1
L&SModel Scale=0.6B, Training Protocol=Capability-continual2026.0529
GCWMModel Scale=0.6B, Training Protocol=Capability-continual2026.0527.8
OPCMModel Scale=0.6B, Training Protocol=Capability-continual2026.0526.7
EWCModel Scale=0.6B, Training Protocol=Capability-continual2026.0526.6
Seq. SFTModel Scale=0.6B, Training Protocol=Capability-continual2026.0524.2
LFM2.5 350MTraining tokens=28T, Evaluation Protocol=0-shot2026.0418.6
Nautile-370MTraining tokens=∼0.8T, Evaluation Protocol=0-shot2026.0414.9
Qwen2.5 0.5BTraining tokens=18T, Evaluation Protocol=0-shot2026.0414.3
Granite 350MTraining tokens=10–12T, Evaluation Protocol=0-shot2026.0411.2
SmolLM2 360MTraining tokens=4T, Evaluation Protocol=0-shot2026.0410.3