List your company
ResearchFormerly WizWand

Math & Code Reasoning on ARC Easy

81.1ScoreLlama 3-8B
25.04439.59754.1568.703May 29, 2026Jun 1, 2026Jun 4, 2026Jun 7, 2026Jun 10, 2026Jun 13, 2026Jun 17, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Llama 3-8BParadigm=AR, Training Tokens=15T, Training Data=Not Released, Evaluation protocol=Reported by prior work, Shots=02026.0681.1
Llama 2-7BParadigm=AR, Training Tokens=2T, Training Data=Not Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0673.8
LLaDA-8BParadigm=Masked Diffusion, Training Tokens=2.3T, Training Data=Not Released, Evaluation protocol=Reported by prior work, Shots=02026.0671.8
Falcon-7BParadigm=AR, Training Tokens=1.5T, Training Data=Partially Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0670.8
Sumi-7BParadigm=Uniform Diffusion, Training Tokens=1.5T, Training Data=Fully Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0670
OLMo-7BParadigm=AR, Training Tokens=2.5T, Training Data=Fully Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0668.8
DiReCTBackbone=Llama-1.1B2026.0547
GradNorm (IS)Backbone=Llama-1.1B2026.0544.3
InfoBatchBackbone=Llama-1.1B2026.0543.7
Perplexity-basedBackbone=Llama-1.1B2026.0542.4
Uniform SamplingBackbone=Llama-1.1B2026.0541.3
Loss-basedBackbone=Llama-1.1B2026.0541
DiReCTBackbone=GPT-2-Medium (355M)2026.0534
InfoBatchBackbone=GPT-2-Medium (355M)2026.0531.7
GradNorm (IS)Backbone=GPT-2-Medium (355M)2026.0530.6
Perplexity-basedBackbone=GPT-2-Medium (355M)2026.0529.8
Loss-basedBackbone=GPT-2-Medium (355M)2026.0528.4
Uniform SamplingBackbone=GPT-2-Medium (355M)2026.0527.2