List your company
ResearchFormerly WizWand

Reasoning on ARC Challenge (Accuracy)

94.3AccuracyGGRO
29.8246.5663.380.04Jun 3, 2026Jun 8, 2026Jun 14, 2026Jun 20, 2026Jun 26, 2026Jul 2, 2026Jul 8, 2026
Updated 3mo ago

Evaluation results

MethodLinks
GGROModel=LLaMA-3.1-8B-Instruct + Skywork-Reward-V2-LLaMA-3.1-8B2026.0694.3
BoN (N=64)Model=LLaMA-3.1-8B-Instruct + Skywork-Reward-V2-LLaMA-3.1-8B2026.0692.8
RSModel=LLaMA-3.1-8B-Instruct + Skywork-Reward-V2-LLaMA-3.1-8B2026.0692.3
ARGS-GModel=LLaMA-3.1-8B-Instruct + Skywork-Reward-V2-LLaMA-3.1-8B2026.0692
CARDSModel=LLaMA-3.1-8B-Instruct + Skywork-Reward-V2-LLaMA-3.1-8B2026.0691.8
CBSModel=LLaMA-3.1-8B-Instruct + Skywork-Reward-V2-LLaMA-3.1-8B2026.0690.5
SEAModel=LLaMA-3.1-8B-Instruct + Skywork-Reward-V2-LLaMA-3.1-8B2026.0687.8
CBSModel=LLaMA-3.2-3B-Instruct + GRM-LLaMA-3.2-3B-rewardmodel-ft2026.0687.5
RSModel=LLaMA-3.2-3B-Instruct + GRM-LLaMA-3.2-3B-rewardmodel-ft2026.0686
BoN (N=64)Model=LLaMA-3.2-3B-Instruct + GRM-LLaMA-3.2-3B-rewardmodel-ft2026.0686
GGROModel=LLaMA-3.2-3B-Instruct + GRM-LLaMA-3.2-3B-rewardmodel-ft2026.0686
Vanilla LLMModel=LLaMA-3.1-8B-Instruct + Skywork-Reward-V2-LLaMA-3.1-8B2026.0684
ARGS-GModel=LLaMA-3.2-3B-Instruct + GRM-LLaMA-3.2-3B-rewardmodel-ft2026.0684
CARDSModel=LLaMA-3.2-3B-Instruct + GRM-LLaMA-3.2-3B-rewardmodel-ft2026.0682.8
Vanilla LLMModel=LLaMA-3.2-3B-Instruct + GRM-LLaMA-3.2-3B-rewardmodel-ft2026.0680
SEAModel=LLaMA-3.2-3B-Instruct + GRM-LLaMA-3.2-3B-rewardmodel-ft2026.0676
FP16BIT=16, Evaluation Protocol=Few-shot2026.0762.37
GSRQBIT=2, Evaluation Protocol=Few-shot2026.0759.3
VQLLMBIT=2, Evaluation Protocol=Few-shot2026.0757.76
LizardTok. (B)=0.04, Cache Size=1322026.0756.7
STILL (concurrent)Tok. (B)=0.04, Cache Size=NA2026.0756.7
Liger-GLATok. (B)=0.02, Cache Size=642026.0755.6
LoLaTok. (B)=0.04, Cache Size=1282026.0755.4
OURS (GDN)Tok. (B)=0.01, Cache Size=64, Param.=10.3M2026.0754.92
OURS (GDN)Tok. (B)=0.01, Cache Size=128, Param.=10.3M2026.0754.86
LLaMA 3.1 8BCache Size=∞2026.0754.78
LlambaTok. (B)=12, Cache Size=0, Param.=8B2026.0754.6
LolCaTTok. (B)=0.04, Cache Size=642026.0754.44
Llama 3-8BParadigm=AR, Training Tokens=15T, Training Data=Not Released, Evaluation protocol=Reported by prior work, Shots=02026.0653.1
GSRQBIT=1, Evaluation Protocol=Few-shot2026.0752.05
LLaDA-8BParadigm=Masked Diffusion, Training Tokens=2.3T, Training Data=Not Released, Evaluation protocol=Reported by prior work, Shots=02026.0645.9
GSRQBIT=0.75, Evaluation Protocol=Few-shot2026.0745.22
Llama 2-7BParadigm=AR, Training Tokens=2T, Training Data=Not Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0645.1
PivotTraceBackbone=Deepseek-R1-Distill-Qwen-1.5B2026.0644.7
Falcon-7BParadigm=AR, Training Tokens=1.5T, Training Data=Partially Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0643.2
Sumi-7BParadigm=Uniform Diffusion, Training Tokens=1.5T, Training Data=Fully Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0643
EntropyBackbone=Deepseek-R1-Distill-Qwen-1.5B2026.0641.7
ConsistencyBackbone=Deepseek-R1-Distill-Qwen-1.5B, Multiple stochastic inferences=true2026.0640.8
OLMo-7BParadigm=AR, Training Tokens=2.5T, Training Data=Fully Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.0640.3
VQLLMBIT=1, Evaluation Protocol=Few-shot2026.0737.03
CoEBackbone=Deepseek-R1-Distill-Qwen-1.5B2026.0636.7
RandomBackbone=Deepseek-R1-Distill-Qwen-1.5B2026.0636.1
Self-CertaintyBackbone=Deepseek-R1-Distill-Qwen-1.5B2026.0632.7
CoT-KineticsBackbone=Deepseek-R1-Distill-Qwen-1.5B2026.0632.3