List your company
ResearchFormerly WizWand

Reasoning on ARC Hard

1.22Accuracy ImprovementN-GRAM SAMPLING
-0.03840.28830.6150.9417Jun 27, 2025
Updated 8mo ago

Evaluation results

MethodLinks
N-GRAM SAMPLINGNumber of Parameters=350M2025.061.22
N-GRAM SAMPLINGNumber of Parameters=1.5B2025.060.82
N-GRAM SAMPLINGNumber of Parameters=3B2025.060.72
PPL SAMPLINGNumber of Parameters=350M2025.060.6
AUTOMIXER-350MNumber of Parameters=350M2025.060.55
PPL SAMPLINGNumber of Parameters=1.5B2025.060.52
AUTOMIXER-350MNumber of Parameters=1.5B2025.060.39
AUTOMIXER-350MNumber of Parameters=3B2025.060.34
PPL SAMPLINGNumber of Parameters=3B2025.060.32
AUTOMIXER-75MNumber of Parameters=350M2025.060.12
AUTOMIXER-75MNumber of Parameters=1.5B2025.060.01
AUTOMIXER-75MNumber of Parameters=3B2025.060.01