List your company
ResearchFormerly WizWand

Mathematical Reasoning on MATH 500 (Accuracy @1)

77.6Accuracy (greedy pass@1)oracle ZS vs Mem
43.69652.49861.370.102Jun 16, 2026
Updated 4mo ago

Evaluation results

MethodLinks
oracle ZS vs MemBase Model=Qwen2.5-Math-7B, #Train Params=-2026.0677.6
ELM offlineBase Model=Qwen2.5-Math-7B, #Train Params=0.07M2026.0673.4
ELM-LoRAofflineBase Model=Qwen2.5-Math-7B, #Train Params=40M2026.0668.6
Full GTBase Model=Qwen2.5-Math-7B, #Train Params=7B2026.0668
ELM-Full ModelofflineBase Model=Qwen2.5-Math-7B, #Train Params=7B2026.0667.8
Full MajBase Model=Qwen2.5-Math-7B, #Train Params=7B2026.0667.2
Zero-shot CoTBase Model=Qwen2.5-Math-7B, #Train Params=-2026.0667
oracle ZS vs MemBase Model=Llama-3.1-8B-Instr., #Train Params=-2026.0655.4
GRPOBase Model=Llama-3.1-8B-Instr., #Train Params=8B2026.0647
ELM-LoRAofflineBase Model=Llama-3.1-8B-Instr., #Train Params=42M2026.0646.2
ELM-Full ModelofflineBase Model=Llama-3.1-8B-Instr., #Train Params=8B2026.0646
TTRLBase Model=Llama-3.1-8B-Instr., #Train Params=8B2026.0645.6
Zero-shot CoTBase Model=Llama-3.1-8B-Instr., #Train Params=-2026.0645
ELM offlineBase Model=Llama-3.1-8B-Instr., #Train Params=0.08M2026.0645