Mathematical Reasoning on MATH 500 (Accuracy @1)
77.6Accuracy (greedy pass@1)oracle ZS vs Mem
Evaluation Results
| Method | Links | |
|---|---|---|
| oracle ZS vs MemBase Model=Qwen2.5-Math-7B, #Train Params=-2026.06 | 77.6 | |
| ELM offlineBase Model=Qwen2.5-Math-7B, #Train Params=0.07M2026.06 | 73.4 | |
| ELM-LoRAofflineBase Model=Qwen2.5-Math-7B, #Train Params=40M2026.06 | 68.6 | |
| Full GTBase Model=Qwen2.5-Math-7B, #Train Params=7B2026.06 | 68 | |
| ELM-Full ModelofflineBase Model=Qwen2.5-Math-7B, #Train Params=7B2026.06 | 67.8 | |
| Full MajBase Model=Qwen2.5-Math-7B, #Train Params=7B2026.06 | 67.2 | |
| Zero-shot CoTBase Model=Qwen2.5-Math-7B, #Train Params=-2026.06 | 67 | |
| oracle ZS vs MemBase Model=Llama-3.1-8B-Instr., #Train Params=-2026.06 | 55.4 | |
| GRPOBase Model=Llama-3.1-8B-Instr., #Train Params=8B2026.06 | 47 | |
| ELM-LoRAofflineBase Model=Llama-3.1-8B-Instr., #Train Params=42M2026.06 | 46.2 | |
| ELM-Full ModelofflineBase Model=Llama-3.1-8B-Instr., #Train Params=8B2026.06 | 46 | |
| TTRLBase Model=Llama-3.1-8B-Instr., #Train Params=8B2026.06 | 45.6 | |
| Zero-shot CoTBase Model=Llama-3.1-8B-Instr., #Train Params=-2026.06 | 45 | |
| ELM offlineBase Model=Llama-3.1-8B-Instr., #Train Params=0.08M2026.06 | 45 |