Arithmetic Reasoning on Matchstick Arithmetic (held-out)
90.7AccuracyCORE
Evaluation results
| Method | Links | |
|---|---|---|
| COREtraining samples=102026.05 | 90.7 | |
| COREtraining samples=52026.05 | 87.3 | |
| COREtraining samples=1002026.05 | 87 | |
| GEPAtraining samples=1002026.05 | 77 | |
| Episodic RAGtraining samples=52026.05 | 70.3 | |
| MemRLtraining samples=1002026.05 | 70.3 | |
| MemRLtraining samples=52026.05 | 70 | |
| GEPAtraining samples=102026.05 | 69.3 | |
| GEPAtraining samples=52026.05 | 68.7 | |
| No Learningtraining samples=52026.05 | 68.1 | |
| No Learningtraining samples=102026.05 | 68.1 | |
| No Learningtraining samples=1002026.05 | 68.1 | |
| MemRLtraining samples=102026.05 | 64.7 | |
| Episodic RAGtraining samples=1002026.05 | 64 | |
| GRPOtraining samples=102026.05 | 63.7 | |
| GRPOtraining samples=52026.05 | 63 | |
| Episodic RAGtraining samples=102026.05 | 62.7 | |
| GRPOtraining samples=1002026.05 | 59 |