Math Reasoning on MultiArith (test)
99.59AccuracyZERA
Evaluation Results
| Method | Links | |
|---|---|---|
| ZERAWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 99.59 | |
| aPSFWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 99.53 | |
| CriSPOWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 99.35 | |
| aPSFWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 99.3 | |
| CriSPOWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 98.94 | |
| OPROWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 98.75 | |
| APEWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 98.54 | |
| ProTeGiWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 98.54 | |
| GrIPSWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 98.54 | |
| GrIPSWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 98.54 | |
| ZERAWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 98.42 | |
| aPSFWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 98.33 | |
| ZERAWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 98.22 | |
| ZERAWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 98.15 | |
| ProTeGiWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 98.12 | |
| aPSFWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 97.92 | |
| CriSPOWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 97.9 | |
| ProTeGiWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 97.5 | |
| OPROWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 97.29 | |
| APEWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 96.88 | |
| GrIPSWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 96.88 | |
| APEWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 96.67 | |
| GrIPSWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 96.67 | |
| CriSPOWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 96.49 | |
| Zero-shot CoTWorker Model=Qwen2.5-7B-Instruct, Optimization Protocol=No optimization (p_init)2026.04 | 96.46 | |
| OPROWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 96.46 | |
| ProTeGiWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 96.46 | |
| Agent-GWOBackbone=Gemma-3-12b-it2026.04 | 95.9 | |
| Agent-GWOBackbone=GPT-4o-mini2026.04 | 95.3 | |
| Zero-shot CoTWorker Model=Llama-3.1-8B-Instruct, Optimization Protocol=No optimization (p_init)2026.04 | 95.21 | |
| OPROWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 95.21 | |
| Agent-GWOBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 93.3 | |
| AoTBackbone=GPT-4o-mini2026.04 | 92.6 | |
| AFlowBackbone=GPT-4o-mini2026.04 | 91.9 | |
| GoTBackbone=GPT-4o-mini2026.04 | 91 | |
| Self-RefineBackbone=GPT-4o-mini2026.04 | 89.9 | |
| ToTBackbone=GPT-4o-mini2026.04 | 89.8 | |
| CoT-SC/n=5Backbone=GPT-4o-mini2026.04 | 89.7 | |
| CoTBackbone=GPT-4o-mini2026.04 | 89.5 | |
| AFlowBackbone=Gemma-3-12b-it2026.04 | 89 | |
| AoTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 88.4 | |
| ToTBackbone=Gemma-3-12b-it2026.04 | 88.2 | |
| Self-RefineBackbone=Gemma-3-12b-it2026.04 | 87.5 | |
| AoTBackbone=Gemma-3-12b-it2026.04 | 87.5 | |
| CoT-SC/n=5Backbone=Qwen2.5-Coder-7B-Instruct2026.04 | 86.8 | |
| AFlowBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 86.8 | |
| GoTBackbone=Gemma-3-12b-it2026.04 | 86.8 | |
| APEWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 86.46 | |
| Self-RefineBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 86.4 | |
| GoTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 85.1 | |
| CoT-SC/n=5Backbone=Gemma-3-12b-it2026.04 | 85.1 | |
| CoTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 84.6 | |
| ToTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 83 | |
| CoTBackbone=Gemma-3-12b-it2026.04 | 82.7 |