Code Generation on APPS (Introductory, Interview, Competition)
45AccuracyMC-Tree-Of-Agents
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MC-Tree-Of-Agents2025.02 | 45 | — | — | — | 59.17 | |
| MC-Tree-Of-AgentsStrategy=Refine2025.02 | 44 | — | — | — | 60.33 | |
| SingleBackbone=Claude2025.02 | 42 | — | — | — | 54.67 | |
| MC-Tree-Of-AgentsStrategy=Pruning2025.02 | 38 | — | — | — | 54 | |
| MetaAgent-X RLTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 38 | — | — | — | — | |
| SA + GRPOTraining Paradigm=Single Agent, Backbone=Qwen3 8B2026.05 | 37 | — | — | — | — | |
| MetaAgent-X RLTraining Paradigm=RL-based Auto MAS2026.05 | 36.7 | — | — | — | — | |
| SA + GRPOTraining Paradigm=Single Agent2026.05 | 35.6 | — | — | — | — | |
| MetaAgent-X SFTTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 32 | — | — | — | — | |
| MetaAgent-X SFTTraining Paradigm=RL-based Auto MAS2026.05 | 32 | — | — | — | — | |
| SATraining Paradigm=Single Agent, Backbone=Qwen3 8B2026.05 | 30.2 | — | — | — | — | |
| MaASTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 30 | — | — | — | — | |
| RethinkMCTS2025.02 | 28 | — | — | — | 42.5 | |
| AFM-CoderTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 28 | — | — | — | — | |
| ADASTraining Paradigm=Search-based Auto MAS2026.05 | 28 | — | — | — | — | |
| AFlowTraining Paradigm=Search-based Auto MAS, Backbone=Qwen3 8B2026.05 | 27.4 | — | — | — | — | |
| SATraining Paradigm=Single Agent2026.05 | 27.4 | — | — | — | — | |
| ADASTraining Paradigm=Search-based Auto MAS, Backbone=Qwen3 8B2026.05 | 27 | — | — | — | — | |
| ScoreFlowTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 26.5 | — | — | — | — | |
| SingleBackbone=GPT4omini2025.02 | 25 | — | — | — | 44.17 | |
| ScoreFlowTraining Paradigm=RL-based Auto MAS2026.05 | 24.5 | — | — | — | — | |
| MaASTraining Paradigm=RL-based Auto MAS2026.05 | 23.75 | — | — | — | — | |
| AFlowTraining Paradigm=Search-based Auto MAS2026.05 | 23.2 | — | — | — | — | |
| ToT2025.02 | 11 | — | — | — | 26.3 | |
| ZeroShot2025.02 | 9 | — | — | — | 23.67 | |
| LDB2025.02 | 8 | — | — | — | 21 | |
| RAP2025.02 | 8 | — | — | — | 22.83 | |
| Reflexion2025.02 | 7 | — | — | — | 17.5 | |
| LATS2025.02 | 7 | — | — | — | 21.83 | |
| AlphaCode 1BFiltered From (k)=N/A, Attempts (n)=10002022.02 | — | 17.67 | 5.24 | 7.06 | — | |
| AlphaCode 1BFiltered From (k)=1000, Attempts (n)=52022.02 | — | 14.36 | 5.63 | 4.58 | — | |
| AlphaCode 1BFiltered From (k)=10000, Attempts (n)=52022.02 | — | 18.18 | 8.21 | 6.65 | — | |
| AlphaCode 1BFiltered From (k)=50000, Attempts (n)=52022.02 | — | 20.36 | 9.66 | 7.75 | — | |
| Codex 12BFiltered From (k)=N/A, Attempts (n)=12022.02 | — | 4.14 | 0.14 | 0.02 | — | |
| Codex 12BFiltered From (k)=N/A, Attempts (n)=52022.02 | — | 9.65 | 0.51 | 0.09 | — | |
| Codex 12BFiltered From (k)=N/A, Attempts (n)=10002022.02 | — | 25.02 | 3.7 | 3.23 | — | |
| Codex 12BFiltered From (k)=1000, Attempts (n)=12022.02 | — | 22.78 | 2.64 | 3.04 | — | |
| Codex 12BFiltered From (k)=1000, Attempts (n)=52022.02 | — | 24.52 | 3.23 | 3.08 | — | |
| GPT-Neo 2.7BFiltered From (k)=N/A, Attempts (n)=12022.02 | — | 3.9 | 0.57 | 0 | — | |
| GPT-Neo 2.7BFiltered From (k)=N/A, Attempts (n)=52022.02 | — | 5.5 | 0.8 | 0 | — |