Code Generation on APPS Competition
38pass@1LCP
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LCPLLM backbone=GPT-4o2026.03 | 38 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LPWLLM backbone=GPT-4o2026.03 | 34.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LDBLLM backbone=GPT-4o2026.03 | 28.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LogitsCoderModel Category=Decoding-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 20 | — | — | — | — | — | — | — | — | — | — | 35 | — | |
| BaselineLLM backbone=GPT-4o2026.03 | 17.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| self-playModel Category=Reflection-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 17 | — | — | — | — | — | — | — | — | — | — | 30.17 | — | |
| Guided DecodingModel Category=Decoding-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 17 | — | — | — | — | — | — | — | — | — | — | 31.5 | — | |
| LATSModel Category=Search-guided Reasoning Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 13 | — | — | — | — | — | — | — | — | — | — | 23.86 | — | |
| MCTSModel Category=Search-guided Reasoning Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 13 | — | — | — | — | — | — | — | — | — | — | 27.33 | — | |
| RethinkMCTSModel Category=Search-guided Reasoning Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 13 | — | — | — | — | — | — | — | — | — | — | 27 | — | |
| Contrastive DecodingModel Category=Decoding-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 13 | — | — | — | — | — | — | — | — | — | — | 28.83 | — | |
| ReflexionModel Category=Reflection-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 12 | — | — | — | — | — | — | — | — | — | — | 24.5 | — | |
| LDBModel Category=Reflection-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 10 | — | — | — | — | — | — | — | — | — | — | 23.67 | — | |
| Zero-shotModel Category=Base, Backbone=Qwen2.5-14B-Instruct, Evaluation Protocol=Zero-shot2026.02 | 7 | — | — | — | — | — | — | — | — | — | — | 19.5 | — | |
| RAPModel Category=Reflection-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.02 | 7 | — | — | — | — | — | — | — | — | — | — | 24.83 | — | |
| CodexBackbone=Codex-12B, Filtering=Filtered, Shot=1-shot2021.07 | 3.04 | 5.25 | 3.08 | 5.53 | — | — | — | — | — | — | — | — | — | |
| CODETModel=code-davinci-002, Setting=zero-shot, Temperature=0.8, Sampling number=502022.07 | 2.2 | — | — | — | — | — | — | — | 4.1 | 8.6 | — | — | — | |
| BaselineModel=code-davinci-002, Setting=zero-shot, Temperature=0.8, Sampling number=502022.07 | 1.8 | — | — | — | — | — | — | — | — | 4.9 | 12.1 | — | — | |
| CodexBackbone=Codex-12B, Filtering=Raw, Shot=1-shot2021.07 | 0.02 | 0.03 | 0.09 | 0.16 | 1.05 | 5.85 | 3.23 | — | — | — | — | — | — | |
| GPT-NeoBackbone=GPT-Neo 2.7B, Filtering=Raw, Shot=Zero-shot2021.07 | 0 | — | 0 | — | — | — | — | — | — | — | — | — | — | |
| AgentScopeBackbone=MiMo-V2-Flash2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 48.42 | |
| AgentScopeBackbone=GPT-oss-120B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 58.71 | |
| AgentScopeBackbone=Llama3-8B-Instruct2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 14.29 | |
| AgentScopeBackbone=Qwen3-8B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 23.12 | |
| DyTopoBackbone=MiMo-V2-Flash2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 49.81 | |
| DyTopoBackbone=GPT-oss-120B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 69.66 | |
| DyTopoBackbone=Llama3-8B-Instruct2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 18.21 | |
| DyTopoBackbone=Qwen3-8B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 25.14 | |
| LLM OutputBackbone=MiMo-V2-Flash2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 41.18 | |
| LLM OutputBackbone=GPT-oss-120B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 31.88 | |
| LLM OutputBackbone=Llama3-8B-Instruct2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 10.13 | |
| LLM OutputBackbone=Qwen3-8B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 9.62 | |
| Random TopologyBackbone=MiMo-V2-Flash2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 42.01 | |
| Random TopologyBackbone=GPT-oss-120B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 52.21 | |
| Random TopologyBackbone=Llama3-8B-Instruct2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 12.44 | |
| Random TopologyBackbone=Qwen3-8B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 24.24 | |
| Single-turn AgentBackbone=MiMo-V2-Flash2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 42.84 | |
| Single-turn AgentBackbone=GPT-oss-120B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 60.55 | |
| Single-turn AgentBackbone=Llama3-8B-Instruct2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 13.28 | |
| Single-turn AgentBackbone=Qwen3-8B2026.02 | — | — | — | — | — | — | — | — | — | — | — | — | 13.47 |