Code on HumanEval+
79.9AccuracySUN
Evaluation results
| Method | Links | |
|---|---|---|
| SUNBackbone=Qwen3-14B-Base, Decode Execution=Shared2026.03 | 79.9 | |
| Full-FTBackbone=Qwen3-14B-Base, Decode Execution=Independent2026.03 | 78 | |
| SUNBackbone=Qwen3-8B-Base, Decode Execution=Shared2026.03 | 76.2 | |
| Full-FTBackbone=Qwen3-8B-Base, Decode Execution=Independent2026.03 | 74.3 | |
| DARLBase=Base, Verifier=✗2026.01 | 73.2 | |
| AdaRASCategory=Steering2026.01 | 73.15 | |
| ProbingCategory=Steering2026.01 | 72.48 | |
| Qwen2.5-7B-InstBase=Base, Verifier=-2026.01 | 70.1 | |
| CoTPrompting=Vanilla CoT, Base Model=Qwen3-1.7B2026.01 | 69.8 | |
| VeriFreeBase=Base, Verifier=✗2026.01 | 65.9 | |
| RLVRBase=Base, Verifier=Rule2026.01 | 64 | |
| Qwen3-14B-BaseDecode Execution=N/A2026.03 | 62.3 | |
| RLPRBase=Inst, Verifier=✗2026.01 | 62.2 | |
| SimpleRL-ZooBase=Base, Verifier=Rule2026.01 | 62.2 | |
| DARLBase=Inst, Verifier=✗2026.01 | 61.6 | |
| Qwen3-8B-BaseDecode Execution=N/A2026.03 | 61.6 | |
| Full-FTBackbone=Qwen3-1.7B-Base, Decode Execution=Independent2026.03 | 59.8 | |
| RLPRBase=Base, Verifier=✗2026.01 | 59.1 | |
| SUNBackbone=Qwen3-1.7B-Base, Decode Execution=Shared2026.03 | 58.5 | |
| Llama3.1-8B-InstBase=Inst, Verifier=-2026.01 | 56.7 | |
| General ReasonerBase=Base, Verifier=Model2026.01 | 56.7 | |
| OpenThinker-3-1.5BCategory=Post-training2026.01 | 53.69 | |
| Oat-ZeroBase=Math, Verifier=Rule2026.01 | 53 | |
| SimpleRL-ZooBase=Math, Verifier=Rule2026.01 | 53 | |
| OpenReasoning-Nemotron-1.5BCategory=Post-training2026.01 | 51.68 | |
| DSRMidtraining Schedule=DSR2026.05 | 49.39 | |
| WSDMidtraining Schedule=WSD2026.05 | 46.95 | |
| Full-FTBackbone=LLaMA3.1-8B, Decode Execution=Independent2026.03 | 45.7 | |
| Unified 15B MoEMidtraining Corpus=4k→8k2026.05 | 45.12 | |
| RLVRBase=Inst, Verifier=Rule2026.01 | 44.5 | |
| TTRLBase=Base, Verifier=Rule2026.01 | 44.5 | |
| Unified 15B MoEMidtraining Corpus=1k→8k2026.05 | 43.9 | |
| Cosine-decayMidtraining Schedule=Cos2026.05 | 43.29 | |
| DeepSeek-R1-Distill-Qwen-1.5BCategory=Post-training2026.01 | 42.95 | |
| SUNBackbone=LLaMA3.1-8B, Decode Execution=Shared2026.03 | 41.5 | |
| Unified 15B MoEMidtraining Corpus=4k2026.05 | 41.46 | |
| Qwen3-1.7B-BaseDecode Execution=N/A2026.03 | 39.6 | |
| Qwen2.5-7BBase=-, Verifier=-2026.01 | 32.3 | |
| LLaMA3.1-8BDecode Execution=N/A2026.03 | 29.9 | |
| Unified 15B MoEPretraining Corpus=4k→8k2026.05 | 21.95 | |
| Unified 15B MoEPretraining Corpus=1k→8k2026.05 | 21.95 | |
| Unified 15B MoEPretraining Corpus=4k2026.05 | 20.12 | |
| PRIMEBase=Math, Verifier=Rule2026.01 | 3.7 |