Coding on HumanEval+
95.12Pass@1Gemini 3-Pro
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemini 3-Pro2026.02 | 95.12 | — | — | — | |
| Gemini 2.5-Pro2026.02 | 94.5 | — | — | — | |
| ERNIE 5.02026.02 | 94.48 | — | — | — | |
| GPT-5 (High)2026.02 | 92.7 | — | — | — | |
| RL initialization modelPass@k protocol=@12026.05 | 92.68 | — | — | — | |
| Solar-Open-100BReasoning=On2026.03 | 92.07 | — | — | — | |
| Mi:dm K 2.5 Pro (March ‘26)Reasoning=On2026.03 | 92.07 | — | — | — | |
| RDPOPass@k protocol=@12026.05 | 91.46 | — | — | — | |
| K-EXAONE-236B-A23BReasoning=On2026.03 | 90.85 | — | — | — | |
| DS V3.2-Thinking2026.02 | 90.8 | — | — | — | |
| JT-Safe-V2-35BParameters=35B2026.05 | 90.24 | — | — | — | |
| LLaDA2.1-flashInference Mode=S Mode2026.02 | 89.63 | 13.81 | — | — | |
| LLaDA2.1-flashInference Mode=Q Mode2026.02 | 89.63 | 9.18 | — | — | |
| Qwen 3 VL 32B InstructParameters=32B2025.12 | 89.3 | — | — | — | |
| GRPOPass@k protocol=@12026.05 | 89.02 | — | — | — | |
| SOTA with Equivalent ParametersModel comparison=Equivalent Parameters2026.05 | 89 | — | — | — | |
| LLaDA2.0-flash2026.02 | 88.41 | 6.45 | — | — | |
| HyperCLOVAX-SEED-Think-32BReasoning=On2026.03 | 88.41 | — | — | — | |
| Qwen-3-30B-A3BReasoning=Off2026.03 | 88.4 | — | — | — | |
| Qwen3-30B-A3B-Inst-25072026.02 | 87.88 | 1 | — | — | |
| Ling-flash-2.02026.02 | 87.58 | 1 | — | — | |
| Qwen-3-30B-A3BReasoning=On2026.03 | 87.2 | — | — | — | |
| Olmo 3.1 32B InstructStage=Final Instruct 3.12025.12 | 86.7 | — | — | — | |
| KodCode-RL Step 256Size=7B-Inst, Training Setting=Training-based2026.05 | 86 | — | — | — | |
| Olmo 3.1 32B InstructStage=DPO2025.12 | 85.7 | — | — | — | |
| HASP-Intervention (w. Teacher)Size=7B-Inst, Training Setting=Training-free / inference-time, Backbone=Qwen2.5-7B-Instruct2026.05 | 85 | — | — | — | |
| HASP-Evolve + RSSize=7B-Inst, Training Setting=Training-based, Backbone=Qwen2.5-7B-Instruct2026.05 | 84.5 | — | — | — | |
| Qwen 3 32BThinking=No, Parameters=32B2025.12 | 83.9 | — | — | — | |
| K-EXAONE-236B-A23BReasoning=Off2026.03 | 83.5 | — | — | — | |
| KodCode-RL Step 128Size=7B-Inst, Training Setting=Training-based2026.05 | 83.5 | — | — | — | |
| LLaDA2.1-minimode=Q Mode2026.02 | 82.93 | 7.77 | — | — | |
| Mi:dm K 2.5 Pro (March ‘26)Reasoning=Off2026.03 | 82.9 | — | — | — | |
| Qwen 2.5 32BParameters=32B2025.12 | 82.6 | — | — | — | |
| HASP-Intervention (PF-only)Size=7B-Inst, Training Setting=Training-free / inference-time, Backbone=Qwen2.5-7B-Instruct2026.05 | 82 | — | — | — | |
| LLaDA2.0-mini2026.02 | 81.71 | 5.16 | — | — | |
| GRPO (Code)Size=7B-Inst, Training Setting=Training-based2026.05 | 81.1 | — | — | — | |
| P-GRPO (Code+RM)Size=7B-Inst, Training Setting=Training-based2026.05 | 81.1 | — | — | — | |
| GPT-4oSize=~200B, Training Setting=Training-free / inference-time2026.05 | 81 | — | — | — | |
| Olmo 3.1 32B InstructStage=SFT2025.12 | 80.8 | — | — | — | |
| LLaDA2.1-minimode=S Mode2026.02 | 80.49 | 12.32 | — | — | |
| Ling-mini-2.02026.02 | 80.03 | — | — | — | |
| GPT-4o-miniSize=~8B, Training Setting=Training-free / inference-time2026.05 | 80 | — | — | — | |
| Qwen3-8Bno think=true2026.02 | 79.5 | — | — | — | |
| Gemma 3 27BParameters=27B2025.12 | 79.2 | — | — | — | |
| AceCoderRMSize=7B-Inst, Training Setting=Training-based2026.05 | 77.4 | — | — | — | |
| AceCoderRuleSize=7B-Inst, Training Setting=Training-based2026.05 | 77.4 | — | — | — | |
| SFT (vanilla)Size=7B-Inst, Training Setting=Training-based2026.05 | 77 | — | — | — | |
| Prompt-Only SkillsSize=7B-Inst, Training Setting=Training-free / inference-time2026.05 | 75 | — | — | — | |
| ReasonerBackbone=Qwen2.5-7B-Base2026.06 | 74.39 | — | — | — | |
| HyperCLOVAX-SEED-Think-32BReasoning=Off2026.03 | 74.3 | — | — | — | |
| Qwen2.5-7B-InstructSize=7B-Inst, Training Setting=Training-free / inference-time2026.05 | 73.2 | — | — | — | |
| ISO-CBackbone=Qwen2.5-7B-Base2026.06 | 73.17 | — | — | — | |
| ISO-CTSBackbone=Qwen2.5-7B-Base2026.06 | 72.56 | — | — | — | |
| SimpleRLBackbone=Qwen2.5-7B-Base2026.06 | 71.34 | — | — | — | |
| RAMBackbone=Qwen2.5-7B-Base2026.06 | 71.34 | — | — | — | |
| RESMERGEBackbone=Qwen2.5-7B-Base, configuration=SRC-A+LHC2026.06 | 71.34 | — | — | — | |
| Qwen2.5-7BBackbone=Qwen2.5-7B-Base2026.06 | 70.73 | — | — | — | |
| TABackbone=Qwen2.5-7B-Base2026.06 | 70.73 | — | — | — | |
| TIESBackbone=Qwen2.5-7B-Base2026.06 | 70.12 | — | — | — | |
| TSV-MergeBackbone=Qwen2.5-7B-Base2026.06 | 70.12 | — | — | — | |
| BaseBackbone=Qwen2.5-3B-Instruct2026.05 | 67.7 | — | — | — | |
| BaseBackbone=Qwen2.5-3B-Instruct2026.05 | 67.7 | — | — | — | |
| BaseBackbone=Qwen2.5-3B-Instruct, Incremental Learning Stage=Pre-trained2026.05 | 67.7 | — | — | — | |
| Gemma 2 27BParameters=27B2025.12 | 67.5 | — | — | — | |
| Self-sftBackbone=Qwen2.5-3B-Instruct2026.05 | 67.1 | — | — | — | |
| DARE + TIESBackbone=Qwen2.5-7B-Base2026.06 | 67.07 | — | — | — | |
| Self-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 66.5 | — | — | — | |
| Anchored LearningBackbone=Qwen2.5-3B-Instruct, Incremental Learning Stage=Stage 3: iGSM -> MedCalc -> IFEval2026.05 | 66.5 | — | — | — | |
| Low-SFTBackbone=Qwen2.5-3B-Instruct, Incremental Learning Stage=Stage 3: iGSM -> MedCalc -> IFEval2026.05 | 65.2 | — | — | — | |
| Low-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 64.6 | — | — | — | |
| STMBackbone=Qwen2.5-3B-Instruct2026.05 | 64.6 | — | — | — | |
| Iter-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 64.6 | — | — | — | |
| Low-SFTBackbone=Qwen2.5-3B-Instruct, Incremental Learning Stage=Stage 1: iGSM2026.05 | 64.6 | — | — | — | |
| Anchored LearningBackbone=Qwen2.5-3B-Instruct2026.05 | 64 | — | — | — | |
| Anchored LearningBackbone=Qwen2.5-3B-Instruct, Incremental Learning Stage=Stage 1: iGSM2026.05 | 64 | — | — | — | |
| Original (T2T)Backbone=LLaDA2.1-mini, Inference Strategy=Text-to-Text2026.04 | 64 | — | — | — | |
| UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.11 | 63.05 | — | — | — | |
| Anchored LearningBackbone=Qwen2.5-3B-Instruct2026.05 | 62.8 | — | — | — | |
| UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.11 | 62.78 | — | — | — | |
| RA-Agent (multi-loop)Size=7B-Inst, Training Setting=Training-free / inference-time2026.05 | 62 | — | — | — | |
| TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.11 | 61.83 | — | — | — | |
| SafeChainbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 61.6 | — | — | — | |
| T2MBackbone=LLaDA2.1-mini, Inference Strategy=Token-to-Mask, Remasking Strategy=LOWPROB, τ=0.3, Cmax=1, ρmax=0.252026.04 | 61 | — | — | — | |
| UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.11 | 60.49 | — | — | — | |
| UnSafeChainbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 60.4 | — | — | — | |
| Low-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 60.4 | — | — | — | |
| PreSafebase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 60.3 | — | — | — | |
| STMBackbone=Qwen2.5-3B-Instruct2026.05 | 59.8 | — | — | — | |
| Low-SFTBackbone=Qwen2.5-3B-Instruct, Incremental Learning Stage=Stage 2: iGSM -> MedCalc2026.05 | 59.1 | — | — | — | |
| Anchored LearningBackbone=Qwen2.5-3B-Instruct, Incremental Learning Stage=Stage 2: iGSM -> MedCalc2026.05 | 57.9 | — | — | — | |
| ZeroBackbone=Qwen2.5-7B-Base2026.06 | 57.32 | — | — | — | |
| Youtu-LLMSize=2B, Type=Base, Prompt Setting=0-shot2025.12 | 57.3 | — | — | — | |
| Iter-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 56.1 | — | — | — | |
| SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.11 | 55.85 | — | — | — | |
| DFTBackbone=Qwen2.5-3B-Instruct2026.05 | 53 | — | — | — | |
| Anchored LearningBackbone=Llama-3.2-3B-Instruct2026.05 | 51.2 | — | — | 52 | |
| Iter-SFTBackbone=Llama-3.2-3B-Instruct2026.05 | 50.9 | — | — | — | |
| Iter-SFTBackbone=Llama-3.2-3B-Instruct2026.05 | 50.3 | — | — | 39.6 | |
| BaseBackbone=Llama-3.2-3B-Instruct2026.05 | 50 | — | — | — | |
| Anchored LearningBackbone=Llama-3.2-3B-Instruct2026.05 | 50 | — | — | — |