Code Generation on HumanEval (Accuracy Metric)
96.3HumanEval AccuracyWeak-to-Strong (AEA)
Evaluation results
| Method | Links | |
|---|---|---|
| Weak-to-Strong (AEA)Base Model=GPT-4o, Aligned=true, AEA Model=AEA-4B2026.05 | 96.3 | |
| Single AgentBase Model=GPT-4o, Aligned=false, AEA Model=None2026.05 | 95.7 | |
| Self-Reflection (AEA)Base Model=GPT-4o, Aligned=true, AEA Model=GPT-4o2026.05 | 94.5 | |
| Self-Reflection (AEA)Base Model=GPT-4o-mini, Aligned=true, AEA Model=GPT-4o-mini2026.05 | 93.3 | |
| Weak-to-Strong (AEA)Base Model=GPT-4o-mini, Aligned=true, AEA Model=AEA-4B2026.05 | 92.2 | |
| Multi AgentBase Model=GPT-4o-mini, Aligned=false, AEA Model=None2026.05 | 92 | |
| Single AgentBase Model=GPT-4o-mini, Aligned=false, AEA Model=None2026.05 | 91.5 | |
| Multi AgentBase Model=GPT-4o, Aligned=false, AEA Model=None2026.05 | 91.5 | |
| Weak-to-Strong (AEA)Base Model=Claude 3.7 Sonnet, Aligned=true, AEA Model=AEA-4B2026.05 | 90.9 | |
| Self-Reflection (AEA)Base Model=Claude 3.7 Sonnet, Aligned=true, AEA Model=Claude 3.7 Sonnet2026.05 | 89.6 | |
| Self-Reflection (AEA)Base Model=Claude 3.5 Sonnet, Aligned=true, AEA Model=Claude 3.5 Sonnet2026.05 | 89 | |
| Weak-to-Strong (AEA)Base Model=Claude 3.5 Sonnet, Aligned=true, AEA Model=AEA-4B2026.05 | 86.6 | |
| Multi AgentBase Model=Claude 3.7 Sonnet, Aligned=false, AEA Model=None2026.05 | 85.4 | |
| Multi AgentBase Model=Claude 3.5 Sonnet, Aligned=false, AEA Model=None2026.05 | 84.8 | |
| Single AgentBase Model=Claude 3.7 Sonnet, Aligned=false, AEA Model=None2026.05 | 84.8 | |
| Single AgentBase Model=Claude 3.5 Sonnet, Aligned=false, AEA Model=None2026.05 | 82.3 | |
| Multi AgentBase Model=Claude 3 Haiku, Aligned=false, AEA Model=None2026.05 | 79.8 | |
| Weak-to-Strong (AEA)Base Model=Claude 3 Haiku, Aligned=true, AEA Model=AEA-4B2026.05 | 79.3 | |
| SFTModel=DS-R1-Qwen-14B2026.04 | 77.4 | |
| D-QRELOModel=DS-R1-Qwen-14B2026.04 | 77.4 | |
| WandaModel=DS-R1-Qwen-14B2026.04 | 75.8 | |
| MagnitudeModel=DS-R1-Qwen-14B2026.04 | 75.2 | |
| BitDeltaModel=DS-R1-Qwen-14B2026.04 | 73.8 | |
| BackboneModel=DS-R1-Qwen-14B2026.04 | 72.6 | |
| Self-Reflection (AEA)Base Model=Claude 3 Haiku, Aligned=true, AEA Model=Claude 3 Haiku2026.05 | 72 | |
| RandomModel=DS-R1-Qwen-14B2026.04 | 69.5 | |
| SVDModel=DS-R1-Qwen-14B2026.04 | 69.5 | |
| MoBA (k=2)w_eff=1282026.04 | 68.3 | |
| MoBA (k=2)w_eff=2562026.04 | 67.1 | |
| Delta-CoMeModel=DS-R1-Qwen-14B2026.04 | 66.5 | |
| ISTBase Model=Qwen3-8B, Params=334M2026.05 | 65.9 | |
| Stochasticw_eff=2562026.04 | 65.2 | |
| DomLoRABase Model=Qwen3-8B, Params=2.1M2026.05 | 64.6 | |
| Fullw_eff=∞2026.04 | 64 | |
| Stochasticw_eff=1282026.04 | 64 | |
| PLoPBase Model=Qwen3-8B, Params=131M2026.05 | 62.8 | |
| Single AgentBase Model=Claude 3 Haiku, Aligned=false, AEA Model=None2026.05 | 61.6 | |
| LoRABase Model=Qwen3-8B, Params=334M2026.05 | 61 | |
| MoBA (k=2)w_eff=642026.04 | 60.4 | |
| DomLoRABase Model=LLaMA-3.1-8B, Params=2.3M2026.05 | 53 | |
| Stochasticw_eff=642026.04 | 52.4 | |
| SWAw_eff=2562026.04 | 52.4 | |
| LoRABase Model=LLaMA-3.1-8B, Params=321M2026.05 | 45.1 | |
| ISTBase Model=LLaMA-3.1-8B, Params=321M2026.05 | 45.1 | |
| PLoPBase Model=LLaMA-3.1-8B, Params=125M2026.05 | 43.3 | |
| Stochasticw_eff=322026.04 | 26.8 | |
| SWAw_eff=1282026.04 | 22.6 | |
| SWAw_eff=642026.04 | 3.7 | |
| SWAw_eff=322026.04 | 0 |