Coding on HumanEval, MBPP
50.2HumanEval ScoreD3
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| D3Paradigm=Ours, Backbone=Llama-3-8B, Evaluation Protocol=SFT2026.05 | 50.2 | 54.6 | 49.6 | |
| DSL (Baseline)Paradigm=Training, Backbone=Llama-3-8B, Evaluation Protocol=SFT2026.05 | 46.5 | 51.2 | 46.5 | |
| Loss low→highParadigm=Curriculum, Backbone=Llama-3-8B, Evaluation Protocol=SFT2026.05 | 45.8 | 50.5 | 46.2 | |
| MTLParadigm=Training, Backbone=Llama-3-8B, Evaluation Protocol=SFT2026.05 | 44.2 | 49.8 | 44.9 | |
| Loss high→lowParadigm=Curriculum, Backbone=Llama-3-8B, Evaluation Protocol=SFT2026.05 | 38.4 | 42.1 | 34.9 | |
| ZIPBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 20.73 | 22.2 | — | |
| ADGBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 20.73 | 24.8 | — | |
| ADGBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 20.62 | 24.2 | — | |
| RethinkingBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 20.12 | 23.2 | — | |
| SuperFilteringBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 20.12 | 24.4 | — | |
| RethinkingBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 20.12 | 22.4 | — | |
| SelectITBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 19.51 | 24.8 | — | |
| ZIPBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 19.51 | 23.4 | — | |
| IFDBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 19.51 | 23.2 | — | |
| IFDBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 18.9 | 24.2 | — | |
| IFDBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 18.9 | 22.4 | — | |
| SelectITBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 18.9 | 22.6 | — | |
| SelectITBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 18.29 | 23.2 | — | |
| MIGBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 18.26 | 21.8 | — | |
| SuperFilteringBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 17.68 | 26.2 | — | |
| MIGBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 17.68 | 23.6 | — | |
| SCARBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 17.68 | 22.4 | — | |
| All DataBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 17.68 | 21.2 | — | |
| RandomBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 17.68 | 24.8 | — | |
| RandomBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 17.68 | 24 | — | |
| SuperFilteringBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 17.68 | 25.4 | — | |
| All DataBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 17.07 | 21.8 | — | |
| RandomBackbone=LLaMA3.2-1B-Instruct, Selection Pool=Alpaca-GPT4(52K), Selection Budget=10K2026.04 | 17.07 | 22.2 | — | |
| SCARBackbone=LLaMA3.2-1B-Instruct, Selection Pool=WizardLM(70K), Selection Budget=10K2026.04 | 17.07 | 21.6 | — | |
| RethinkingBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 17.07 | 23 | — | |
| ADGBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 17.07 | 24.4 | — | |
| ZIPBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 15.85 | 26.2 | — | |
| MIGBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 15.24 | 22.8 | — | |
| SCARBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 15.24 | 22.2 | — | |
| All DataBackbone=LLaMA3.2-1B-Instruct, Selection Pool=CoT(100K), Selection Budget=10K2026.04 | 11.59 | 15.2 | — |