Code Generation on MBPP (MBPP (%), Average (%))
85.2MBPP AccuracyCRDS-W
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CRDS-WModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 85.2 | 83.86 | |
| CRDS-WModel=Ling-mini-2.0 (16B), Data Selection Budget=3.5%, Truncation length=2048, Parallel runs=Average of 52026.02 | 84.26 | 83.96 | |
| CRDS-RModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 84.26 | 83.31 | |
| Ling-mini-2.0Model=Ling-mini-2.0 (16B), Data Selection Budget=100%, Truncation length=2048, Parallel runs=Average of 52026.02 | 84.07 | 83.25 | |
| Ling-mini-2.0Model=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=100%, Number of parallel runs=52026.02 | 84.07 | 83.25 | |
| Mid PPLModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 83.98 | 82.49 | |
| CRDS-RModel=Ling-mini-2.0 (16B), Data Selection Budget=3.5%, Truncation length=2048, Parallel runs=Average of 52026.02 | 83.75 | 83.26 | |
| RDS+Model=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 83.42 | 82.28 | |
| RandomModel=Ling-mini-2.0 (16B), Data Selection Budget=3.5%, Truncation length=2048, Parallel runs=Average of 52026.02 | 83.37 | 82.58 | |
| Mid PPLModel=Ling-mini-2.0 (16B), Data Selection Budget=3.5%, Truncation length=2048, Parallel runs=Average of 52026.02 | 83.28 | 81.87 | |
| RandomModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 83.28 | 82.55 | |
| RDS+Model=Ling-mini-2.0 (16B), Data Selection Budget=3.5%, Truncation length=2048, Parallel runs=Average of 52026.02 | 82.58 | 82.6 | |
| LengthModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 78.08 | 74.53 | |
| LengthModel=Ling-mini-2.0 (16B), Data Selection Budget=3.5%, Truncation length=2048, Parallel runs=Average of 52026.02 | 74.85 | 74.24 | |
| CodeUltraFeedback SpecialistTraining Type=Specialist2025.05 | 52.16 | 38.82 | |
| AMA ReweightingMechanism=Reweighting2025.05 | 51.44 | 41.75 | |
| AMA ResamplingMechanism=Resampling2025.05 | 51.08 | 40.19 | |
| Zephyr-7b-sft-fullBackbone=Zephyr-7b, Stage=SFT2025.05 | 49.9 | 36.08 | |
| Model Averaging2025.05 | 48.56 | 39.43 | |
| Chatbot Arena 2024 SpecialistTraining Type=Specialist2025.05 | 44.6 | 38.68 | |
| Standard2025.05 | 37.77 | 35.14 | |
| Standard UniformWeighting=Uniform2025.05 | 36.33 | 35.47 |