Language Understanding on MMLU (Score x100)
65.81MMLU Score (x100)UNIFORM RANDOM (50K×0.6)
Evaluation results
| Method | Links | |
|---|---|---|
| UNIFORM RANDOM (50K×0.6)Base Model=LLaMA-3.1-8B, Token Budget=50K, Selection Proportion=0.62025.02 | 65.81 | |
| FULL TOKENS (50K)Base Model=LLaMA-3.1-8B, Token Budget=50K2025.02 | 65.78 | |
| DS² (10K)Base Model=LLaMA-3.1-8B, Token Budget=10K2025.02 | 65.77 | |
| RHOBase Model=LLaMA-3.1-8B, Selection Proportion=0.62025.02 | 65.74 | |
| FIXED-MODEL CLEANINGBase Model=LLaMA-3.1-8B, Selection Proportion=0.62025.02 | 65.71 | |
| BASEBase Model=LLaMA-3.1-8B2025.02 | 65.29 | |
| SELF-EVOLVING CLEANINGBase Model=LLaMA-3.1-8B, Selection Proportion=0.62025.02 | 65.07 | |
| DS² (10K)Base Model=Mistral-7B-v0.3, Token Budget=10K2025.02 | 62.5 | |
| UNIFORM RANDOM (50K×0.6)Base Model=Mistral-7B-v0.3, Token Budget=50K, Selection Proportion=0.62025.02 | 62.47 | |
| BASEBase Model=Mistral-7B-v0.32025.02 | 62.41 | |
| FULL TOKENS (50K)Base Model=Mistral-7B-v0.3, Token Budget=50K2025.02 | 62.41 | |
| SELF-EVOLVING CLEANINGBase Model=Mistral-7B-v0.3, Selection Proportion=0.62025.02 | 62.3 | |
| RHOBase Model=Mistral-7B-v0.3, Selection Proportion=0.62025.02 | 62.12 | |
| FIXED-MODEL CLEANINGBase Model=Mistral-7B-v0.3, Selection Proportion=0.62025.02 | 61.45 | |
| RHOBase Model=LLaMA-3.2-3B, Selection Proportion=0.62025.02 | 57.1 | |
| FIXED-MODEL CLEANINGBase Model=LLaMA-3.2-3B, Selection Proportion=0.62025.02 | 57.09 | |
| UNIFORM RANDOM (50K×0.6)Base Model=LLaMA-3.2-3B, Token Budget=50K, Selection Proportion=0.62025.02 | 56.96 | |
| DS² (10K)Base Model=LLaMA-3.2-3B, Token Budget=10K2025.02 | 56.93 | |
| FULL TOKENS (50K)Base Model=LLaMA-3.2-3B, Token Budget=50K2025.02 | 56.87 | |
| BASEBase Model=LLaMA-3.2-3B2025.02 | 56.29 | |
| SELF-EVOLVING CLEANINGBase Model=LLaMA-3.2-3B, Selection Proportion=0.62025.02 | 56.18 |