Code Reasoning on CRUX official (test)
51.1Pass@1 AccuracyQwen2.5-7B-FULL
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen2.5-7B-FULLBackbone=Qwen2.5-7B, Data Selection Method=Full Dataset, Sample Size=17K2025.06 | 51.1 | |
| LearnAlignBackbone=Qwen2.5-7B, Data Selection Method=LearnAlign, Sample Size=1,0002025.06 | 44 | |
| PPL-MiddleBackbone=Qwen2.5-7B, Data Selection Method=PPL-Middle, Sample Size=1,0002025.06 | 43.3 | |
| PPL-TopBackbone=Qwen2.5-7B, Data Selection Method=PPL-Top, Sample Size=1,0002025.06 | 42.5 | |
| SelectITBackbone=Qwen2.5-7B, Data Selection Method=SelectIT, Sample Size=1,0002025.06 | 41.5 | |
| Random SamplingBackbone=Qwen2.5-7B, Data Selection Method=Random Sampling, Sample Size=1,0002025.06 | 40.8 | |
| LIMRBackbone=Qwen2.5-7B, Data Selection Method=LIMR, Sample Size=1,0002025.06 | 39.9 | |
| Token LengthBackbone=Qwen2.5-7B, Data Selection Method=Token Length, Sample Size=1,0002025.06 | 38.1 | |
| IFDBackbone=Qwen2.5-7B, Data Selection Method=IFD, Sample Size=1,0002025.06 | 34.9 | |
| Qwen2.5-7BBackbone=Qwen2.5-7B, Data Selection Method=Base Model (Zero-shot), Sample Size=N/A2025.06 | 25.1 | |
| Qwen2.5-3B-FULLBackbone=Qwen2.5-3B, Data Selection Method=Full Dataset, Sample Size=17K2025.06 | 24.3 | |
| LearnAlignBackbone=Qwen2.5-3B, Data Selection Method=LearnAlign, Sample Size=1,0002025.06 | 21 | |
| PPL-MiddleBackbone=Qwen2.5-3B, Data Selection Method=PPL-Middle, Sample Size=1,0002025.06 | 18.1 | |
| SelectITBackbone=Qwen2.5-3B, Data Selection Method=SelectIT, Sample Size=1,0002025.06 | 17.8 | |
| Random SamplingBackbone=Qwen2.5-3B, Data Selection Method=Random Sampling, Sample Size=1,0002025.06 | 17.4 | |
| LIMRBackbone=Qwen2.5-3B, Data Selection Method=LIMR, Sample Size=1,0002025.06 | 16.5 | |
| PPL-TopBackbone=Qwen2.5-3B, Data Selection Method=PPL-Top, Sample Size=1,0002025.06 | 16.1 | |
| IFDBackbone=Qwen2.5-3B, Data Selection Method=IFD, Sample Size=1,0002025.06 | 15.4 | |
| Token LengthBackbone=Qwen2.5-3B, Data Selection Method=Token Length, Sample Size=1,0002025.06 | 15.3 | |
| Qwen2.5-3BBackbone=Qwen2.5-3B, Data Selection Method=Base Model (Zero-shot), Sample Size=N/A2025.06 | 14.6 |