Code-writing on HumanEval & MBPP EvalPlus (test)
39.02HumanEval Pass RateCRITIQ
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CRITIQBackbone=Llama-3.2-3B, Continual Pretraining=true, Data Selection Strategy=CRITIQ Scorer2025.02 | 39.02 | 33.54 | 68.73 | 48.41 | 53.88 | 40.98 | |
| QR-EduBackbone=Llama-3.2-3B, Continual Pretraining=true, Data Selection Strategy=QuRators (Educational Value)2025.02 | 36.59 | 32.32 | 59.26 | 47.35 | 47.93 | 39.84 | |
| StackBackbone=Llama-3.2-3B, Continual Pretraining=true, Data Selection Strategy=Stack v2 subset2025.02 | 31.71 | 27.44 | 56.61 | 46.3 | 44.16 | 36.87 | |
| RawBackbone=Llama-3.2-3B, Continual Pretraining=false2025.02 | 28.66 | 25.61 | 48.94 | 39.15 | 38.8 | 32.38 |