Mathematical Reasoning on GSM8K (test) (Finetune Accuracy)
72.5Finetune AccuracyConstrainedSFT
Evaluation results
| Method | Links | |
|---|---|---|
| ConstrainedSFTn (sample number)=500, p (harmful ratio)=0.12026.02 | 72.5 | |
| Surgeryn (sample number)=500, p (harmful ratio)=0.12026.02 | 69.7 | |
| Lisan (sample number)=1000, p (harmful ratio)=0.12026.02 | 69.5 | |
| ConstrainedSFTn (sample number)=2500, p (harmful ratio)=0.12026.02 | 69.5 | |
| Lisan (sample number)=500, p (harmful ratio)=0.12026.02 | 69.3 | |
| SPARDn (sample number)=500, p (harmful ratio)=0.12026.02 | 68.9 | |
| AsFTn (sample number)=2500, p (harmful ratio)=0.12026.02 | 68.7 | |
| ConstrainedSFTn (sample number)=2000, p (harmful ratio)=0.12026.02 | 68.6 | |
| Surgeryn (sample number)=1000, p (harmful ratio)=0.12026.02 | 68.5 | |
| SFTn (sample number)=1500, p (harmful ratio)=0.12026.02 | 68.2 | |
| Lisan (sample number)=1500, p (harmful ratio)=0.12026.02 | 68.2 | |
| SafeGradn (sample number)=500, p (harmful ratio)=0.12026.02 | 68.2 | |
| AsFTn (sample number)=1000, p (harmful ratio)=0.12026.02 | 68.2 | |
| SPARDn (sample number)=1000, p (harmful ratio)=0.12026.02 | 68.2 | |
| AsFTn (sample number)=1500, p (harmful ratio)=0.12026.02 | 68 | |
| Surgeryn (sample number)=2000, p (harmful ratio)=0.12026.02 | 67.9 | |
| Surgeryn (sample number)=2500, p (harmful ratio)=0.12026.02 | 67.9 | |
| ConstrainedSFTn (sample number)=1500, p (harmful ratio)=0.12026.02 | 67.8 | |
| Lisan (sample number)=2000, p (harmful ratio)=0.12026.02 | 67.7 | |
| Surgeryn (sample number)=1500, p (harmful ratio)=0.12026.02 | 67.7 | |
| ConstrainedSFTn (sample number)=1000, p (harmful ratio)=0.12026.02 | 67.6 | |
| SFTn (sample number)=500, p (harmful ratio)=0.12026.02 | 67 | |
| SFTn (sample number)=2000, p (harmful ratio)=0.12026.02 | 66.7 | |
| SafeGradn (sample number)=2500, p (harmful ratio)=0.12026.02 | 66.7 | |
| AsFTn (sample number)=2000, p (harmful ratio)=0.12026.02 | 66.7 | |
| SafeGradn (sample number)=1500, p (harmful ratio)=0.12026.02 | 66.3 | |
| SPARDn (sample number)=2500, p (harmful ratio)=0.12026.02 | 66.3 | |
| SafeGradn (sample number)=2000, p (harmful ratio)=0.12026.02 | 65.9 | |
| Lisan (sample number)=2500, p (harmful ratio)=0.12026.02 | 65.6 | |
| SafeGradn (sample number)=1000, p (harmful ratio)=0.12026.02 | 65.3 | |
| SPARDn (sample number)=1500, p (harmful ratio)=0.12026.02 | 65.3 | |
| AsFTn (sample number)=500, p (harmful ratio)=0.12026.02 | 65.2 | |
| DSSn (sample number)=2000, p (harmful ratio)=0.12026.02 | 65.1 | |
| SFTn (sample number)=1000, p (harmful ratio)=0.12026.02 | 64.7 | |
| SPARDn (sample number)=2000, p (harmful ratio)=0.12026.02 | 64.3 | |
| SFTn (sample number)=2500, p (harmful ratio)=0.12026.02 | 63.4 | |
| DSSn (sample number)=1500, p (harmful ratio)=0.12026.02 | 61.3 | |
| DSSn (sample number)=2500, p (harmful ratio)=0.12026.02 | 60 | |
| DSSn (sample number)=500, p (harmful ratio)=0.12026.02 | 57.1 | |
| DSSn (sample number)=1000, p (harmful ratio)=0.12026.02 | 57.1 |