Commonsense Reasoning on Hellaswag (HS and Accuracy)
50HS ScorePre-unlearn
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Pre-unlearn2025.10 | 50 | — | |
| NPO w/ ZOAlgorithm=NPO, Optimizer=ZO2025.10 | 44.7 | — | |
| NoneModel=Mistral, Mixing ratio (r)=0.12026.03 | 44.5 | 94.1 | |
| BeavertailsModel=Qwen2.5, Mixing ratio (r)=0.12026.03 | 44.22 | 92.55 | |
| BeavertailsModel=Mistral, Mixing ratio (r)=0.12026.03 | 42.91 | 93.83 | |
| GradDiff w/ signAdamAlgorithm=GradDiff, Optimizer=signAdam2025.10 | 42.7 | — | |
| GradDiff w/ signSGDAlgorithm=GradDiff, Optimizer=signSGD2025.10 | 42.2 | — | |
| GradDiff w/ SAMAlgorithm=GradDiff, Optimizer=SAM2025.10 | 42.1 | — | |
| GradDiffAlgorithm=GradDiff2025.10 | 41.7 | — | |
| GradDiff w/ RSAlgorithm=GradDiff, Optimizer=RS2025.10 | 39.4 | — | |
| GradDiff w/ ZOAlgorithm=GradDiff, Optimizer=ZO2025.10 | 36.7 | — | |
| NPO w/ signSGDAlgorithm=NPO, Optimizer=signSGD2025.10 | 35.8 | — | |
| NPO w/ signAdamAlgorithm=NPO, Optimizer=signAdam2025.10 | 33.2 | — | |
| NPO w/ RSAlgorithm=NPO, Optimizer=RS2025.10 | 31.1 | — | |
| NPOAlgorithm=NPO2025.10 | 31 | — | |
| GradDiff w/ HybridAlgorithm=GradDiff, Optimizer=Hybrid2025.10 | 30.8 | — | |
| PLDRv51-SOC-110M-5Zero-shot=true, Criticality=SOC-110M-52026.03 | 30.55 | — | |
| GPT-Neo-125MZero-shot=true2026.03 | 30.4 | — | |
| BeavertailsModel=Llama3, Mixing ratio (r)=0.12026.03 | 30.35 | 91.98 | |
| NPO w/ SAMAlgorithm=NPO, Optimizer=SAM2025.10 | 29.7 | — | |
| PLDRv51-SOC-110M-2Zero-shot=true, Criticality=SOC-110M-22026.03 | 29.27 | — | |
| PLDRv51-SOC-110M-4Zero-shot=true, Criticality=SOC-110M-42026.03 | 29.13 | — | |
| PLDRv51-SOC-110M-3Zero-shot=true, Criticality=SOC-110M-32026.03 | 28.99 | — | |
| PLDRv51-SOC-110M-1Zero-shot=true, Criticality=SOC-110M-12026.03 | 28.65 | — | |
| NPO w/ HybridAlgorithm=NPO, Optimizer=Hybrid2025.10 | 28.4 | — | |
| ABL-SOC-110M-1Zero-shot=true2026.03 | 26.88 | — | |
| SUB-SOC-110M-1Zero-shot=true2026.03 | 26.03 | — | |
| ABL-SOC-110M-2Zero-shot=true2026.03 | 26.03 | — | |
| SUB-SOC-110M-2Zero-shot=true2026.03 | 25.93 | — | |
| ABL-SOC-110M-3Zero-shot=true2026.03 | 25.45 | — | |
| BeavertailsModel=OLMo2, Mixing ratio (r)=0.12026.03 | 16.43 | 88.47 | |
| NoneModel=Qwen2.5, Mixing ratio (r)=0.12026.03 | 15.15 | 92.81 | |
| GR-SAPModel=Mistral, Mixing ratio (r)=0.12026.03 | 14.11 | 93.78 | |
| AegisModel=Mistral, Mixing ratio (r)=0.12026.03 | 13.82 | 93.7 | |
| GR-SAPModel=Qwen2.5, Mixing ratio (r)=0.12026.03 | 10.69 | 92.65 | |
| AegisModel=Qwen2.5, Mixing ratio (r)=0.12026.03 | 8.72 | 92.56 | |
| NoneModel=Llama3, Mixing ratio (r)=0.12026.03 | 6.86 | 92.33 | |
| AegisModel=Llama3, Mixing ratio (r)=0.12026.03 | 2.71 | 92.22 | |
| AegisModel=OLMo2, Mixing ratio (r)=0.12026.03 | 2.22 | 88.57 | |
| NoneModel=OLMo2, Mixing ratio (r)=0.12026.03 | 0.81 | 88.76 | |
| GR-SAPModel=Llama3, Mixing ratio (r)=0.12026.03 | 0.76 | 91.62 | |
| OriginalModel=OLMo2, Mixing ratio (r)=0.12026.03 | 0.48 | 88.45 | |
| GR-SAPModel=OLMo2, Mixing ratio (r)=0.12026.03 | 0.47 | 88.42 | |
| BF16Model Size=1.3B, Precision=BF16, Zero-shot=true2025.01 | — | 50.8 | |
| BF16Model Size=7B, Precision=BF16, Zero-shot=true2025.01 | — | 52.03 | |
| BF16Model Size=13B, Precision=BF16, Zero-shot=true2025.01 | — | 53.56 | |
| FP4Model Size=1.3B, Precision=FP4, Zero-shot=true2025.01 | — | 50.82 | |
| FP4Model Size=7B, Precision=FP4, Zero-shot=true2025.01 | — | 52.97 | |
| FP4Model Size=13B, Precision=FP4, Zero-shot=true2025.01 | — | 54.12 |