Machine Unlearning on WMDP (Bio, Chem, Cyber Acc)
24.8Bio AccuracyCAP(Ours)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CAP(Ours)Model=Zephyr-7B2026.04 | 24.8 | 24.5 | 26.6 | |
| CAPModel=Yi-34B-Chat2026.04 | 25.9 | 24.8 | 23.3 | |
| CAPModel=o32026.04 | 29.5 | 22.1 | 25.8 | |
| CAPModel=Gemini-2.5-Flash2026.04 | 29.8 | 22.4 | 26.1 | |
| CAP(Ours)Model=Claude-Sonnet-42026.04 | 30.1 | 37.1 | 34.3 | |
| CAP(Ours)Model=DeepSeek-V32026.04 | 30.3 | 34.5 | 32.6 | |
| CAPModel=Qwen-2.5-Turbo2026.04 | 30.6 | 26.9 | 22.6 | |
| CAPModel=Claude-3.7-Sonnet2026.04 | 30.9 | 23.2 | 26.9 | |
| CAPModel=Doubao-Lite-32k2026.04 | 31.2 | 27.4 | 27.2 | |
| CAPModel=Gemini-1.5-Flash2026.04 | 31.5 | 23.6 | 27.3 | |
| CAPModel=GPT-4o mini2026.04 | 31.8 | 27.7 | 23.4 | |
| CAPModel=Doubao-1.5-Lite-32k2026.04 | 32 | 27 | 24.9 | |
| SPULModel=Zephyr-7B2026.04 | 32.1 | 30.6 | 30.9 | |
| CAP(Ours)Model=Qwen2.5-14B2026.04 | 32.5 | 22.3 | 28.9 | |
| CAP(Ours)Model=GLM-42026.04 | 32.5 | 27.8 | 29.5 | |
| CAPModel=Claude-3.5-Sonnet2026.04 | 32.6 | 28.1 | 24.3 | |
| CAPModel=o1-mini2026.04 | 33.1 | 27.9 | 25.6 | |
| ICULModel=DeepSeek-V32026.04 | 33.2 | 31.5 | 23.3 | |
| CAPModel=Qwen-2.5-max2026.04 | 33.3 | 28.8 | 25 | |
| NPOModel=Qwen2.5-14B2026.04 | 33.8 | 35.4 | 39.3 | |
| CAPModel=GPT-4 Turbo2026.04 | 34.3 | 26 | 29.3 | |
| ICULModel=GLM-42026.04 | 34.5 | 35.7 | 9.2 | |
| CAP(Ours)Model=GPT-4.12026.04 | 35.9 | 37.8 | 29.1 | |
| SPULModel=Qwen2.5-14B2026.04 | 36.2 | 33.8 | 35.6 | |
| CAP(Ours)Model=Qwen3-235B-A22B2026.04 | 36.3 | 29.5 | 32.4 | |
| ICULModel=GPT-4.12026.04 | 38.6 | 49.2 | 24.6 | |
| LLMUModel=Qwen2.5-14B2026.04 | 40 | 42.3 | 43.2 | |
| PromptingModel=Yi-34B-Chat2026.04 | 43 | 47.6 | 50.2 | |
| NPOModel=Zephyr-7B2026.04 | 43.1 | 34.8 | 30.6 | |
| ICULModel=Zephyr-7B2026.04 | 44.9 | 33.1 | 15.9 | |
| ICULModel=Qwen2.5-14B2026.04 | 46 | 39.2 | 24.8 | |
| ICULModel=Qwen3-235B-A22B2026.04 | 46.7 | 32.6 | 30.4 | |
| ICULModel=Claude-Sonnet-42026.04 | 48.4 | 46.1 | 28.9 | |
| PromptingModel=o1-mini2026.04 | 52.6 | 60.2 | 55.1 | |
| PromptingModel=Zephyr-7B2026.04 | 53.1 | 36 | 35.4 | |
| PromptingModel=GPT-4 Turbo2026.04 | 53.2 | 61.1 | 55.9 | |
| LLMUModel=Zephyr-7B2026.04 | 56.1 | 39.2 | 36.2 | |
| ICULModel=Yi-34B-Chat2026.04 | 56.5 | 36 | 47.2 | |
| PromptingModel=GPT-4o mini2026.04 | 60.4 | 56.5 | 51.2 | |
| PromptingModel=Qwen-2.5-Turbo2026.04 | 62.5 | 57 | 41.9 | |
| PromptingModel=Qwen-2.5-max2026.04 | 62.8 | 60.4 | 55.3 | |
| OriginalModel=Qwen-2.5-Turbo2026.04 | 63.2 | 69.8 | 48.5 | |
| OriginalModel=GPT-4o mini2026.04 | 63.6 | 56.9 | 52.5 | |
| OriginalModel=Zephyr-7B2026.04 | 63.7 | 44.3 | 45.8 | |
| PromptingModel=Gemini-1.5-Flash2026.04 | 65.2 | 71.7 | 56.5 | |
| PromptingModel=Claude-3.7-Sonnet2026.04 | 65.5 | 63.2 | 58.2 | |
| PromptingModel=GLM-42026.04 | 66.1 | 48.5 | 54.1 | |
| OriginalModel=o1-mini2026.04 | 67.4 | 63.1 | 56.5 | |
| OriginalModel=GPT-4 Turbo2026.04 | 68.3 | 58 | 57.1 | |
| PromptingModel=DeepSeek-V32026.04 | 68.4 | 65.3 | 58.7 | |
| PromptingModel=Qwen2.5-14B2026.04 | 68.7 | 47.3 | 46.7 | |
| PromptingModel=Doubao-Lite-32k2026.04 | 68.8 | 68.1 | 63 | |
| OriginalModel=Gemini-1.5-Flash2026.04 | 69.3 | 64.5 | 57.9 | |
| OriginalModel=Doubao-Lite-32k2026.04 | 74.5 | 70.9 | 64.6 | |
| OriginalModel=Qwen2.5-14B2026.04 | 74.6 | 57.8 | 60.5 | |
| OriginalModel=Yi-34B-Chat2026.04 | 76.2 | 56.9 | 56.9 | |
| OriginalModel=Qwen-2.5-max2026.04 | 77.6 | 63.3 | 56.7 | |
| OriginalModel=GLM-42026.04 | 78.2 | 64.6 | 55.2 | |
| OriginalModel=Claude-3.7-Sonnet2026.04 | 79.3 | 66 | 62.6 | |
| OriginalModel=Claude-3.5-Sonnet2026.04 | 79.5 | 65.2 | 58.7 | |
| PromptingModel=Claude-3.5-Sonnet2026.04 | 80.6 | 62.4 | 57.3 | |
| PromptingModel=Claude-Sonnet-42026.04 | 80.6 | 64.4 | 63.8 | |
| PromptingModel=Doubao-1.5-Lite-32k2026.04 | 81 | 69.3 | 64.1 | |
| PromptingModel=Qwen3-235B-A22B2026.04 | 82.1 | 64.2 | 61.7 | |
| OriginalModel=Claude-Sonnet-42026.04 | 83.2 | 70.6 | 66.5 | |
| PromptingModel=o32026.04 | 83.3 | 68.7 | 68.5 | |
| OriginalModel=Qwen3-235B-A22B2026.04 | 83.5 | 65.9 | 61.9 | |
| OriginalModel=DeepSeek-V32026.04 | 83.6 | 69.9 | 67.5 | |
| PromptingModel=Gemini-2.5-Flash2026.04 | 84 | 70.5 | 69.4 | |
| PromptingModel=GPT-4.12026.04 | 85 | 71.6 | 66.1 | |
| OriginalModel=Doubao-1.5-Lite-32k2026.04 | 85.7 | 72.1 | 65.7 | |
| OriginalModel=o32026.04 | 87.2 | 72.8 | 70.3 | |
| OriginalModel=Gemini-2.5-Flash2026.04 | 88.8 | 72.3 | 69.8 | |
| OriginalModel=GPT-4.12026.04 | 88.8 | 74 | 66.5 |