Policy Perturbation Detection on Health Insurance Portability and Accountability Act (HIPAA)
7,210Detection RateDEEPSEEK-R1
Evaluation results
| Method | Links | |
|---|---|---|
| DEEPSEEK-R1Perturbation=Authorization, DEF Stage=DEF 32026.01 | 7,210 | |
| DEEPSEEK-R1Perturbation=Deontic Norm, DEF Stage=DEF 32026.01 | 5,880 | |
| DEEPSEEK-R1Perturbation=Deontic Norm, DEF Stage=DEF 22026.01 | 5,000 | |
| DEEPSEEK-R1Perturbation=Deontic Norm, DEF Stage=DEF 12026.01 | 4,930 | |
| DEEPSEEK-R1Perturbation=Authorization, DEF Stage=DEF 12026.01 | 3,240 | |
| DEEPSEEK-R1Perturbation=Authorization, DEF Stage=DEF 22026.01 | 2,790 | |
| GPT-5-MINIPerturbation=Authorization, DEF Stage=DEF 32026.01 | 880 | |
| GPT-5-MINIPerturbation=Deontic Norm, DEF Stage=DEF 32026.01 | 810 | |
| QWEN3-30B-THINKPerturbation=Deontic Norm, DEF Stage=DEF 32026.01 | 590 | |
| QWEN3-30B-THINKPerturbation=Authorization, DEF Stage=DEF 12026.01 | 440 | |
| QWEN3-30B-THINKPerturbation=Authorization, DEF Stage=DEF 22026.01 | 370 | |
| QWEN3-30B-THINKPerturbation=Authorization, DEF Stage=DEF 32026.01 | 370 | |
| QWEN3-30B-THINKPerturbation=Deontic Norm, DEF Stage=DEF 12026.01 | 290 | |
| GPT-5-MINIPerturbation=Authorization, DEF Stage=DEF 12026.01 | 290 | |
| DEEPSEEK-R1Perturbation=Deontic Norm, DEF Stage=Correct2026.01 | 180 | |
| DEEPSEEK-R1Perturbation=Authorization, DEF Stage=Correct2026.01 | 180 | |
| QWEN3-30B-THINKPerturbation=Deontic Norm, DEF Stage=DEF 22026.01 | 150 | |
| QWEN3-30B-THINKPerturbation=Authorization, DEF Stage=No DEF2026.01 | 150 | |
| GPT-5-MINIPerturbation=Deontic Norm, DEF Stage=DEF 22026.01 | 150 | |
| GPT-5-MINIPerturbation=Authorization, DEF Stage=DEF 22026.01 | 150 | |
| GPT-5-MINIPerturbation=Deontic Norm, DEF Stage=DEF 12026.01 | 70 | |
| GPT-5-MINIPerturbation=Authorization, DEF Stage=No DEF2026.01 | 70 | |
| QWEN3-30B-THINKPerturbation=Deontic Norm, DEF Stage=Correct2026.01 | 0 | |
| QWEN3-30B-THINKPerturbation=Deontic Norm, DEF Stage=No DEF2026.01 | 0 | |
| QWEN3-30B-THINKPerturbation=Authorization, DEF Stage=Correct2026.01 | 0 | |
| DEEPSEEK-R1Perturbation=Deontic Norm, DEF Stage=No DEF2026.01 | 0 | |
| DEEPSEEK-R1Perturbation=Authorization, DEF Stage=No DEF2026.01 | 0 | |
| GPT-5-MINIPerturbation=Deontic Norm, DEF Stage=Correct2026.01 | 0 | |
| GPT-5-MINIPerturbation=Deontic Norm, DEF Stage=No DEF2026.01 | 0 | |
| GPT-5-MINIPerturbation=Authorization, DEF Stage=Correct2026.01 | 0 |