Biosecurity Misuse Evaluation on BSD Biosecurity
1.3Misuse RateDirect query
Evaluation results
| Method | Links | |
|---|---|---|
| Direct queryTarget model=GPT-52025.06 | 1.3 | |
| AdaptiveTarget model=GPT-52025.06 | 1.3 | |
| Direct queryTarget model=Claude-3.5-Sonnet2025.06 | 3 | |
| CrescendoTarget model=GPT-52025.06 | 13 | |
| PAIRTarget model=GPT-52025.06 | 13.3 | |
| Direct queryTarget model=Claude-3.7-Sonnet2025.06 | 15 | |
| Adversarial ReasoningTarget model=GPT-52025.06 | 18.7 | |
| CrescendoTarget model=Claude-3.5-Sonnet2025.06 | 20.7 | |
| AdaptiveTarget model=Claude-3.5-Sonnet2025.06 | 27.3 | |
| Direct queryTarget model=o32025.06 | 31.3 | |
| AdaptiveTarget model=o32025.06 | 32.7 | |
| PAIRTarget model=Claude-3.5-Sonnet2025.06 | 35.3 | |
| Decomposition AttackTarget model=Claude-3.5-Sonnet2025.06 | 41.6 | |
| Direct queryTarget model=GPT-4o2025.06 | 42 | |
| AdaptiveTarget model=GPT-4o2025.06 | 42 | |
| Decomposition AttackTarget model=GPT-52025.06 | 45.3 | |
| Decomposition Attack (New)Target model=Claude-3.5-Sonnet2025.06 | 46 | |
| PAIRTarget model=o32025.06 | 46 | |
| Adversarial ReasoningTarget model=Claude-3.5-Sonnet2025.06 | 46.7 | |
| Decomposition Attack (New)Target model=GPT-52025.06 | 50.6 | |
| Decomposition AttackTarget model=o32025.06 | 52 | |
| CrescendoTarget model=Claude-3.7-Sonnet2025.06 | 52.7 | |
| Decomposition AttackTarget model=Claude-3.7-Sonnet2025.06 | 52.8 | |
| CrescendoTarget model=o32025.06 | 53.3 | |
| Adversarial ReasoningTarget model=o32025.06 | 56.7 | |
| PAIRTarget model=Claude-3.7-Sonnet2025.06 | 62.7 | |
| PAIRTarget model=GPT-4o2025.06 | 64.7 | |
| Direct queryTarget model=Gemini-2.5-pro2025.06 | 64.7 | |
| Adversarial ReasoningTarget model=Claude-3.7-Sonnet2025.06 | 65.3 | |
| CrescendoTarget model=GPT-4o2025.06 | 65.3 | |
| Decomposition Attack (New)Target model=Claude-3.7-Sonnet2025.06 | 65.6 | |
| AdaptiveTarget model=Claude-3.7-Sonnet2025.06 | 67.3 | |
| Decomposition AttackTarget model=GPT-4o2025.06 | 68.4 | |
| Decomposition Attack (New)Target model=o32025.06 | 68.8 | |
| Adversarial ReasoningTarget model=GPT-4o2025.06 | 70.7 | |
| Decomposition Attack (New)Target model=GPT-4o2025.06 | 74 | |
| Direct queryTarget model=o3-mini2025.06 | 77.7 | |
| Decomposition AttackTarget model=Gemini-2.5-pro2025.06 | 79.3 | |
| Decomposition Attack (New)Target model=o3-mini2025.06 | 81.2 | |
| Decomposition AttackTarget model=o3-mini2025.06 | 82 | |
| Decomposition Attack (New)Target model=Gemini-2.5-pro2025.06 | 82 | |
| Adversarial ReasoningTarget model=o3-mini2025.06 | 84 | |
| AdaptiveTarget model=o3-mini2025.06 | 84.7 | |
| PAIRTarget model=o3-mini2025.06 | 84.7 | |
| CrescendoTarget model=Gemini-2.5-pro2025.06 | 86 | |
| CrescendoTarget model=o3-mini2025.06 | 86.1 | |
| PAIRTarget model=Gemini-2.5-pro2025.06 | 88 | |
| AdaptiveTarget model=Gemini-2.5-pro2025.06 | 88.7 | |
| Adversarial ReasoningTarget model=Gemini-2.5-pro2025.06 | 88.7 |