Jailbreak Evaluation on Harmful Prompts Curated April 13, 2023
0Bad Bot RateAIM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AIMTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| few_shot_jsonTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| dev_mode_v2Target Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| wikipedia_with_titleTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| wikipediaTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| prefix_injectionTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| distractors_negatedTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| base64_input_onlyTarget Model=Claude v1.32023.07 | 0 | 97 | 3 | |
| leetspeakTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| prefix_injection_helloTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| noneTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| refusal_suppression_invTarget Model=GPT-42023.07 | 0 | 97 | 3 | |
| refusal_suppression_invTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| evil_confidantTarget Model=GPT-42023.07 | 0 | 100 | 0 | |
| evil_confidantTarget Model=Claude v1.32023.07 | 0 | 100 | 0 | |
| rot13Target Model=Claude v1.32023.07 | 3 | 6 | 91 | |
| base64_rawTarget Model=Claude v1.32023.07 | 3 | 94 | 3 | |
| base64_output_onlyTarget Model=Claude v1.32023.07 | 3 | 94 | 3 | |
| noneTarget Model=GPT-42023.07 | 3 | 94 | 3 | |
| disemvowelTarget Model=Claude v1.32023.07 | 6 | 91 | 3 | |
| base64_output_onlyTarget Model=GPT-42023.07 | 6 | 94 | 0 | |
| prefix_injection_helloTarget Model=GPT-42023.07 | 6 | 91 | 3 | |
| dev_mode_with_rantTarget Model=Claude v1.32023.07 | 9 | 91 | 0 | |
| style_injection_jsonTarget Model=Claude v1.32023.07 | 9 | 91 | 0 | |
| base64_input_onlyTarget Model=GPT-42023.07 | 9 | 88 | 3 | |
| leetspeakTarget Model=GPT-42023.07 | 9 | 84 | 6 | |
| auto_obfuscationTarget Model=Claude v1.32023.07 | 12 | 78 | 9 | |
| poemsTarget Model=GPT-42023.07 | 12 | 88 | 0 | |
| poemsTarget Model=Claude v1.32023.07 | 12 | 88 | 0 | |
| refusal_suppressionTarget Model=Claude v1.32023.07 | 16 | 84 | 0 | |
| disemvowelTarget Model=GPT-42023.07 | 16 | 81 | 3 | |
| rot13Target Model=GPT-42023.07 | 16 | 22 | 62 | |
| base64_rawTarget Model=GPT-42023.07 | 16 | 81 | 3 | |
| distractors_negatedTarget Model=GPT-42023.07 | 19 | 81 | 0 | |
| style_injection_shortTarget Model=GPT-42023.07 | 22 | 78 | 0 | |
| auto_obfuscationTarget Model=GPT-42023.07 | 22 | 69 | 9 | |
| prefix_injectionTarget Model=GPT-42023.07 | 22 | 78 | 0 | |
| style_injection_jsonTarget Model=GPT-42023.07 | 24 | 69 | 6 | |
| style_injection_shortTarget Model=Claude v1.32023.07 | 25 | 75 | 0 | |
| refusal_suppressionTarget Model=GPT-42023.07 | 25 | 72 | 3 | |
| auto_payload_splittingTarget Model=GPT-42023.07 | 34 | 38 | 28 | |
| base64Target Model=GPT-42023.07 | 34 | 66 | 0 | |
| base64Target Model=Claude v1.32023.07 | 38 | 56 | 6 | |
| wikipediaTarget Model=GPT-42023.07 | 38 | 47 | 16 | |
| distractorsTarget Model=GPT-42023.07 | 44 | 50 | 6 | |
| distractorsTarget Model=Claude v1.32023.07 | 47 | 53 | 0 | |
| dev_mode_with_rantTarget Model=GPT-42023.07 | 50 | 47 | 3 | |
| wikipedia_with_titleTarget Model=GPT-42023.07 | 50 | 31 | 19 | |
| evil_system_promptTarget Model=GPT-42023.07 | 53 | 47 | 0 | |
| few_shot_jsonTarget Model=GPT-42023.07 | 53 | 41 | 6 | |
| dev_mode_v2Target Model=GPT-42023.07 | 53 | 44 | 3 | |
| combination_1Target Model=GPT-42023.07 | 56 | 34 | 9 | |
| auto_payload_splittingTarget Model=Claude v1.32023.07 | 59 | 25 | 16 | |
| combination_1Target Model=Claude v1.32023.07 | 66 | 19 | 16 | |
| combination_2Target Model=GPT-42023.07 | 69 | 12 | 19 | |
| AIMTarget Model=GPT-42023.07 | 75 | 19 | 6 | |
| combination_3Target Model=Claude v1.32023.07 | 81 | 6 | 12 | |
| combination_2Target Model=Claude v1.32023.07 | 84 | 0 | 16 | |
| combination_3Target Model=GPT-42023.07 | 94 | 3 | 3 | |
| Adaptive attackTarget Model=GPT-42023.07 | 100 | 0 | — | |
| Adaptive attackTarget Model=Claude v1.32023.07 | 100 | 0 | — |