Email Address Extraction on Synthetic dataset
100Accuracygpt-4
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| gpt-4Defense=No Defense2024.08 | 100 | 1 | |
| flan-ul2Defense=No Defense2024.08 | 100 | 1 | |
| gpt-4Defense=Symbol Replacement (AT)2024.08 | 99 | 0.99 | |
| gpt-4Defense=Symbol Replacement (DOT)2024.08 | 99 | 0.99 | |
| gpt-4Defense=Hyperlink2024.08 | 99 | 0.99 | |
| flan-ul2Defense=Symbol Replacement (AT)2024.08 | 99 | 0.99 | |
| flan-ul2Defense=Symbol Replacement (DOT)2024.08 | 99 | 0.99 | |
| flan-ul2Defense=Symbol Replacement (AT+DOT)2024.08 | 99 | 0.99 | |
| gpt-4Defense=Symbol Replacement (AT+DOT)2024.08 | 98 | 0.98 | |
| gemini-proDefense=No Defense2024.08 | 98 | 0.98 | |
| gemini-proDefense=Symbol Replacement (AT)2024.08 | 98 | 0.98 | |
| gemini-proDefense=Hyperlink2024.08 | 98 | 0.98 | |
| internlm-chat-7bDefense=Symbol Replacement (AT)2024.08 | 98 | 0.98 | |
| flan-ul2Defense=Hyperlink2024.08 | 98 | 0.98 | |
| flan-ul2Defense=Keyword Replacement2024.08 | 98 | 0.98 | |
| gpt-3.5-turboDefense=Hyperlink2024.08 | 97 | 0.97 | |
| gemini-proDefense=Symbol Replacement (DOT)2024.08 | 97 | 0.97 | |
| gpt-3.5-turboDefense=No Defense2024.08 | 96 | 0.96 | |
| gpt-3.5-turboDefense=Symbol Replacement (AT)2024.08 | 96 | 0.96 | |
| gemini-proDefense=Symbol Replacement (AT+DOT)2024.08 | 96 | 0.96 | |
| gpt-3.5-turboDefense=Symbol Replacement (DOT)2024.08 | 95 | 0.95 | |
| internlm-chat-7bDefense=No Defense2024.08 | 95 | 0.95 | |
| gpt-3.5-turboDefense=Symbol Replacement (AT+DOT)2024.08 | 94 | 0.94 | |
| text-bison-001Defense=Symbol Replacement (AT)2024.08 | 94 | 0.94 | |
| text-bison-001Defense=No Defense2024.08 | 93 | 0.93 | |
| text-bison-001Defense=Symbol Replacement (DOT)2024.08 | 93 | 0.93 | |
| text-bison-001Defense=Hyperlink2024.08 | 93 | 0.93 | |
| text-bison-001Defense=Symbol Replacement (AT+DOT)2024.08 | 92 | 0.92 | |
| chat-bison-001Defense=Hyperlink2024.08 | 92 | 0.92 | |
| chat-bison-001Defense=Symbol Replacement (AT)2024.08 | 91 | 0.91 | |
| chat-bison-001Defense=No Defense2024.08 | 90 | 0.9 | |
| chat-bison-001Defense=Symbol Replacement (DOT)2024.08 | 89 | 0.89 | |
| chat-bison-001Defense=Symbol Replacement (AT+DOT)2024.08 | 88 | 0.88 | |
| internlm-chat-7bDefense=Symbol Replacement (AT+DOT)2024.08 | 87 | 0.87 | |
| gemini-proDefense=Keyword Replacement2024.08 | 85 | 0.85 | |
| vicuna-7b-v1.3Defense=Symbol Replacement (AT)2024.08 | 84 | 0.84 | |
| internlm-chat-7bDefense=Hyperlink2024.08 | 82 | 82 | |
| gpt-4Defense=Keyword Replacement2024.08 | 81 | 0.81 | |
| vicuna-7b-v1.3Defense=Symbol Replacement (AT+DOT)2024.08 | 79 | 0.79 | |
| internlm-chat-7bDefense=Symbol Replacement (DOT)2024.08 | 78 | 0.78 | |
| text-bison-001Defense=Keyword Replacement2024.08 | 76 | 0.76 | |
| vicuna-7b-v1.3Defense=Symbol Replacement (DOT)2024.08 | 69 | 0.69 | |
| gpt-3.5-turboDefense=Keyword Replacement2024.08 | 67 | 0.67 | |
| chat-bison-001Defense=Keyword Replacement2024.08 | 65 | 0.65 | |
| vicuna-13b-v1.3Defense=Symbol Replacement (AT)2024.08 | 65 | 0.65 | |
| vicuna-7b-v1.3Defense=No Defense2024.08 | 65 | 0.65 | |
| vicuna-13b-v1.3Defense=No Defense2024.08 | 59 | 0.59 | |
| vicuna-7b-v1.3Defense=Hyperlink2024.08 | 57 | 57 | |
| flan-ul2Defense=Prompt Injection2024.08 | 54 | 0.54 | |
| vicuna-13b-v1.3Defense=Symbol Replacement (AT+DOT)2024.08 | 44 | 0.44 | |
| vicuna-13b-v1.3Defense=Symbol Replacement (DOT)2024.08 | 43 | 0.43 | |
| llama-2-7b-chat-hfDefense=Symbol Replacement (AT)2024.08 | 39 | 0.39 | |
| llama-2-7b-chat-hfDefense=Hyperlink2024.08 | 38 | 38 | |
| vicuna-13b-v1.3Defense=Hyperlink2024.08 | 36 | 0.36 | |
| llama-2-7b-chat-hfDefense=Symbol Replacement (AT+DOT)2024.08 | 32 | 0.32 | |
| llama-2-7b-chat-hfDefense=Symbol Replacement (DOT)2024.08 | 27 | 0.27 | |
| llama-2-7b-chat-hfDefense=No Defense2024.08 | 26 | 0.26 | |
| internlm-chat-7bDefense=Keyword Replacement2024.08 | 20 | 0.2 | |
| vicuna-7b-v1.3Defense=Keyword Replacement2024.08 | 17 | 0.17 | |
| vicuna-13b-v1.3Defense=Keyword Replacement2024.08 | 8 | 0.08 | |
| gpt-3.5-turboDefense=Prompt Injection2024.08 | 5 | 0.05 | |
| internlm-chat-7bDefense=Prompt Injection2024.08 | 5 | 0.05 | |
| vicuna-7b-v1.3Defense=Prompt Injection2024.08 | 3 | 0.01 | |
| llama-2-7b-chat-hfDefense=Keyword Replacement2024.08 | 3 | 0.03 | |
| gemini-proDefense=Prompt Injection2024.08 | 1 | 0.01 | |
| llama-2-7b-chat-hfDefense=Prompt Injection2024.08 | 1 | 0.01 | |
| gpt-4Defense=Prompt Injection2024.08 | 0 | 0 | |
| text-bison-001Defense=Prompt Injection2024.08 | 0 | 0 | |
| chat-bison-001Defense=Prompt Injection2024.08 | 0 | 0 | |
| vicuna-13b-v1.3Defense=Prompt Injection2024.08 | 0 | 0 |