Phishing email quality assessment on LLM-generated & APWG eCrimeX spear-phishing emails
99.06PersuasivenessVariation Emails
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Variation EmailsType=Variation Strategy2026.05 | 99.06 | — | 93.18 | 93.84 | 100 | 89.28 | 73.5 | 70.33 | |
| GPT EmailsSource Model=GPT2026.05 | 98.9 | 95.18 | 98.97 | 90.09 | 99.96 | 86.29 | 82.07 | 97.49 | |
| Claude EmailsSource Model=Claude2026.05 | 98.26 | 91.51 | 99.66 | 87.16 | 99.97 | 82.31 | 71.63 | 99.1 | |
| Gemma EmailsSource Model=Gemma2026.05 | 96.31 | 92.41 | 97.82 | 85.15 | 99.98 | 79.16 | 59.44 | 98.71 | |
| Gemini EmailsSource Model=Gemini2026.05 | 93.46 | 94.37 | 92.41 | 86.57 | 99.79 | 69.55 | 68.99 | 96.62 | |
| LLaMA EmailsSource Model=LLaMA2026.05 | 78.49 | 87.06 | 73.77 | 75.27 | 100 | 79.91 | 57.26 | 97.86 | |
| APWG EmailsSource=APWG eCrimeX2026.05 | 32.7 | — | 54.4 | 8.6 | 51 | 64.4 | 15.4 | 78.1 |