Attacker Effectiveness in Fraudulent Dialogues on English Scam Dialogues
14.5Success Rateclaude-3.5-sonnet-thinking
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| claude-3.5-sonnet-thinkingTOTAL=12802026.01 | 14.5 | 51.1 | 29.9 | 4.5 | |
| deepseek-r1TOTAL=11192026.01 | 12.2 | 25.6 | 24.6 | 37.6 | |
| grok-1TOTAL=12802026.01 | 8.5 | 46 | 33.7 | 11.8 | |
| SparkDesk-v3.5TOTAL=12082026.01 | 2.5 | 21.7 | 59.3 | 16.6 | |
| glm-4-thinkingTOTAL=12762026.01 | 2.3 | 51.1 | 42.6 | 4 | |
| llama-3.1-405b-instructTOTAL=12802026.01 | 1.4 | 8.5 | 75 | 15.1 | |
| ERNIE-4.0-8KTOTAL=11202026.01 | 1.1 | 27.3 | 31.5 | 40.1 | |
| qwen-max-latestTOTAL=11202026.01 | 0.4 | 27.2 | 29.9 | 42.4 |