Financial Safety Evaluation on FinRED R1: Cyber Threats
0ASRGPTFuzzer (Black-box)
Evaluation results
| Method | Links | |
|---|---|---|
| GPTFuzzer (Black-box)Target Model=GPT-5, Model Category=API-Based LLM2026.06 | 0 | |
| GPTFuzzer (Black-box)Target Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 0 | |
| AutoDAN-TurboTarget Model=GPT-5, Model Category=API-Based LLM2026.06 | 0.37 | |
| Direct RequestTarget Model=GPT-5, Model Category=API-Based LLM2026.06 | 0.63 | |
| TAP (Black-box)Target Model=GPT-5, Model Category=API-Based LLM2026.06 | 0.63 | |
| TAP (Black-box)Target Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 0.63 | |
| AutoDAN-TurboTarget Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 0.89 | |
| Direct RequestTarget Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 1.27 | |
| TAP (Black-box)Target Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 1.27 | |
| TAP (Black-box)Target Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 1.9 | |
| GPTFuzzer (Black-box)Target Model=Claude-4, Model Category=API-Based LLM2026.06 | 4.43 | |
| TAP (Black-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 6.96 | |
| AutoDAN-TurboTarget Model=Claude-4, Model Category=API-Based LLM2026.06 | 7.14 | |
| TAP (Black-box)Target Model=Claude-4, Model Category=API-Based LLM2026.06 | 8.86 | |
| AutoDAN-TurboTarget Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 12.66 | |
| GCG (White-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 19.62 | |
| Direct RequestTarget Model=Claude-4, Model Category=API-Based LLM2026.06 | 22.78 | |
| AutoDAN-TurboTarget Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 23.42 | |
| Direct RequestTarget Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 24.05 | |
| GPTFuzzer (Black-box)Target Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 27.85 | |
| AutoDAN-TurboTarget Model=gemma3, Model Category=General-Purpose sLM2026.06 | 33.75 | |
| AutoDAN-TurboTarget Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 35.97 | |
| AutoDAN (White-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 37.34 | |
| AutoDAN (White-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 42.41 | |
| TAP (Black-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 46.2 | |
| GCG (White-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 49.37 | |
| Direct RequestTarget Model=gemma3, Model Category=General-Purpose sLM2026.06 | 53.8 | |
| Direct RequestTarget Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 53.8 | |
| TAP (Black-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 55.06 | |
| GPTFuzzer (Black-box)Target Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 58.38 | |
| GCG (White-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 59.49 | |
| Direct RequestTarget Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 63.29 | |
| GPTFuzzer (Black-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 63.29 | |
| GPTFuzzer (Black-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 65.18 | |
| TAP (Black-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 65.19 | |
| TAP (Black-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 68.35 | |
| AutoDAN-TurboTarget Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 68.99 | |
| AutoDAN (White-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 73.42 | |
| Direct RequestTarget Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 75.32 | |
| TAP (Black-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 75.32 | |
| GPTFuzzer (Black-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 75.95 | |
| GPTFuzzer (Black-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 78.15 | |
| AutoDAN-TurboTarget Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 78.48 | |
| GPTFuzzer (Black-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 79.11 | |
| Direct RequestTarget Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 82.41 | |
| Direct RequestTarget Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 83.04 | |
| GCG (White-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 85.25 | |
| AutoDAN-TurboTarget Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 85.44 | |
| Direct RequestTarget Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 87.34 | |
| GCG (White-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 87.97 | |
| TAP (Black-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 87.97 | |
| GPTFuzzer (Black-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 88.24 | |
| GCG (White-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 88.61 | |
| GPTFuzzer (Black-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 88.73 | |
| AutoDAN-TurboTarget Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 89.51 | |
| AutoDAN (White-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 90.51 | |
| AutoDAN-TurboTarget Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 90.51 | |
| GCG (White-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 91.77 | |
| Direct RequestTarget Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 92.41 | |
| AutoDAN (White-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 98.73 | |
| AutoDAN (White-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 98.73 | |
| AutoDAN (White-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 98.73 |