Financial Safety Evaluation on FinRED R2: Financial Crime
87.14ASRGPTFuzzer (Black-box)
Evaluation results
| Method | Links | |
|---|---|---|
| GPTFuzzer (Black-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 87.14 | |
| Direct RequestTarget Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 83.57 | |
| GPTFuzzer (Black-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 82.86 | |
| GPTFuzzer (Black-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 82.14 | |
| GPTFuzzer (Black-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 76.43 | |
| GPTFuzzer (Black-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 75.71 | |
| GPTFuzzer (Black-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 74.95 | |
| GPTFuzzer (Black-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 68.57 | |
| AutoDAN (White-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 65.71 | |
| GCG (White-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 60.71 | |
| GCG (White-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 60.71 | |
| AutoDAN-TurboTarget Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 60 | |
| TAP (Black-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 55 | |
| GPTFuzzer (Black-box)Target Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 52.14 | |
| GCG (White-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 51.43 | |
| AutoDAN (White-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 50.71 | |
| AutoDAN (White-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 43.57 | |
| TAP (Black-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 42.14 | |
| AutoDAN-TurboTarget Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 37.14 | |
| AutoDAN-TurboTarget Model=gemma3, Model Category=General-Purpose sLM2026.06 | 36.83 | |
| GCG (White-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 33.57 | |
| AutoDAN-TurboTarget Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 32.14 | |
| AutoDAN-TurboTarget Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 31.43 | |
| AutoDAN (White-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 27.86 | |
| Direct RequestTarget Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 25.71 | |
| AutoDAN-TurboTarget Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 24.28 | |
| Direct RequestTarget Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 23.57 | |
| GPTFuzzer (Black-box)Target Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 23.57 | |
| Direct RequestTarget Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 21.43 | |
| Direct RequestTarget Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 21.43 | |
| GCG (White-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 20.71 | |
| Direct RequestTarget Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 20 | |
| Direct RequestTarget Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 20 | |
| GCG (White-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 18.57 | |
| AutoDAN (White-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 17.14 | |
| TAP (Black-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 17.14 | |
| AutoDAN (White-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 15.71 | |
| TAP (Black-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 12.86 | |
| TAP (Black-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 11.43 | |
| Direct RequestTarget Model=gemma3, Model Category=General-Purpose sLM2026.06 | 8.57 | |
| TAP (Black-box)Target Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 8.57 | |
| AutoDAN (White-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 7.86 | |
| TAP (Black-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 7.86 | |
| AutoDAN-TurboTarget Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 7.86 | |
| Direct RequestTarget Model=Claude-4, Model Category=API-Based LLM2026.06 | 7.14 | |
| AutoDAN-TurboTarget Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 7.14 | |
| AutoDAN-TurboTarget Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 6.43 | |
| Direct RequestTarget Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 4.29 | |
| TAP (Black-box)Target Model=GPT-5, Model Category=API-Based LLM2026.06 | 4.29 | |
| TAP (Black-box)Target Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 3.57 | |
| AutoDAN-TurboTarget Model=GPT-5, Model Category=API-Based LLM2026.06 | 3.57 | |
| GCG (White-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 2.86 | |
| GPTFuzzer (Black-box)Target Model=Claude-4, Model Category=API-Based LLM2026.06 | 2.14 | |
| AutoDAN-TurboTarget Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 1.78 | |
| TAP (Black-box)Target Model=Claude-4, Model Category=API-Based LLM2026.06 | 1.43 | |
| TAP (Black-box)Target Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 1.43 | |
| TAP (Black-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 0.71 | |
| AutoDAN-TurboTarget Model=Claude-4, Model Category=API-Based LLM2026.06 | 0.35 | |
| Direct RequestTarget Model=GPT-5, Model Category=API-Based LLM2026.06 | 0 | |
| Direct RequestTarget Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 0 | |
| GPTFuzzer (Black-box)Target Model=GPT-5, Model Category=API-Based LLM2026.06 | 0 | |
| GPTFuzzer (Black-box)Target Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 0 |