Financial Safety Evaluation on FinRED R3: Misinformation
97.76ASRGPTFuzzer (Black-box)
Evaluation Results
| Method | Links | |
|---|---|---|
| GPTFuzzer (Black-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 97.76 | |
| GPTFuzzer (Black-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 97.01 | |
| GPTFuzzer (Black-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 87.01 | |
| GPTFuzzer (Black-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 86.57 | |
| GCG (White-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 76.87 | |
| GCG (White-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 73.13 | |
| GPTFuzzer (Black-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 71.04 | |
| Direct RequestTarget Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 70.15 | |
| GCG (White-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 66.42 | |
| GPTFuzzer (Black-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 66.41 | |
| GPTFuzzer (Black-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 64.17 | |
| GCG (White-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 59.7 | |
| GPTFuzzer (Black-box)Target Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 58.81 | |
| AutoDAN-TurboTarget Model=gemma3, Model Category=General-Purpose sLM2026.06 | 54.18 | |
| AutoDAN (White-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 52.99 | |
| GPTFuzzer (Black-box)Target Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 42.39 | |
| TAP (Black-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 41.79 | |
| AutoDAN-TurboTarget Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 39.55 | |
| Direct RequestTarget Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 38.06 | |
| AutoDAN-TurboTarget Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 37.55 | |
| TAP (Black-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 35.07 | |
| AutoDAN (White-box)Target Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 34.33 | |
| AutoDAN-TurboTarget Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 33.58 | |
| Direct RequestTarget Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 32.09 | |
| AutoDAN (White-box)Target Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 32.09 | |
| Direct RequestTarget Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 31.34 | |
| Direct RequestTarget Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 31.34 | |
| GCG (White-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 31.34 | |
| AutoDAN (White-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 31.34 | |
| TAP (Black-box)Target Model=FinMA, Model Category=Finance-Specific sLM2026.06 | 29.1 | |
| Direct RequestTarget Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 27.61 | |
| GCG (White-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 27.61 | |
| AutoDAN-TurboTarget Model=qqWen-7B, Model Category=Finance-Specific sLM2026.06 | 27.61 | |
| Direct RequestTarget Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 26.87 | |
| AutoDAN (White-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 26.87 | |
| AutoDAN-TurboTarget Model=Qwen2.5, Model Category=General-Purpose sLM2026.06 | 23.88 | |
| AutoDAN-TurboTarget Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 23.88 | |
| TAP (Black-box)Target Model=EXAONE, Model Category=General-Purpose sLM2026.06 | 20.15 | |
| TAP (Black-box)Target Model=qqWen-32B, Model Category=Finance-Specific sLM2026.06 | 20.15 | |
| Direct RequestTarget Model=gemma3, Model Category=General-Purpose sLM2026.06 | 19.4 | |
| AutoDAN (White-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 16.42 | |
| Direct RequestTarget Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 15.67 | |
| AutoDAN (White-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 14.93 | |
| AutoDAN-TurboTarget Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 14.18 | |
| AutoDAN-TurboTarget Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 9.2 | |
| GPTFuzzer (Black-box)Target Model=Claude-4, Model Category=API-Based LLM2026.06 | 8.96 | |
| AutoDAN-TurboTarget Model=GPT-5, Model Category=API-Based LLM2026.06 | 8.15 | |
| TAP (Black-box)Target Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 7.46 | |
| AutoDAN-TurboTarget Model=Claude-4, Model Category=API-Based LLM2026.06 | 7.14 | |
| GCG (White-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 6.72 | |
| TAP (Black-box)Target Model=gemma3, Model Category=General-Purpose sLM2026.06 | 6.72 | |
| AutoDAN-TurboTarget Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 6.72 | |
| TAP (Black-box)Target Model=GPT-5, Model Category=API-Based LLM2026.06 | 5.22 | |
| TAP (Black-box)Target Model=Gemini-Pro, Model Category=API-Based LLM2026.06 | 5.22 | |
| Direct RequestTarget Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 3.73 | |
| GPTFuzzer (Black-box)Target Model=GPT-5-mini, Model Category=API-Based LLM2026.06 | 3.73 | |
| TAP (Black-box)Target Model=Gemini-Flash, Model Category=API-Based LLM2026.06 | 2.99 | |
| TAP (Black-box)Target Model=Llama3.1, Model Category=General-Purpose sLM2026.06 | 2.24 | |
| Direct RequestTarget Model=GPT-5, Model Category=API-Based LLM2026.06 | 1.49 | |
| GPTFuzzer (Black-box)Target Model=GPT-5, Model Category=API-Based LLM2026.06 | 0.75 | |
| Direct RequestTarget Model=Claude-4, Model Category=API-Based LLM2026.06 | 0 | |
| Claude-4Target Model=Claude-4, Model Category=API-Based LLM2026.06 | 0 |