Document Summarization on GovReport (Utility and ASR Metrics)
89ASR ScoreNo Defense
Evaluation results
| Method | Links | ||
|---|---|---|---|
| No DefenseAttack Type=Combined, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 89 | — | |
| PromptArmorAttack Type=Combined, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 89 | — | |
| No DefenseAttack Type=Direct, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 86 | — | |
| PromptGuardAttack Type=Direct, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 86 | — | |
| PromptArmorAttack Type=Direct, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 85 | — | |
| PromptGuardAttack Type=Combined, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 83 | — | |
| PIGuardAttack Type=Direct, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 80 | — | |
| PIGuardAttack Type=Combined, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 75 | — | |
| GPT-OSS-SafeguardAttack Type=Direct, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 23 | — | |
| DataSentinelAttack Type=Direct, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 12 | — | |
| AgentWatcherAttack Type=Direct, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 6 | — | |
| DataSentinelAttack Type=Combined, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 4 | — | |
| GPT-OSS-SafeguardAttack Type=Combined, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 0 | — | |
| AgentWatcherAttack Type=Combined, Target LLM=Qwen-3-4B-Instruct-25072026.04 | 0 | — | |
| AgentWatcherAttack Type=Clean, Target LLM=Qwen-3-4B-Instruct-25072026.04 | — | 31 | |
| DataSentinelAttack Type=Clean, Target LLM=Qwen-3-4B-Instruct-25072026.04 | — | 7 | |
| GPT-OSS-SafeguardAttack Type=Clean, Target LLM=Qwen-3-4B-Instruct-25072026.04 | — | 31 | |
| No DefenseAttack Type=Clean, Target LLM=Qwen-3-4B-Instruct-25072026.04 | — | 31 | |
| PIGuardAttack Type=Clean, Target LLM=Qwen-3-4B-Instruct-25072026.04 | — | 31 | |
| PromptArmorAttack Type=Clean, Target LLM=Qwen-3-4B-Instruct-25072026.04 | — | 31 | |
| PromptGuardAttack Type=Clean, Target LLM=Qwen-3-4B-Instruct-25072026.04 | — | 31 |