RAG Attack on HotpotQA
96.1Attack Success Rate (ASR)TabooRAG
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TabooRAGAttack Type=Blocking, Target LLM=GPT-5.22026.03 | 96.1 | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=DeepSeek-V3.22026.03 | 87.1 | — | — | — | |
| KEPo-MultiSearch Mode=Local, Framework=LightRAG2026.03 | 73.2 | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=Llama-3-8B2026.03 | 73.1 | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=GPT-5-mini2026.03 | 72.3 | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=Llama-3-8B2026.03 | 71.3 | — | — | — | |
| KEPo-SingleSearch Mode=Local, Framework=LightRAG2026.03 | 70.5 | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=Qwen-3-32B2026.03 | 68.4 | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=Gemma-3-12B2026.03 | 67.5 | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=Qwen-3-32B2026.03 | 66.7 | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=Qwen-3-32B2026.03 | 66 | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=DeepSeek-V3.22026.03 | 64.3 | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=Ministral-3-8B2026.03 | 63.6 | — | — | — | |
| KEPo-MultiSearch Mode=Global, Framework=LightRAG2026.03 | 62.9 | — | — | — | |
| KEPo-SingleSearch Mode=Global, Framework=LightRAG2026.03 | 61.4 | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=Gemma-3-12B2026.03 | 60 | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=Llama-3-8B2026.03 | 59.5 | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=Ministral-3-8B2026.03 | 58 | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=Gemma-3-12B2026.03 | 55.3 | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=DeepSeek-V3.22026.03 | 55.3 | — | — | — | |
| PoisonedRAGSearch Mode=Local, Framework=LightRAG2026.03 | 55.3 | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=GPT-5-mini2026.03 | 53.3 | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=Llama-3-8B2026.03 | 50.7 | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=GPT-5.22026.03 | 50.7 | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=GPT-5-mini2026.03 | 48.7 | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=Ministral-3-8B2026.03 | 48 | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=Ministral-3-8B2026.03 | 47.3 | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=GPT-5.22026.03 | 47.3 | — | — | — | |
| PoisonedRAGSearch Mode=Global, Framework=LightRAG2026.03 | 45.6 | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=Llama-3-8B2026.03 | 44.2 | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=DeepSeek-V3.22026.03 | 40.9 | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=GPT-5-mini2026.03 | 34.6 | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=Qwen-3-32B2026.03 | 32 | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=GPT-5.22026.03 | 24.6 | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=Gemma-3-12B2026.03 | 23.1 | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=Gemma-3-12B2026.03 | 7.9 | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=Qwen-3-32B2026.03 | 7.3 | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=DeepSeek-V3.22026.03 | 7.1 | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=GPT-5.22026.03 | 5.9 | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=GPT-5-mini2026.03 | 5.8 | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=Ministral-3-8B2026.03 | 4.2 | — | — | — | |
| DEJAGenerator=Llama-2-7B, Retriever=GTR-base2026.04 | — | 79.91 | 13.67 | 0.52 | |
| DEJAGenerator=Llama-2-13B, Retriever=GTR-base2026.04 | — | 82.13 | 14.49 | 0.56 | |
| DEJAGenerator=Mistral-7B, Retriever=GTR-base2026.04 | — | 82.09 | 4.48 | 0.49 | |
| JammingGenerator=Llama-2-7B, Retriever=GTR-base2026.04 | — | 26.67 | 38 | 1.66 | |
| JammingGenerator=Llama-2-13B, Retriever=GTR-base2026.04 | — | 27 | 51 | 1.76 | |
| JammingGenerator=Mistral-7B, Retriever=GTR-base2026.04 | — | 26.33 | 42 | 1.81 | |
| PoisonedRAGGenerator=Llama-2-7B, Retriever=GTR-base2026.04 | — | 29.06 | 38.89 | 1.42 | |
| PoisonedRAGGenerator=Llama-2-13B, Retriever=GTR-base2026.04 | — | 26.57 | 50.72 | 1.46 | |
| PoisonedRAGGenerator=Mistral-7B, Retriever=GTR-base2026.04 | — | 34.83 | 14.43 | 1.32 | |
| Prompt InjectionGenerator=Llama-2-7B, Retriever=GTR-base2026.04 | — | 16.24 | 78.63 | 1.44 | |
| Prompt InjectionGenerator=Llama-2-13B, Retriever=GTR-base2026.04 | — | 26.09 | 71.01 | 1.33 | |
| Prompt InjectionGenerator=Mistral-7B, Retriever=GTR-base2026.04 | — | 72.63 | 25.87 | 0.9 |