Retrieval Attack Defense on Natural Questions (NQ)
0ASRRAGPart
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RAGPartRetriever=Contriever, Attack Type=HotFlip, N=5, k=32025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=ANCE, Attack Type=HotFlip, N=5, k=12025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=ANCE, Attack Type=HotFlip (spread out), N=5, k=32025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=ANCE, Attack Type=Query as poison, N=5, k=12025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=ANCE, Attack Type=Query as poison, N=5, k=32025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=Multilingual E5, Attack Type=HotFlip, N=5, k=12025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=Multilingual E5, Attack Type=HotFlip, N=5, k=32025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=Multilingual E5, Attack Type=HotFlip (spread out), N=5, k=12025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=Multilingual E5, Attack Type=Query as poison, N=5, k=32025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=GTE Large, Attack Type=HotFlip, N=5, k=12025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=GTE Large, Attack Type=HotFlip, N=5, k=32025.12 | 0 | — | — | — | — | — | |
| RAGPartRetriever=ANCE, Attack Type=HotFlip, N=5, k=32025.12 | 1 | — | — | — | — | — | |
| RAGPartRetriever=ANCE, Attack Type=HotFlip (spread out), N=5, k=12025.12 | 1 | — | — | — | — | — | |
| RAGMaskRetriever=Multilingual E5, Attack Type=HotFlip, m=102025.12 | 1 | — | — | — | — | — | |
| RAGMaskRetriever=GTE Large, Attack Type=HotFlip, m=102025.12 | 1 | — | — | — | — | — | |
| RAGPartRetriever=GTE Large, Attack Type=HotFlip (spread out), N=5, k=12025.12 | 1 | — | — | — | — | — | |
| RAGPartRetriever=GTE Large, Attack Type=HotFlip (spread out), N=5, k=32025.12 | 1 | — | — | — | — | — | |
| RAGPartRetriever=Contriever, Attack Type=HotFlip, N=5, k=12025.12 | 2 | — | — | — | — | — | |
| RAGPartRetriever=Contriever, Attack Type=Query as poison, N=5, k=12025.12 | 2 | — | — | — | — | — | |
| RAGPartRetriever=GTE Large, Attack Type=Query as poison, N=5, k=12025.12 | 2 | — | — | — | — | — | |
| RAGPartRetriever=GTE Large, Attack Type=Query as poison, N=5, k=32025.12 | 2 | — | — | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=DeepSeek-V3.22026.03 | 2.7 | — | — | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=Ministral-3-8B2026.03 | 2.8 | — | — | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=Qwen-3-32B2026.03 | 2.8 | — | — | — | — | — | |
| RAGPartRetriever=Contriever, Attack Type=Query as poison, N=5, k=32025.12 | 3 | — | — | — | — | — | |
| RAGPartRetriever=ANCE, Attack Type=AdvRAGgen, N=5, k=32025.12 | 3 | — | — | — | — | — | |
| RAGMaskRetriever=Multilingual E5, Attack Type=HotFlip (spread out), m=102025.12 | 3 | — | — | — | — | — | |
| RAGPartRetriever=Multilingual E5, Attack Type=Query as poison, N=5, k=12025.12 | 3 | — | — | — | — | — | |
| RAGPartRetriever=Contriever, Attack Type=HotFlip (spread out), N=5, k=32025.12 | 4 | — | — | — | — | — | |
| RAGMaskRetriever=Contriever, Attack Type=Query as poison, m=102025.12 | 4 | — | — | — | — | — | |
| RAGMaskRetriever=ANCE, Attack Type=HotFlip, m=102025.12 | 4 | — | — | — | — | — | |
| RAGMaskRetriever=Multilingual E5, Attack Type=AdvRAGgen, m=102025.12 | 4 | — | — | — | — | — | |
| RAGMaskRetriever=GTE Large, Attack Type=HotFlip (spread out), m=102025.12 | 4 | — | — | — | — | — | |
| RAGMaskRetriever=GTE Large, Attack Type=Query as poison, m=102025.12 | 4 | — | — | — | — | — | |
| RAGPartRetriever=Contriever, Attack Type=HotFlip (spread out), N=5, k=12025.12 | 5 | — | — | — | — | — | |
| RAGPartRetriever=GTE Large, Attack Type=AdvRAGgen, N=5, k=32025.12 | 5 | — | — | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=GPT-5-mini2026.03 | 5.3 | — | — | — | — | — | |
| RAGPartRetriever=Contriever, Attack Type=AdvRAGgen, N=5, k=32025.12 | 6 | — | — | — | — | — | |
| RAGMaskRetriever=ANCE, Attack Type=HotFlip (spread out), m=102025.12 | 6 | — | — | — | — | — | |
| RAGPartRetriever=ANCE, Attack Type=AdvRAGgen, N=5, k=12025.12 | 6 | — | — | — | — | — | |
| RAGPartRetriever=GTE Large, Attack Type=AdvRAGgen, N=5, k=12025.12 | 6 | — | — | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=GPT-5.22026.03 | 6 | — | — | — | — | — | |
| RAGMaskRetriever=Contriever, Attack Type=HotFlip, m=102025.12 | 7 | — | — | — | — | — | |
| RAGPartRetriever=Multilingual E5, Attack Type=AdvRAGgen, N=5, k=32025.12 | 7 | — | — | — | — | — | |
| RAGMaskRetriever=GTE Large, Attack Type=AdvRAGgen, m=102025.12 | 7 | — | — | — | — | — | |
| RAGMaskRetriever=Contriever, Attack Type=AdvRAGgen, m=102025.12 | 8 | — | — | — | — | — | |
| RAGMaskRetriever=ANCE, Attack Type=Query as poison, m=102025.12 | 8 | — | — | — | — | — | |
| RAGMaskRetriever=ANCE, Attack Type=AdvRAGgen, m=102025.12 | 8 | — | — | — | — | — | |
| RAGPartRetriever=Multilingual E5, Attack Type=HotFlip (spread out), N=5, k=32025.12 | 8 | — | — | — | — | — | |
| RAGPartRetriever=Multilingual E5, Attack Type=AdvRAGgen, N=5, k=12025.12 | 8 | — | — | — | — | — | |
| RAGMaskRetriever=Contriever, Attack Type=HotFlip (spread out), m=102025.12 | 9 | — | — | — | — | — | |
| RAGMaskRetriever=Multilingual E5, Attack Type=Query as poison, m=102025.12 | 9 | — | — | — | — | — | |
| RAGPartRetriever=Contriever, Attack Type=AdvRAGgen, N=5, k=12025.12 | 10 | — | — | — | — | — | |
| No DefenseRetriever=GTE Large, Attack Type=HotFlip2025.12 | 10 | — | — | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=Gemma-3-12B2026.03 | 13.3 | — | — | — | — | — | |
| No DefenseRetriever=Multilingual E5, Attack Type=HotFlip (spread out)2025.12 | 15 | — | — | — | — | — | |
| No DefenseRetriever=GTE Large, Attack Type=HotFlip (spread out)2025.12 | 16 | — | — | — | — | — | |
| No DefenseRetriever=Multilingual E5, Attack Type=HotFlip2025.12 | 18 | — | — | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=Ministral-3-8B2026.03 | 18.7 | — | — | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=GPT-5.22026.03 | 21.2 | — | — | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=GPT-5.22026.03 | 22 | — | — | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=Gemma-3-12B2026.03 | 22 | — | — | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=GPT-5-mini2026.03 | 24 | — | — | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=Gemma-3-12B2026.03 | 26 | — | — | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=DeepSeek-V3.22026.03 | 26.7 | — | — | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=GPT-5-mini2026.03 | 26.8 | — | — | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=Qwen-3-32B2026.03 | 27 | — | — | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=Qwen-3-32B2026.03 | 29.3 | — | — | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=DeepSeek-V3.22026.03 | 29.7 | — | — | — | — | — | |
| AuthChainAttack Type=Misleading, Target LLM=Llama-3-8B2026.03 | 32 | — | — | — | — | — | |
| No DefenseRetriever=ANCE, Attack Type=HotFlip (spread out)2025.12 | 45 | — | — | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=Ministral-3-8B2026.03 | 49.5 | — | — | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=Gemma-3-12B2026.03 | 51.7 | — | — | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=Ministral-3-8B2026.03 | 54 | — | — | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=GPT-5.22026.03 | 55.3 | — | — | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=Llama-3-8B2026.03 | 56.7 | — | — | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=GPT-5-mini2026.03 | 56.7 | — | — | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=DeepSeek-V3.22026.03 | 58.7 | — | — | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=Llama-3-8B2026.03 | 60.3 | — | — | — | — | — | |
| No DefenseRetriever=ANCE, Attack Type=HotFlip2025.12 | 61 | — | — | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=Qwen-3-32B2026.03 | 61.6 | — | — | — | — | — | |
| No DefenseRetriever=GTE Large, Attack Type=Query as poison2025.12 | 63 | — | — | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=Gemma-3-12B2026.03 | 63.2 | — | — | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=GPT-5-mini2026.03 | 63.4 | — | — | — | — | — | |
| JammingAttack Type=Blocking, Target LLM=Llama-3-8B2026.03 | 63.9 | — | — | — | — | — | |
| No DefenseRetriever=ANCE, Attack Type=Query as poison2025.12 | 68 | — | — | — | — | — | |
| PoisonedAttack Type=Misleading, Target LLM=Qwen-3-32B2026.03 | 68 | — | — | — | — | — | |
| No DefenseRetriever=GTE Large, Attack Type=AdvRAGgen2025.12 | 70 | — | — | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=Ministral-3-8B2026.03 | 71 | — | — | — | — | — | |
| No DefenseRetriever=Multilingual E5, Attack Type=Query as poison2025.12 | 73 | — | — | — | — | — | |
| MuteRAGAttack Type=Blocking, Target LLM=Llama-3-8B2026.03 | 75.9 | — | — | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=GPT-5.22026.03 | 77.1 | — | — | — | — | — | |
| No DefenseRetriever=Contriever, Attack Type=Query as poison2025.12 | 78 | — | — | — | — | — | |
| No DefenseRetriever=ANCE, Attack Type=AdvRAGgen2025.12 | 79 | — | — | — | — | — | |
| No DefenseRetriever=Multilingual E5, Attack Type=AdvRAGgen2025.12 | 81 | — | — | — | — | — | |
| TabooRAGAttack Type=Blocking, Target LLM=DeepSeek-V3.22026.03 | 82.8 | — | — | — | — | — | |
| No DefenseRetriever=Contriever, Attack Type=HotFlip (spread out)2025.12 | 85 | — | — | — | — | — | |
| No DefenseRetriever=Contriever, Attack Type=HotFlip2025.12 | 87 | — | — | — | — | — | |
| No DefenseRetriever=Contriever, Attack Type=AdvRAGgen2025.12 | 91 | — | — | — | — | — | |
| Corpusn (poison budget)=5, k (context budget)=52026.03 | — | — | — | — | — | 2.4 |