RAG Poisoning Attack on MS MARCO
82.04E2E Attack Success RateEYES-ON-ME + LLM-Gen
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| EYES-ON-ME + LLM-GenRetriever=Qwen3 Emb 0.6B, Generator=Llama3.2 1B2025.10 | 82.04 | — | — | — | — | — | — | |
| EYES-ON-ME + LLM-GenRetriever=BCE, Generator=Qwen2.5 0.5B2025.10 | 76.92 | — | — | — | — | — | — | |
| EYES-ON-ME + LLM-GenRetriever=Qwen3 Emb 0.6B, Generator=Qwen2.5 0.5B2025.10 | 64.9 | — | — | — | — | — | — | |
| EYES-ON-ME + LLM-GenRetriever=Qwen3 Emb 0.6B, Generator=Gemma 2B2025.10 | 54.81 | — | — | — | — | — | — | |
| EYES-ON-ME + LLM-GenRetriever=BCE, Generator=Llama3.2 1B2025.10 | 53.39 | — | — | — | — | — | — | |
| EYES-ON-ME + LLM-GenRetriever=BCE, Generator=Gemma 2B2025.10 | 42.72 | — | — | — | — | — | — | |
| EYES-ON-MERetriever=BCE, Generator=Llama3.2 1B2025.10 | 36.69 | — | — | — | — | — | — | |
| EYES-ON-MERetriever=BCE, Generator=Qwen2.5 0.5B2025.10 | 36.66 | — | — | — | — | — | — | |
| LLM-GenRetriever=Qwen3 Emb 0.6B, Generator=Qwen2.5 0.5B2025.10 | 36.01 | — | — | — | — | — | — | |
| LLM-GenRetriever=BCE, Generator=Qwen2.5 0.5B2025.10 | 36.01 | — | — | — | — | — | — | |
| EYES-ON-MERetriever=BCE, Generator=Gemma 2B2025.10 | 35.32 | — | — | — | — | — | — | |
| EYES-ON-MERetriever=Qwen3 Emb 0.6B, Generator=Gemma 2B2025.10 | 35.19 | — | — | — | — | — | — | |
| Phantom (MCG)Retriever=BCE, Generator=Qwen2.5 0.5B2025.10 | 34.95 | — | — | — | — | — | — | |
| EYES-ON-MERetriever=Qwen3 Emb 0.6B, Generator=Llama3.2 1B2025.10 | 32.96 | — | — | — | — | — | — | |
| Phantom (MCG)Retriever=BCE, Generator=Gemma 2B2025.10 | 27.66 | — | — | — | — | — | — | |
| LLM-GenRetriever=Qwen3 Emb 0.6B, Generator=Gemma 2B2025.10 | 26.06 | — | — | — | — | — | — | |
| LLM-GenRetriever=BCE, Generator=Gemma 2B2025.10 | 26.06 | — | — | — | — | — | — | |
| EYES-ON-MERetriever=Qwen3 Emb 0.6B, Generator=Qwen2.5 0.5B2025.10 | 25.9 | — | — | — | — | — | — | |
| AutoDANRetriever=Qwen3 Emb 0.6B, Generator=Llama3.2 1B2025.10 | 21.12 | — | — | — | — | — | — | |
| Phantom (MCG)Retriever=Qwen3 Emb 0.6B, Generator=Gemma 2B2025.10 | 18.27 | — | — | — | — | — | — | |
| AutoDANRetriever=BCE, Generator=Llama3.2 1B2025.10 | 17.98 | — | — | — | — | — | — | |
| Phantom (MCG)Retriever=Qwen3 Emb 0.6B, Generator=Qwen2.5 0.5B2025.10 | 17.65 | — | — | — | — | — | — | |
| LLM-GenRetriever=Qwen3 Emb 0.6B, Generator=Llama3.2 1B2025.10 | 17.12 | — | — | — | — | — | — | |
| LLM-GenRetriever=BCE, Generator=Llama3.2 1B2025.10 | 17.12 | — | — | — | — | — | — | |
| GCGRetriever=Qwen3 Emb 0.6B, Generator=Gemma 2B2025.10 | 16.54 | — | — | — | — | — | — | |
| AutoDANRetriever=BCE, Generator=Qwen2.5 0.5B2025.10 | 16.13 | — | — | — | — | — | — | |
| Phantom (MCG)Retriever=BCE, Generator=Llama3.2 1B2025.10 | 15.75 | — | — | — | — | — | — | |
| GCGRetriever=Qwen3 Emb 0.6B, Generator=Qwen2.5 0.5B2025.10 | 15.49 | — | — | — | — | — | — | |
| AutoDANRetriever=Qwen3 Emb 0.6B, Generator=Qwen2.5 0.5B2025.10 | 15.45 | — | — | — | — | — | — | |
| Phantom (MCG)Retriever=Qwen3 Emb 0.6B, Generator=Llama3.2 1B2025.10 | 14.98 | — | — | — | — | — | — | |
| GCGRetriever=BCE, Generator=Qwen2.5 0.5B2025.10 | 14.51 | — | — | — | — | — | — | |
| AutoDANRetriever=Qwen3 Emb 0.6B, Generator=Gemma 2B2025.10 | 13.92 | — | — | — | — | — | — | |
| AutoDANRetriever=BCE, Generator=Gemma 2B2025.10 | 12.65 | — | — | — | — | — | — | |
| GCGRetriever=Qwen3 Emb 0.6B, Generator=Llama3.2 1B2025.10 | 12.24 | — | — | — | — | — | — | |
| GCGRetriever=BCE, Generator=Llama3.2 1B2025.10 | 10.19 | — | — | — | — | — | — | |
| GCGRetriever=BCE, Generator=Gemma 2B2025.10 | 9.11 | — | — | — | — | — | — | |
| Eyes-on-MeOptimization Cost (mins.)=5, Generator Model=Llama3.2-1B2025.10 | — | 0.39 | 66.3 | 56 | 89.5 | 0 | 84.5 | |
| Eyes-on-Me + LLM-GenOptimization Cost (mins.)=5+N, Generator Model=Llama3.2-1B2025.10 | — | 0.86 | 72.2 | 63.7 | 85.7 | 0 | 84.6 | |
| GCGOptimization Cost (mins.)=6N, Generator Model=Llama3.2-1B2025.10 | — | 5.2 | 2.7 | 36.4 | 85.8 | 0 | 69.1 | |
| LLM-GenOptimization Cost (mins.)=1N, Generator Model=Llama3.2-1B2025.10 | — | 1.11 | 96.1 | 60.7 | 88.6 | 0 | 72.8 | |
| Phantom (MCG)Optimization Cost (mins.)=5N, Generator Model=Llama3.2-1B2025.10 | — | 3.46 | 3.6 | 34.5 | 80.6 | 0 | 71.4 |