Long-form Generation on Bio
81LLM-Judge ScoreVanilla RAG
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Vanilla RAGModel=GPT-4o-mini, Attack Scenario=Benign, k=102025.09 | 81 | — | — | — | |
| MISModel=GPT-4o-mini, Attack Scenario=Benign, k=102025.09 | 80.1 | — | — | — | |
| MISModel=GPT-4o-mini, Attack Scenario=@Pos 10, k=102025.09 | 79 | — | — | — | |
| VanillaLLM=Mistral-I7B, Defense=Vanilla2024.05 | 78.4 | — | — | — | |
| MISModel=GPT-4o-mini, Attack Scenario=@Pos 1, k=102025.09 | 77.9 | — | — | — | |
| MISModel=Mistral-7B, Attack Scenario=Benign, k=102025.09 | 73.5 | — | — | — | |
| MISModel=Llama3.2-3B, Attack Scenario=Benign, k=102025.09 | 73 | — | — | — | |
| Vanilla RAGModel=Mistral-7B, Attack Scenario=Benign, k=102025.09 | 72.9 | — | — | — | |
| Vanilla RAGModel=Llama3.2-3B, Attack Scenario=Benign, k=102025.09 | 72.6 | — | — | — | |
| MISModel=Llama3.2-3B, Attack Scenario=@Pos 10, k=102025.09 | 72.1 | — | — | — | |
| VanillaLLM=Llama2-C7B, Defense=Vanilla2024.05 | 71.8 | — | — | — | |
| MISModel=Mistral-7B, Attack Scenario=@Pos 10, k=102025.09 | 71.5 | — | — | — | |
| InstructRAGModel=Llama3.2-3B, Attack Scenario=Benign, k=102025.09 | 71.3 | — | — | — | |
| Decoding_cLLM=Mistral-I7B, Defense=Decoding_c2024.05 | 71.2 | 45.6 | — | — | |
| MISModel=Llama3.2-3B, Attack Scenario=@Pos 1, k=102025.09 | 71 | — | — | — | |
| Decoding_cLLM=Llama2-C7B, Defense=Decoding_c2024.05 | 70.6 | 38.8 | — | — | |
| MISModel=Mistral-7B, Attack Scenario=@Pos 1, k=102025.09 | 69.7 | — | — | — | |
| InstructRAGModel=Mistral-7B, Attack Scenario=@Pos 1, k=102025.09 | 69.4 | — | — | — | |
| InstructRAGModel=Mistral-7B, Attack Scenario=Benign, k=102025.09 | 68.4 | — | — | — | |
| AstuteRAGModel=Mistral-7B, Attack Scenario=Benign, k=102025.09 | 66 | — | — | — | |
| Vanilla RAGModel=GPT-4o-mini, Attack Scenario=@Pos 1, k=102025.09 | 65.6 | — | — | — | |
| Vanilla RAGModel=Mistral-7B, Attack Scenario=@Pos 1, k=102025.09 | 65.5 | — | — | — | |
| Vanilla RAGModel=Llama3.2-3B, Attack Scenario=@Pos 1, k=102025.09 | 65.1 | — | — | — | |
| KeywordLLM=Mistral-I7B, Defense=Keyword2024.05 | 64.8 | 46.6 | — | — | |
| AstuteRAGModel=GPT-4o-mini, Attack Scenario=@Pos 10, k=102025.09 | 63.9 | — | — | — | |
| Decoding_rLLM=Mistral-I7B, Defense=Decoding_r2024.05 | 63.4 | 51.2 | — | — | |
| InstructRAGModel=GPT-4o-mini, Attack Scenario=@Pos 10, k=102025.09 | 63.1 | — | — | — | |
| AstuteRAGModel=Llama3.2-3B, Attack Scenario=Benign, k=102025.09 | 62.7 | — | — | — | |
| Decoding_rLLM=Llama2-C7B, Defense=Decoding_r2024.05 | 62.4 | 41.6 | — | — | |
| KeywordLLM=Llama2-C7B, Defense=Keyword2024.05 | 62.2 | 46.4 | — | — | |
| InstructRAGModel=GPT-4o-mini, Attack Scenario=Benign, k=102025.09 | 61.9 | — | — | — | |
| RobustRAGModel=GPT-4o-mini, Attack Scenario=@Pos 10, k=102025.09 | 61.4 | — | — | — | |
| RobustRAGModel=GPT-4o-mini, Attack Scenario=Benign, k=102025.09 | 61.2 | — | — | — | |
| RobustRAGModel=GPT-4o-mini, Attack Scenario=@Pos 1, k=102025.09 | 60.4 | — | — | — | |
| InstructRAGModel=Llama3.2-3B, Attack Scenario=@Pos 1, k=102025.09 | 59.9 | — | — | — | |
| No RAGLLM=Mistral-I7B, Defense=No RAG2024.05 | 59.4 | — | — | — | |
| AstuteRAGModel=GPT-4o-mini, Attack Scenario=Benign, k=102025.09 | 59.1 | — | — | — | |
| RobustRAGModel=Mistral-7B, Attack Scenario=Benign, k=102025.09 | 58.6 | — | — | — | |
| RobustRAGModel=Mistral-7B, Attack Scenario=@Pos 10, k=102025.09 | 57.1 | — | — | — | |
| KeywordLLM=GPT-3.5, Attack=PIA, k=10, k'=12024.05 | 56.6 | — | 0 | — | |
| RobustRAGModel=Mistral-7B, Attack Scenario=@Pos 1, k=102025.09 | 56.5 | — | — | — | |
| RobustRAGModel=Llama3.2-3B, Attack Scenario=Benign, k=102025.09 | 56 | — | — | — | |
| AstuteRAGModel=Mistral-7B, Attack Scenario=@Pos 1, k=102025.09 | 54.5 | — | — | — | |
| AstuteRAGModel=GPT-4o-mini, Attack Scenario=@Pos 1, k=102025.09 | 54.2 | — | — | — | |
| RobustRAGModel=Llama3.2-3B, Attack Scenario=@Pos 1, k=102025.09 | 53 | — | — | — | |
| KeywordLLM=GPT-3.5, Attack=Poison, k=10, k'=12024.05 | 52.4 | — | 0 | — | |
| RobustRAGModel=Llama3.2-3B, Attack Scenario=@Pos 10, k=102025.09 | 51.9 | — | — | — | |
| AstuteRAGModel=Llama3.2-3B, Attack Scenario=@Pos 1, k=102025.09 | 46.7 | — | — | — | |
| AstuteRAGModel=Mistral-7B, Attack Scenario=@Pos 10, k=102025.09 | 43.9 | — | — | — | |
| VanillaLLM=GPT-3.5, Attack=Poison, k=10, k'=12024.05 | 43 | — | 56 | — | |
| AstuteRAGModel=Llama3.2-3B, Attack Scenario=@Pos 10, k=102025.09 | 38.6 | — | — | — | |
| InstructRAGModel=GPT-4o-mini, Attack Scenario=@Pos 1, k=102025.09 | 37.9 | — | — | — | |
| InstructRAGModel=Llama3.2-3B, Attack Scenario=@Pos 10, k=102025.09 | 29 | — | — | — | |
| No RAGLLM=Llama2-C7B, Defense=No RAG2024.05 | 19.6 | — | — | — | |
| Vanilla RAGModel=Llama3.2-3B, Attack Scenario=@Pos 10, k=102025.09 | 18.5 | — | — | — | |
| VanillaLLM=GPT-3.5, Attack=PIA, k=10, k'=12024.05 | 17.2 | — | 90 | — | |
| Vanilla RAGModel=Mistral-7B, Attack Scenario=@Pos 10, k=102025.09 | 11.5 | — | — | — | |
| InstructRAGModel=Mistral-7B, Attack Scenario=@Pos 10, k=102025.09 | 9.8 | — | — | — | |
| Vanilla RAGModel=GPT-4o-mini, Attack Scenario=@Pos 10, k=102025.09 | 9.8 | — | — | — | |
| No RAGLLM=GPT-3.5, Retrieved passages (k)=0, Corruption size (k')=12024.05 | — | — | — | 12.6 | |
| RobustRAG (Keyword)LLM=GPT-3.5, Retrieved passages (k)=10, Corruption size (k')=12024.05 | — | 24 | — | 59.4 | |
| VanillaLLM=GPT-3.5, Retrieved passages (k)=10, Corruption size (k')=12024.05 | — | 0 | — | 76.6 |