Question Answering on RQA
88.2ASRVanilla
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| VanillaLLM=Llama2-C2025.06 | 88.2 | — | 5.8 | |
| Vanilla RAGTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 84 | 15 | — | |
| Vanilla RAGTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 79 | 19 | — | |
| Vanilla RAGTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=22026.05 | 74 | 25 | — | |
| InstructRAGTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=32026.05 | 68 | 23 | — | |
| Vanilla RAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=22026.05 | 65 | 35 | — | |
| InstructRAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 62 | 21 | — | |
| Vanilla RAGTop-k=10, Pos=Pos 10, Generator LLM=Grok-4-fast2026.05 | 57 | 37 | — | |
| InstructRAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 55 | 27 | — | |
| VanillaLLM=Deepseek-R12025.06 | 49.6 | — | 24.3 | |
| AstuteRAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 47 | 27 | — | |
| InstructRAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=42026.05 | 46 | 33 | — | |
| Vanilla RAGTop-k=50, Pos=Pos 50, Generator LLM=Grok-4-fast2026.05 | 43 | 42 | — | |
| Vanilla RAGTop-k=50, Pos=Pos 25, Generator LLM=Grok-4-fast2026.05 | 42 | 45 | — | |
| AstuteRAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 41 | 32 | — | |
| Vanilla RAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 39 | 53 | — | |
| VanillaLLM=GPT-4o2025.06 | 33.4 | — | 52.4 | |
| InstructRAGTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 33 | 53 | — | |
| Vanilla RAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 32 | 57 | — | |
| InstructRAGTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 32 | 59 | — | |
| InstructRAGTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 32 | 26 | — | |
| VanillaLLM=Llama-3.12025.06 | 29.4 | — | 21 | |
| Vanilla RAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 28 | 59 | — | |
| ReliabilityRAGTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 28 | 62 | — | |
| InstructRAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 27 | 53 | — | |
| VanillaLLM=Mistral-7B2025.06 | 26.6 | — | 52.2 | |
| VanillaLLM=GPT-4o2025.06 | 26.6 | — | 55.6 | |
| Vanilla RAGTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 26 | 65 | — | |
| AV FilterLLM=Deepseek-R1, alpha=52025.06 | 25.6 | — | 39.3 | |
| InstructRAGTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 25 | 12 | — | |
| ReliabilityRAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 24 | 68 | — | |
| Vanilla RAGTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 24 | 64 | — | |
| VanillaLLM=Mistral-7B2025.06 | 23.4 | — | 50 | |
| ReliabilityRAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 23 | 67 | — | |
| ReliabilityRAGTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 21 | 52 | — | |
| AV FilterLLM=Deepseek-R1, alpha=52025.06 | 20.3 | — | 45.3 | |
| ReliabilityRAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=41, NLI(s)=0.40, Inference(s)=0.1072, Total(s)=432026.05 | 18 | 67 | — | |
| VanillaLLM=Llama2-C2025.06 | 17.2 | — | 57.4 | |
| DecodingLLM=Llama-3.12025.06 | 17 | — | 51 | |
| VanillaLLM=Deepseek-R12025.06 | 17 | — | 46.3 | |
| RobustRAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 16 | 69 | — | |
| RADARTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 16 | 69 | — | |
| ReliabilityRAGTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=20, NLI(s)=0.32, Inference(s)=0.0005, Total(s)=212026.05 | 15 | 69 | — | |
| RobustRAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 15 | 69 | — | |
| AV FilterLLM=Deepseek-R1, alpha=102025.06 | 14.6 | — | 46.6 | |
| VanillaLLM=Llama-3.12025.06 | 14 | — | 48.4 | |
| AV FilterLLM=Deepseek-R1, alpha=102025.06 | 14 | — | 47 | |
| RADARTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 13 | 74 | — | |
| DecodingLLM=Mistral-7B2025.06 | 13 | — | 54 | |
| DecodingLLM=Llama2-C2025.06 | 12 | — | 38 | |
| RADARTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=20, NLI(s)=0.32, Inference(s)=0.0007, Total(s)=212026.05 | 11 | 69 | — | |
| RobustRAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 11 | 74 | — | |
| RADARTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 11 | 75 | — | |
| AV FilterLLM=GPT-4o, alpha=102025.06 | 11 | — | 57.2 | |
| AV FilterLLM=GPT-4o, alpha=52025.06 | 10.6 | — | 57.8 | |
| AV FilterLLM=Deepseek-R1, alpha=∞2025.06 | 10.3 | — | 50.3 | |
| RobustRAGTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 10 | 72 | — | |
| ReliabilityRAGTop-k=10, Pos=Pos 10, Generator LLM=Grok-4-fast2026.05 | 10 | 60 | — | |
| RobustRAGTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 10 | 63 | — | |
| AV FilterLLM=GPT-4o, alpha=∞2025.06 | 9.2 | — | 61.4 | |
| RobustRAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=86, Inference(s)=0.0945, Total(s)=942026.05 | 9 | 69 | — | |
| ReliabilityRAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 9 | 76 | — | |
| RADARTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 9 | 76 | — | |
| RADARTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 9 | 66 | — | |
| AV FilterLLM=Llama2-C, alpha=102025.06 | 8.8 | — | 54.8 | |
| AV FilterLLM=Mistral-7B, alpha=∞2025.06 | 8.6 | — | 56.8 | |
| AV FilterLLM=Deepseek-R1, alpha=∞2025.06 | 8.6 | — | 53 | |
| AV FilterLLM=Mistral-7B, alpha=102025.06 | 8.4 | — | 56.8 | |
| RobustRAGTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 8 | 72 | — | |
| RobustRAGTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 8 | 54 | — | |
| ReliabilityRAGTop-k=50, Pos=Pos 25, Generator LLM=Grok-4-fast2026.05 | 8 | 76 | — | |
| AV FilterLLM=Mistral-7B, alpha=52025.06 | 7.4 | — | 58.2 | |
| AV FilterLLM=Llama-3.1, alpha=52025.06 | 7.2 | — | 53.8 | |
| RobustRAGTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=20, Inference(s)=0.0365, Total(s)=212026.05 | 7 | 64 | — | |
| RADARTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05 | 7 | 68 | — | |
| KeywordLLM=Llama-3.12025.06 | 7 | — | 61 | |
| DecodingLLM=Llama-3.12025.06 | 7 | — | 55 | |
| AV FilterLLM=GPT-4o, alpha=∞2025.06 | 7 | — | 61.2 | |
| AV FilterLLM=Llama2-C, alpha=52025.06 | 7 | — | 55.4 | |
| AV FilterLLM=Llama2-C, alpha=∞2025.06 | 6.6 | — | 56.8 | |
| AV FilterLLM=Llama-3.1, alpha=102025.06 | 6.4 | — | 53.2 | |
| AV FilterLLM=Llama-3.1, alpha=∞2025.06 | 6.2 | — | 57.4 | |
| AstuteRAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 6 | 35 | — | |
| AstuteRAGTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 6 | 28 | — | |
| KeywordLLM=Mistral-7B2025.06 | 6 | — | 54 | |
| KeywordLLM=Llama2-C2025.06 | 6 | — | 53 | |
| KeywordLLM=Mistral-7B2025.06 | 6 | — | 55 | |
| KeywordLLM=Llama-3.12025.06 | 6 | — | 57 | |
| AV FilterLLM=GPT-4o, alpha=52025.06 | 5.2 | — | 63.6 | |
| AstuteRAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=72026.05 | 5 | 21 | — | |
| RADARTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=86, NLI(s)=0.52, Inference(s)=0.0025, Total(s)=872026.05 | 5 | 72 | — | |
| AstuteRAGTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 5 | 43 | — | |
| RADARTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05 | 5 | 75 | — | |
| InstructRAGTop-k=10, Pos=Pos 10, Generator LLM=Grok-4-fast2026.05 | 5 | 47 | — | |
| RADARTop-k=10, Pos=Pos 10, Generator LLM=Grok-4-fast2026.05 | 5 | 69 | — | |
| DecodingLLM=Mistral-7B2025.06 | 5 | — | 55 | |
| AV FilterLLM=Llama2-C, alpha=∞2025.06 | 5 | — | 60.2 | |
| KeywordLLM=Llama2-C2025.06 | 5 | — | 53 | |
| DecodingLLM=Llama2-C2025.06 | 5 | — | 40 | |
| AV FilterLLM=GPT-4o, alpha=102025.06 | 4.8 | — | 63.6 |