RAG Attack Defense on Natural Questions
0ASRTrustRAG
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| TrustRAGAttack Scenario=GL2026.05 | 0 | — | — | — | — | — | — | 0 | |
| CleanBaseAttack Scenario=GL2026.05 | 0 | — | — | — | — | — | — | 0 | |
| CleanBase + MetaSecAlignAttack Scenario=GL2026.05 | 0 | — | — | — | — | — | — | 0 | |
| TrustRAGAttack Scenario=PW2026.05 | 1 | — | — | — | — | — | — | 1.6 | |
| TrustRAGAttack Scenario=PB2026.05 | 1 | — | — | — | — | — | — | 1 | |
| PromptGuardAttack Scenario=FC2026.05 | 1 | — | — | — | — | — | — | 0 | |
| CleanBase + MetaSecAlignAttack Scenario=NA2026.05 | 1 | — | — | — | — | — | — | 0 | |
| TrustRAGAttack Scenario=NA2026.05 | 2 | — | — | — | — | — | — | 87.2 | |
| TrustRAGAttack Scenario=CI2026.05 | 2 | — | — | — | — | — | — | 2.4 | |
| TrustRAGAttack Scenario=FC2026.05 | 2 | — | — | — | — | — | — | 2.2 | |
| TrustRAGAttack Scenario=CA2026.05 | 2 | — | — | — | — | — | — | 3.2 | |
| TrustRAGAttack Scenario=MA2026.05 | 2 | — | — | — | — | — | — | 2.6 | |
| PromptGuardAttack Scenario=CI2026.05 | 2 | — | — | — | — | — | — | 0 | |
| PromptGuardAttack Scenario=CA2026.05 | 2 | — | — | — | — | — | — | 0 | |
| CleanBaseAttack Scenario=CA2026.05 | 2 | — | — | — | — | — | — | 0 | |
| PromptGuardAttack Scenario=NA2026.05 | 3 | — | — | — | — | — | — | 0 | |
| PromptGuardAttack Scenario=MA2026.05 | 3 | — | — | — | — | — | — | 0 | |
| CleanBaseAttack Scenario=NA2026.05 | 3 | — | — | — | — | — | — | 0 | |
| CleanBase + MetaSecAlignAttack Scenario=CI2026.05 | 3 | — | — | — | — | — | — | 1.2 | |
| CleanBase + MetaSecAlignAttack Scenario=CA2026.05 | 3 | — | — | — | — | — | — | 0 | |
| CleanBaseAttack Scenario=FC2026.05 | 4 | — | — | — | — | — | — | 1.6 | |
| PromptGuardAttack Scenario=PB2026.05 | 5 | — | — | — | — | — | — | 3.6 | |
| CleanBaseAttack Scenario=CI2026.05 | 5 | — | — | — | — | — | — | 1.2 | |
| CleanBase + MetaSecAlignAttack Scenario=FC2026.05 | 5 | — | — | — | — | — | — | 1.6 | |
| CleanBase + MetaSecAlignAttack Scenario=MA2026.05 | 6 | — | — | — | — | — | — | 3.4 | |
| TrustRAGAttack Scenario=PT2026.05 | 7 | — | — | — | — | — | — | 1.4 | |
| PromptGuardAttack Scenario=GL2026.05 | 10 | — | — | — | — | — | — | 7 | |
| CleanBase + MetaSecAlignAttack Scenario=PB2026.05 | 11 | — | — | — | — | — | — | 9.2 | |
| CleanBaseAttack Scenario=MA2026.05 | 12 | — | — | — | — | — | — | 3.4 | |
| CleanBaseAttack Scenario=PB2026.05 | 12 | — | — | — | — | — | — | 9.2 | |
| PromptGuardAttack Scenario=PT2026.05 | 17 | — | — | — | — | — | — | 13.4 | |
| CleanBase + MetaSecAlignAttack Scenario=PT2026.05 | 17 | — | — | — | — | — | — | 16.6 | |
| CleanBaseAttack Scenario=PT2026.05 | 20 | — | — | — | — | — | — | 16.6 | |
| PromptGuardAttack Scenario=PW2026.05 | 21 | — | — | — | — | — | — | 9.2 | |
| DataSentinelAttack Scenario=CA2026.05 | 31 | — | — | — | — | — | — | 12.4 | |
| DataSentinelAttack Scenario=GL2026.05 | 35 | — | — | — | — | — | — | 42 | |
| MetaSecAlignAttack Scenario=FC2026.05 | 39 | — | — | — | — | — | — | 73.6 | |
| MetaSecAlignAttack Scenario=NA2026.05 | 40 | — | — | — | — | — | — | 94.8 | |
| MetaSecAlignAttack Scenario=CA2026.05 | 43 | — | — | — | — | — | — | 64.6 | |
| DataSentinelAttack Scenario=CI2026.05 | 43 | — | — | — | — | — | — | 27.2 | |
| DataSentinelAttack Scenario=FC2026.05 | 43 | — | — | — | — | — | — | 24.4 | |
| MetaSecAlignAttack Scenario=MA2026.05 | 46 | — | — | — | — | — | — | 72.8 | |
| MetaSecAlignAttack Scenario=GL2026.05 | 50 | — | — | — | — | — | — | 100 | |
| MetaSecAlignAttack Scenario=CI2026.05 | 51 | — | — | — | — | — | — | 80.8 | |
| DataSentinelAttack Scenario=MA2026.05 | 53 | — | — | — | — | — | — | 28 | |
| MetaSecAlignAttack Scenario=PT2026.05 | 58 | — | — | — | — | — | — | 97.4 | |
| No DefenseAttack Scenario=PT2026.05 | 60 | — | — | — | — | — | — | 97.4 | |
| CleanBase + MetaSecAlignAttack Scenario=PW2026.05 | 60 | — | — | — | — | — | — | 70.6 | |
| MetaSecAlignAttack Scenario=PB2026.05 | 61 | — | — | — | — | — | — | 98 | |
| DataSentinelAttack Scenario=PT2026.05 | 61 | — | — | — | — | — | — | 97 | |
| No DefenseAttack Scenario=GL2026.05 | 70 | — | — | — | — | — | — | 100 | |
| MetaSecAlignAttack Scenario=PW2026.05 | 70 | — | — | — | — | — | — | 97.4 | |
| CleanBaseAttack Scenario=PW2026.05 | 73 | — | — | — | — | — | — | 70.6 | |
| No DefenseAttack Scenario=CA2026.05 | 74 | — | — | — | — | — | — | 64.6 | |
| DataSentinelAttack Scenario=NA2026.05 | 76 | — | — | — | — | — | — | 77.6 | |
| No DefenseAttack Scenario=FC2026.05 | 85 | — | — | — | — | — | — | 73.6 | |
| DataSentinelAttack Scenario=PB2026.05 | 86 | — | — | — | — | — | — | 97.2 | |
| No DefenseAttack Scenario=PB2026.05 | 88 | — | — | — | — | — | — | 98 | |
| No DefenseAttack Scenario=MA2026.05 | 89 | — | — | — | — | — | — | 72.8 | |
| DataSentinelAttack Scenario=PW2026.05 | 89 | — | — | — | — | — | — | 96 | |
| No DefenseAttack Scenario=PW2026.05 | 91 | — | — | — | — | — | — | 97.4 | |
| No DefenseAttack Scenario=NA2026.05 | 95 | — | — | — | — | — | — | 94.8 | |
| No DefenseAttack Scenario=CI2026.05 | 95 | — | — | — | — | — | — | 80.8 | |
| L1-OnlyNumber of passages=500, Number of queries=200, Confidence Interval=95% Wilson CI2026.04 | — | 0 | 0 | 8 | 0 | 11 | 0 | — | |
| L3-OnlyNumber of passages=500, Number of queries=200, Confidence Interval=95% Wilson CI2026.04 | — | 0 | 0 | 0 | 0 | 0 | 0 | — | |
| No DefenseNumber of passages=500, Number of queries=200, Confidence Interval=95% Wilson CI2026.04 | — | 13 | 8 | 8 | 8.5 | 11 | — | — | |
| RAGDefenderNumber of passages=500, Number of queries=200, Confidence Interval=95% Wilson CI2026.04 | — | 12.5 | 7.5 | 7.5 | 8.5 | 11 | — | — | |
| RAGShieldNumber of passages=500, Number of queries=200, Confidence Interval=95% Wilson CI2026.04 | — | 0 | 0 | 0 | 0 | 0 | 0 | — | |
| RobustRAGNumber of passages=500, Number of queries=200, Confidence Interval=95% Wilson CI2026.04 | — | 0.5 | 0 | 0 | 0 | 0 | — | — |