Adversarial Attack Effectiveness in RAG Pipeline on ESCI position 6 (test)
100Success Rate @10IOA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IOARetriever=BM25, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 65.5 | 48 | |
| ReviewRetriever=BM25, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 72.5 | 60 | |
| ReasonRetriever=BM25, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 73.5 | 57 | |
| TAPRetriever=BM25, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 62.5 | 57.5 | |
| SRPRetriever=BM25, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 21 | 6 | |
| RAFRetriever=BM25, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 20.5 | 6.5 | |
| STSRetriever=BM25, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 16 | 5 | |
| IOARetriever=Dense, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 60.5 | 42.5 | |
| ReviewRetriever=Dense, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 69 | 56 | |
| ReasonRetriever=Dense, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 68 | 55 | |
| TAPRetriever=Dense, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 60 | 54.5 | |
| SRPRetriever=Dense, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 14.5 | 3.5 | |
| RAFRetriever=Dense, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 16.5 | 5 | |
| STSRetriever=Dense, Evaluation Mode=FC, Injection Position=62026.05 | 100 | 11.5 | 3 | |
| ReasonRetriever=Dense, Evaluation Mode=E2E, Injection Position=62026.05 | 99.5 | 86 | 75.5 | |
| ReviewRetriever=Dense, Evaluation Mode=E2E, Injection Position=62026.05 | 96.5 | 79.5 | 67.5 | |
| SRPRetriever=Dense, Evaluation Mode=E2E, Injection Position=62026.05 | 93 | 12.5 | 3 | |
| TAPRetriever=BM25, Evaluation Mode=E2E, Injection Position=62026.05 | 92 | 60.5 | 55 | |
| ReasonRetriever=BM25, Evaluation Mode=E2E, Injection Position=62026.05 | 91 | 76 | 63.5 | |
| RAFRetriever=Dense, Evaluation Mode=E2E, Injection Position=62026.05 | 91 | 12.5 | 2.5 | |
| STSRetriever=Dense, Evaluation Mode=E2E, Injection Position=62026.05 | 91 | 15.5 | 3 | |
| ReviewRetriever=BM25, Evaluation Mode=E2E, Injection Position=62026.05 | 90.5 | 72 | 65 | |
| RAFRetriever=BM25, Evaluation Mode=E2E, Injection Position=62026.05 | 90 | 16 | 6.5 | |
| SRPRetriever=BM25, Evaluation Mode=E2E, Injection Position=62026.05 | 88.5 | 14.5 | 4 | |
| TAPRetriever=Dense, Evaluation Mode=E2E, Injection Position=62026.05 | 88.5 | 63 | 57.5 | |
| IOARetriever=Dense, Evaluation Mode=E2E, Injection Position=62026.05 | 81.5 | 43.5 | 30 | |
| STSRetriever=BM25, Evaluation Mode=E2E, Injection Position=62026.05 | 79 | 4.5 | 2 | |
| IOARetriever=BM25, Evaluation Mode=E2E, Injection Position=62026.05 | 67.5 | 40.5 | 28 |