Defense against Indirect Prompt Injection on Filtered QA dataset

97.65ASR (Naive)Reminder

Evaluation Results

Method
Reminder2024.11	97.65	97.95	100	100	100
Instructional2024.11	95.55	95.75	99.95	100	100
Spotlight2024.11	94.35	96.45	100	100	100
Spotlight2024.11	94.35	96.45	100	100	100
None2024.11	92.45	95.9	100	100	100
None2024.11	85.9	91.1	81.7	95.25	92.3
Reminder2024.11	79.05	77.3	71.75	84.35	80.65
Isolation2024.11	77.8	88.85	99.35	99.7	100
Isolation2024.11	76.75	85	89.75	91.7	88.75
Instructional2024.11	60.15	68.35	88.1	84.7	84.85
None2024.11	10.55	53.35	88.25	75.3	86
Reminder2024.11	10.55	39.9	67.5	37.85	51.2
Spotlight2024.11	8.8	32.85	76.35	74.45	56.6
Instructional2024.11	6.95	35	80.1	64.45	62.75
Sandwich2024.11	4.8	6.15	14	34.2	34.6
Sandwich2024.11	2.5	3.05	22.9	3.35	9.55
Isolation2024.11	2.2	33.75	83.35	67.4	77.75
Fake Completion Defense2024.11	1.75	2.45	8.75	0.8	0.6
Escape Defense2024.11	1.45	1.7	0.75	0.68	4.95
Escape Defense2024.11	1.25	2.7	1.05	0.9	1.65
Ignore Defense2024.11	0.85	0.7	0.8	0.95	4.1
Sandwich2024.11	0.45	9.35	49.55	7.3	21.25
Fake Completion Defense2024.11	0.3	0.7	0.55	0.45	0.3
Escape Defense2024.11	0.25	1.7	1.05	0.55	1.45
Ignore Defense2024.11	0.25	0.3	0.35	0.45	1.1
Fake Completion Defense2024.11	0.1	1.8	17.7	0.05	0.1
Fake Completion Defense with Template2024.11	0.1	0.25	0.2	0.15	0.05
Ignore Defense2024.11	0.05	0.35	0.3	0.1	1.35
Fake Completion Defense with Template2024.11	0.05	0.05	0.3	0.05	0.05
Fake Completion Defense with Template2024.11	0.05	0.25	0.2	0.15	0.05