List your company
ResearchFormerly WizWand

Question Answering on RQA

88.2ASRVanilla
1.46423.98246.569.018Jun 4, 2025Aug 1, 2025Sep 29, 2025Nov 26, 2025Jan 24, 2026Mar 23, 2026May 21, 2026
Updated 5mo ago

Evaluation results

MethodLinks
VanillaLLM=Llama2-C2025.0688.2—5.8
Vanilla RAGTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.058415—
Vanilla RAGTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.057919—
Vanilla RAGTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=22026.057425—
InstructRAGTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=32026.056823—
Vanilla RAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=22026.056535—
InstructRAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.056221—
Vanilla RAGTop-k=10, Pos=Pos 10, Generator LLM=Grok-4-fast2026.055737—
InstructRAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.055527—
VanillaLLM=Deepseek-R12025.0649.6—24.3
AstuteRAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.054727—
InstructRAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=42026.054633—
Vanilla RAGTop-k=50, Pos=Pos 50, Generator LLM=Grok-4-fast2026.054342—
Vanilla RAGTop-k=50, Pos=Pos 25, Generator LLM=Grok-4-fast2026.054245—
AstuteRAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.054132—
Vanilla RAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.053953—
VanillaLLM=GPT-4o2025.0633.4—52.4
InstructRAGTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.053353—
Vanilla RAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.053257—
InstructRAGTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.053259—
InstructRAGTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.053226—
VanillaLLM=Llama-3.12025.0629.4—21
Vanilla RAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.052859—
ReliabilityRAGTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.052862—
InstructRAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.052753—
VanillaLLM=Mistral-7B2025.0626.6—52.2
VanillaLLM=GPT-4o2025.0626.6—55.6
Vanilla RAGTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.052665—
AV FilterLLM=Deepseek-R1, alpha=52025.0625.6—39.3
InstructRAGTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.052512—
ReliabilityRAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.052468—
Vanilla RAGTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.052464—
VanillaLLM=Mistral-7B2025.0623.4—50
ReliabilityRAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.052367—
ReliabilityRAGTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.052152—
AV FilterLLM=Deepseek-R1, alpha=52025.0620.3—45.3
ReliabilityRAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=41, NLI(s)=0.40, Inference(s)=0.1072, Total(s)=432026.051867—
VanillaLLM=Llama2-C2025.0617.2—57.4
DecodingLLM=Llama-3.12025.0617—51
VanillaLLM=Deepseek-R12025.0617—46.3
RobustRAGTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.051669—
RADARTop-k=10, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.051669—
ReliabilityRAGTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=20, NLI(s)=0.32, Inference(s)=0.0005, Total(s)=212026.051569—
RobustRAGTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.051569—
AV FilterLLM=Deepseek-R1, alpha=102025.0614.6—46.6
VanillaLLM=Llama-3.12025.0614—48.4
AV FilterLLM=Deepseek-R1, alpha=102025.0614—47
RADARTop-k=50, Pos=1, Attack Type=PIA attack, Generator LLM=GPT-4o2026.051374—
DecodingLLM=Mistral-7B2025.0613—54
DecodingLLM=Llama2-C2025.0612—38
RADARTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=20, NLI(s)=0.32, Inference(s)=0.0007, Total(s)=212026.051169—
RobustRAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.051174—
RADARTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.051175—
AV FilterLLM=GPT-4o, alpha=102025.0611—57.2
AV FilterLLM=GPT-4o, alpha=52025.0610.6—57.8
AV FilterLLM=Deepseek-R1, alpha=∞2025.0610.3—50.3
RobustRAGTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.051072—
ReliabilityRAGTop-k=10, Pos=Pos 10, Generator LLM=Grok-4-fast2026.051060—
RobustRAGTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.051063—
AV FilterLLM=GPT-4o, alpha=∞2025.069.2—61.4
RobustRAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=86, Inference(s)=0.0945, Total(s)=942026.05969—
ReliabilityRAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05976—
RADARTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05976—
RADARTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05966—
AV FilterLLM=Llama2-C, alpha=102025.068.8—54.8
AV FilterLLM=Mistral-7B, alpha=∞2025.068.6—56.8
AV FilterLLM=Deepseek-R1, alpha=∞2025.068.6—53
AV FilterLLM=Mistral-7B, alpha=102025.068.4—56.8
RobustRAGTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05872—
RobustRAGTop-k=10, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05854—
ReliabilityRAGTop-k=50, Pos=Pos 25, Generator LLM=Grok-4-fast2026.05876—
AV FilterLLM=Mistral-7B, alpha=52025.067.4—58.2
AV FilterLLM=Llama-3.1, alpha=52025.067.2—53.8
RobustRAGTop-k=10, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=20, Inference(s)=0.0365, Total(s)=212026.05764—
RADARTop-k=50, Pos=Pos 1, Generator LLM=Grok-4-fast2026.05768—
KeywordLLM=Llama-3.12025.067—61
DecodingLLM=Llama-3.12025.067—55
AV FilterLLM=GPT-4o, alpha=∞2025.067—61.2
AV FilterLLM=Llama2-C, alpha=52025.067—55.4
AV FilterLLM=Llama2-C, alpha=∞2025.066.6—56.8
AV FilterLLM=Llama-3.1, alpha=102025.066.4—53.2
AV FilterLLM=Llama-3.1, alpha=∞2025.066.2—57.4
AstuteRAGTop-k=10, Pos=10, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05635—
AstuteRAGTop-k=50, Pos=25, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05628—
KeywordLLM=Mistral-7B2025.066—54
KeywordLLM=Llama2-C2025.066—53
KeywordLLM=Mistral-7B2025.066—55
KeywordLLM=Llama-3.12025.066—57
AV FilterLLM=GPT-4o, alpha=52025.065.2—63.6
AstuteRAGTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Total(s)=72026.05521—
RADARTop-k=50, Backbone=Deepseek, Attack Position=Pos 1, Atomic Gen.(s)=86, NLI(s)=0.52, Inference(s)=0.0025, Total(s)=872026.05572—
AstuteRAGTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05543—
RADARTop-k=50, Pos=50, Attack Type=PIA attack, Generator LLM=GPT-4o2026.05575—
InstructRAGTop-k=10, Pos=Pos 10, Generator LLM=Grok-4-fast2026.05547—
RADARTop-k=10, Pos=Pos 10, Generator LLM=Grok-4-fast2026.05569—
DecodingLLM=Mistral-7B2025.065—55
AV FilterLLM=Llama2-C, alpha=∞2025.065—60.2
KeywordLLM=Llama2-C2025.065—53
DecodingLLM=Llama2-C2025.065—40
AV FilterLLM=GPT-4o, alpha=102025.064.8—63.6