Question Answering on MuSiQue (Guard Rate)
100Guard RateNaive
Evaluation Results
| Method | Links | |
|---|---|---|
| NaiveFramework=ReAct, LLM Backbone=Qwen2.5-32B-Inst2026.07 | 100 | |
| IgnoreFramework=ReAct, LLM Backbone=Qwen2.5-32B-Inst2026.07 | 100 | |
| Fake CompletionFramework=ReAct, LLM Backbone=Qwen2.5-32B-Inst2026.07 | 100 | |
| CombinedFramework=ReAct, LLM Backbone=Llama-3.3-70B-Inst2026.07 | 100 | |
| CombinedFramework=ReAct, LLM Backbone=Qwen2.5-32B-Inst2026.07 | 98.9 | |
| IgnoreFramework=ReAct, LLM Backbone=Llama-3.3-70B-Inst2026.07 | 98.9 | |
| Fake CompletionFramework=ReAct, LLM Backbone=Llama-3.3-70B-Inst2026.07 | 98.9 | |
| NaiveFramework=ReAct, LLM Backbone=Llama-3.3-70B-Inst2026.07 | 97.5 | |
| PoisonedRAGFramework=ReAct, LLM Backbone=Qwen2.5-32B-Inst2026.07 | 93.9 | |
| PoisonedRAGFramework=ReAct, LLM Backbone=Llama-3.3-70B-Inst2026.07 | 93.9 | |
| PARADOXFramework=ReAct, LLM Backbone=Llama-3.3-70B-Inst2026.07 | 93 | |
| PARADOXFramework=ReAct, LLM Backbone=Qwen2.5-32B-Inst2026.07 | 91.9 | |
| TopicFramework=ReAct, LLM Backbone=Qwen2.5-32B-Inst2026.07 | 83.2 | |
| TopicFramework=ReAct, LLM Backbone=Llama-3.3-70B-Inst2026.07 | 81.2 | |
| PARADOXFramework=WebThinker, LLM Backbone=QwQ-32B2026.07 | 74.7 | |
| NaiveFramework=WebThinker, LLM Backbone=QwQ-32B2026.07 | 58.6 | |
| Fake CompletionFramework=WebThinker, LLM Backbone=QwQ-32B2026.07 | 58.5 | |
| PoisonedRAGFramework=WebThinker, LLM Backbone=QwQ-32B2026.07 | 52.8 | |
| PARADOXFramework=WebThinker, LLM Backbone=DeepSeek-32B2026.07 | 47.9 | |
| IgnoreFramework=WebThinker, LLM Backbone=QwQ-32B2026.07 | 42.7 | |
| TopicFramework=WebThinker, LLM Backbone=QwQ-32B2026.07 | 41.6 | |
| TopicFramework=WebThinker, LLM Backbone=DeepSeek-32B2026.07 | 40 | |
| CombinedFramework=WebThinker, LLM Backbone=QwQ-32B2026.07 | 35.7 | |
| PoisonedRAGFramework=WebThinker, LLM Backbone=DeepSeek-32B2026.07 | 33.7 | |
| NaiveFramework=WebThinker, LLM Backbone=DeepSeek-32B2026.07 | 31.7 | |
| Fake CompletionFramework=WebThinker, LLM Backbone=DeepSeek-32B2026.07 | 31.1 | |
| IgnoreFramework=WebThinker, LLM Backbone=DeepSeek-32B2026.07 | 25.3 | |
| CombinedFramework=WebThinker, LLM Backbone=DeepSeek-32B2026.07 | 16.7 | |
| KidnapRAGFramework=WebThinker, LLM Backbone=DeepSeek-32B2026.07 | 7.9 | |
| KidnapRAGFramework=WebThinker, LLM Backbone=QwQ-32B2026.07 | 6.2 | |
| KidnapRAGFramework=ReAct, LLM Backbone=Qwen2.5-32B-Inst2026.07 | 0 | |
| KidnapRAGFramework=ReAct, LLM Backbone=Llama-3.3-70B-Inst2026.07 | 0 |