Multi-hop Question Answering on HotpotQA (Adversarial Evaluation)
94.5Accuracy (No Attack)DeepSeek-R1 (subset)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DeepSeek-R1 (subset)prompting_strategy=batch prompting2025.03 | 94.5 | 92.8 | 92.8 | 96.7 | 58.8 | |
| Claude-3.5-Sonnetprompting_strategy=batch prompting2025.03 | 92.8 | 72.8 | 79.9 | 63.7 | 62.5 | |
| GPT-4oprompting_strategy=batch prompting2025.03 | 88.7 | 93.7 | 79.9 | 94 | 54.5 | |
| GPT-4o-miniprompting_strategy=batch prompting2025.03 | 77.8 | 97.8 | 72.5 | 86.6 | 51.4 | |
| Llama3-70b-Instructprompting_strategy=batch prompting2025.03 | 77.5 | 83.5 | 75.2 | 59.6 | 51.2 | |
| Qwen2.5-7B-Instructprompting_strategy=batch prompting2025.03 | 73 | 68.2 | 69.5 | 42.9 | 49.6 | |
| Llama3.2-3B-Instructprompting_strategy=batch prompting2025.03 | 71.2 | 67.3 | 64.6 | 55.6 | 41.2 |