Safety Evaluation on StrongReject (H, Hp)
62H ScoreSearch-R1
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Search-R1Judge=Claude Sonnet 4.5, Base Model=Qwen2.5-7B-Instruct2026.05 | 62 | 1.89 | |
| Base AgentJudge=Claude Sonnet 4.5, Base Model=Qwen2.5-7B-Instruct2026.05 | 49.2 | 1.96 | |
| Naive RAGBackbone=Qwen2.5-7B-Instruct2026.05 | 48.9 | 2.1 | |
| Naive RAGJudge=Claude Sonnet 4.5, Base Model=Qwen2.5-7B-Instruct2026.05 | 46.7 | 1.98 | |
| Search-R1(79k)Backbone=Qwen2.5-7B-Instruct2026.05 | 41.9 | 1.95 | |
| Base AgentBackbone=Qwen2.5-7B-Instruct2026.05 | 32.3 | 1.96 | |
| Base Agent + Document FilteringBackbone=Qwen2.5-7B-Instruct2026.05 | 30.3 | 2.23 | |
| Base Agent + Query FilteringBackbone=Qwen2.5-7B-Instruct2026.05 | 28.1 | 2.19 | |
| BaseBackbone=Qwen2.5-7B-Instruct2026.05 | 25.9 | 2.13 | |
| BaseJudge=Claude Sonnet 4.5, Base Model=Qwen2.5-7B-Instruct2026.05 | 25.2 | 2.01 | |
| Naive RAGBackbone=Llama3.1-8B-Instruct2026.05 | 24.6 | 2.02 | |
| SafesearchJudge=Claude Sonnet 4.5, Base Model=Qwen2.5-7B-Instruct2026.05 | 24.3 | 2.01 | |
| COMPASSJudge=Claude Sonnet 4.5, Base Model=Qwen2.5-7B-Instruct2026.05 | 23.6 | 1.94 | |
| Search-R1(79k)Backbone=Llama3.1-8B-Instruct2026.05 | 22.5 | 1.56 | |
| Safesearch(79k+12k)Backbone=Llama3.1-8B-Instruct2026.05 | 19.8 | 2.01 | |
| Safesearch(79k+12k)Backbone=Qwen2.5-7B-Instruct2026.05 | 17.9 | 2.5 | |
| COMPASS(8k+2k)Backbone=Qwen2.5-7B-Instruct2026.05 | 16.6 | 2.57 | |
| Base AgentBackbone=Llama3.1-8B-Instruct2026.05 | 10.2 | 1.29 | |
| Base Agent + Query FilteringBackbone=Llama3.1-8B-Instruct2026.05 | 8.6 | 1.36 | |
| Base Agent + Document FilteringBackbone=Llama3.1-8B-Instruct2026.05 | 7 | 1.38 | |
| COMPASS(8k+2k)Backbone=Llama3.1-8B-Instruct2026.05 | 5.8 | 2.06 | |
| BaseBackbone=Llama3.1-8B-Instruct2026.05 | 4.8 | 1.85 |