Safety and Helpfulness Evaluation on StrongREJECT
0.2Harm RatePPO SafeSearch
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| PPO SafeSearchBackbone=Qwen-2.5-7B-Instruct2025.10 | 0.2 | 2.65 | |
| PPO SafeSearch w/o qr. & hf.Backbone=Qwen-2.5-7B-Instruct2025.10 | 0.2 | 1.61 | |
| GRPO SafeSearchBackbone=Qwen-2.5-7B-Instruct2025.10 | 0.5 | 2.59 | |
| PPO SafeSearch w/o hf.Backbone=Qwen-2.5-7B-Instruct2025.10 | 1.6 | 1.7 | |
| Base LLMBackbone=Mistral-NeMo-8B-Instruct2025.10 | 1.9 | 3.32 | |
| PPO SafeSearch w/o qr.Backbone=Qwen-2.5-7B-Instruct2025.10 | 5.8 | 2.36 | |
| Base AgentBackbone=Qwen-2.5-14B-Instruct2025.10 | 7.9 | 1.84 | |
| Base LLMBackbone=Qwen-2.5-14B-Instruct2025.10 | 8.8 | 1.78 | |
| Base AgentBackbone=Mistral-NeMo-8B-Instruct2025.10 | 11.2 | 3.15 | |
| Naive RAGBackbone=Mistral-NeMo-8B-Instruct2025.10 | 11.4 | 3.26 | |
| Base LLMBackbone=Qwen-2.5-7B-Instruct2025.10 | 14.3 | 2.15 | |
| PPO SafeSearchBackbone=Qwen-2.5-3B-Instruct2025.10 | 15 | 2.22 | |
| Naive RAGBackbone=Qwen-2.5-14B-Instruct2025.10 | 17.3 | 1.74 | |
| GRPO Utility-Only Ft.Backbone=Qwen-2.5-7B-Instruct2025.10 | 20 | 1.96 | |
| BaseBackbone=Qwen-2.5-7B-Instruct2025.10 | 27.3 | 2.19 | |
| Base AgentBackbone=Qwen-2.5-7B-Instruct2025.10 | 27.3 | 2.19 | |
| Search BlockBackbone=Qwen-2.5-7B-Instruct2025.10 | 31.1 | 2.18 | |
| Base LLMBackbone=Qwen-2.5-3B-Instruct2025.10 | 31.3 | 2.17 | |
| Document FilterBackbone=Qwen-2.5-7B-Instruct2025.10 | 35.2 | 2.27 | |
| PPO Utility-Only Ft.Backbone=Qwen-2.5-7B-Instruct2025.10 | 36.3 | 2.24 | |
| Ft. AgentBackbone=Qwen-2.5-7B-Instruct2025.10 | 36.3 | 2.24 | |
| Naive RAGBackbone=Qwen-2.5-7B-Instruct2025.10 | 51 | 2.17 | |
| BaseBackbone=Qwen-2.5-3B-Instruct2025.10 | 52.7 | 2.33 | |
| Base AgentBackbone=Qwen-2.5-3B-Instruct2025.10 | 52.7 | 2.33 | |
| Search BlockBackbone=Qwen-2.5-3B-Instruct2025.10 | 55.3 | 2.2 | |
| Document FilterBackbone=Qwen-2.5-3B-Instruct2025.10 | 62.1 | 2.21 | |
| PPO Utility-Only Ft.Backbone=Qwen-2.5-3B-Instruct2025.10 | 63.5 | 2.22 | |
| Ft. AgentBackbone=Qwen-2.5-3B-Instruct2025.10 | 63.5 | 2.22 | |
| Naive RAGBackbone=Qwen-2.5-3B-Instruct2025.10 | 70.4 | 2.12 |