Truthfulness Evaluation on TruthfulQA (Accuracy (%))
75AccuracyInitial RLHF
Evaluation results
| Method | Links | |
|---|---|---|
| Initial RLHFTraining Configuration=Initial RLHF2026.04 | 75 | |
| ARES (Qwen)Training Configuration=Ours (Qwen)2026.04 | 73 | |
| ARES (Ministral)Training Configuration=Ours (Ministral)2026.04 | 72 | |
| General Safe-RLHFTraining Configuration=General Safe-RLHF2026.04 | 71 | |
| BaseBackbone=Qwen2.5-14B2026.02 | 70.8 | |
| Original ModelTraining Configuration=Original Model2026.04 | 69 | |
| GRP-OblitBackbone=Qwen2.5-14B2026.02 | 66.7 | |
| GRP-Oblit-1Backbone=Qwen2.5-14B2026.02 | 66.5 | |
| AbliterationBackbone=Qwen2.5-14B2026.02 | 65.3 | |
| BaseBackbone=Qwen2.5-7B2026.02 | 63 | |
| BaseBackbone=Ministral3-8B Instruct2026.02 | 62.6 | |
| GRP-OblitBackbone=Qwen2.5-7B2026.02 | 62.6 | |
| BaseBackbone=Ministral3-14B Instruct2026.02 | 62.5 | |
| GRP-Oblit-1Backbone=Qwen2.5-7B2026.02 | 61.8 | |
| GRP-Oblit-1Backbone=Gemma3-12B2026.02 | 61.6 | |
| BaseBackbone=Gemma2-9B2026.02 | 61.4 | |
| BaseBackbone=Gemma3-12B2026.02 | 60.8 | |
| GRP-OblitBackbone=Ministral3-14B Instruct2026.02 | 60.5 | |
| GRP-OblitBackbone=Gemma3-12B2026.02 | 59.4 | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct2026.03 | 58.8 | |
| TwinBreakBackbone=Gemma3-12B2026.02 | 58.1 | |
| GRP-OblitBackbone=Ministral3-8B Instruct2026.02 | 58 | |
| GRP-Oblit-1Backbone=Ministral3-8B Instruct2026.02 | 58 | |
| AbliterationBackbone=Qwen2.5-7B2026.02 | 57.6 | |
| RKL-GRPOBackbone=Qwen2.5-7B2026.01 | 57.6 | |
| Deepseek 7BArchitecture=Autoregressive, Parameters=7B, Number of shots=02026.01 | 57.4 | |
| AbliterationBackbone=Llama3.1-8B2026.02 | 57.1 | |
| GRP-OblitBackbone=GPT-OSS-20B2026.02 | 57 | |
| BaseBackbone=Ministral3-14B Reasoning2026.02 | 57 | |
| Base ModelBackbone=Qwen2.5-7B2026.01 | 56.4 | |
| Qwen2.5 7BArchitecture=Autoregressive, Parameters=7B, Number of shots=02026.01 | 56.4 | |
| AbliterationBackbone=Gemma2-9B2026.02 | 55.8 | |
| GRP-Oblit-1Backbone=Ministral3-14B Instruct2026.02 | 55.8 | |
| TwinBreakBackbone=Qwen2.5-14B2026.02 | 55.8 | |
| BaseBackbone=DeepSeek-R1-Qwen-14B2026.02 | 55.7 | |
| GRP-Oblit-1Backbone=Gemma2-9B2026.02 | 55.7 | |
| TwinBreakBackbone=Gemma2-9B2026.02 | 55.7 | |
| CARE-GRPOBackbone=Qwen2.5-7B2026.01 | 55.7 | |
| BaseBackbone=Ministral3-8B Reasoning2026.02 | 55.5 | |
| BaseBackbone=Qwen3-14B2026.02 | 55.5 | |
| BaseBackbone=GPT-OSS-20B2026.02 | 55.4 | |
| BaseBackbone=Llama3.1-8B2026.02 | 55.2 | |
| GRP-OblitBackbone=DeepSeek-R1-Qwen-14B2026.02 | 55.1 | |
| RKL-GSPOBackbone=Qwen2.5-7B2026.01 | 55 | |
| RKL-DAPOBackbone=Qwen2.5-7B2026.01 | 54.8 | |
| CARE-GSPOBackbone=Qwen2.5-7B2026.01 | 54.8 | |
| GRP-OblitBackbone=Llama3.1-8B2026.02 | 54.7 | |
| GRP-OblitBackbone=Gemma2-9B2026.02 | 54.2 | |
| Qwen2 7BArchitecture=Autoregressive, Parameters=7B, Number of shots=02026.01 | 54.2 | |
| CARE-DAPOBackbone=Qwen2.5-7B2026.01 | 54.1 | |
| GRP-OblitBackbone=Ministral3-8B Reasoning2026.02 | 53.8 | |
| AbliterationBackbone=DeepSeek-R1-Qwen-14B2026.02 | 53.4 | |
| GRP-Oblit-1Backbone=Ministral3-8B Reasoning2026.02 | 53.1 | |
| GRP-OblitBackbone=Qwen3-14B2026.02 | 52.9 | |
| GRP-OblitBackbone=Qwen3-8B2026.02 | 52.9 | |
| GRP-Oblit-1Backbone=GPT-OSS-20B2026.02 | 52.6 | |
| BaseBackbone=Qwen3-8B2026.02 | 52.6 | |
| Llama-3.1-8B-InstructModel=Llama-3.1-8B-Instruct2026.03 | 52.4 | |
| Ministral-3-8B-Instruct-2512Model=Ministral-3-8B-Instruct-25122026.03 | 51.9 | |
| GRP-Oblit-1Backbone=Qwen3-8B2026.02 | 51.8 | |
| GRP-Oblit-1Backbone=Ministral3-14B Reasoning2026.02 | 51.6 | |
| TwinBreakBackbone=Qwen3-14B2026.02 | 51.6 | |
| TwinBreakBackbone=DeepSeek-R1-Llama-8B2026.02 | 51.2 | |
| TwinBreakBackbone=Qwen2.5-7B2026.02 | 51.2 | |
| AbliterationBackbone=Ministral3-8B Instruct2026.02 | 51.1 | |
| GRP-Oblit-1Backbone=Llama3.1-8B2026.02 | 51 | |
| GRP-OblitBackbone=Ministral3-14B Reasoning2026.02 | 50.8 | |
| GRP-Oblit-1Backbone=DeepSeek-R1-Qwen-14B2026.02 | 50.7 | |
| BaseBackbone=DeepSeek-R1-Llama-8B2026.02 | 50.5 | |
| TwinBreakBackbone=Llama3.1-8B2026.02 | 50.2 | |
| VDLMArchitecture=Diffusion, Number of shots=0, Protocol=Same protocol (*)2026.01 | 50.2 | |
| AbliterationBackbone=DeepSeek-R1-Llama-8B2026.02 | 50.1 | |
| AbliterationBackbone=Qwen3-8B2026.02 | 50 | |
| TwinBreakBackbone=DeepSeek-R1-Qwen-14B2026.02 | 49.7 | |
| GRP-OblitBackbone=DeepSeek-R1-Llama-8B2026.02 | 49.6 | |
| GRP-Oblit-1Backbone=Qwen3-14B2026.02 | 49.4 | |
| TwinBreakBackbone=DeepSeek-R1-Qwen-7B2026.02 | 49.3 | |
| TwinBreakBackbone=Qwen3-8B2026.02 | 48.9 | |
| AbliterationBackbone=Ministral3-14B Instruct2026.02 | 48.7 | |
| REWARDAGENT_LLAMA-OPDPO Training Data=On-Policy, Preference Pair Construction=REWARDAGENT2025.02 | 48.5 | |
| AbliterationBackbone=Ministral3-8B Reasoning2026.02 | 47.9 | |
| AbliterationBackbone=DeepSeek-R1-Qwen-7B2026.02 | 47.8 | |
| AbliterationBackbone=Ministral3-14B Reasoning2026.02 | 47.8 | |
| AbliterationBackbone=GPT-OSS-20B2026.02 | 47.6 | |
| AbliterationBackbone=Qwen3-14B2026.02 | 46.5 | |
| BaseBackbone=DeepSeek-R1-Qwen-7B2026.02 | 46.3 | |
| LLaDAArchitecture=Diffusion, Number of shots=0, Protocol=Same protocol (*)2026.01 | 46.1 | |
| GRP-Oblit-1Backbone=DeepSeek-R1-Qwen-7B2026.02 | 46 | |
| GRP-OblitBackbone=DeepSeek-R1-Qwen-7B2026.02 | 45.8 | |
| GRP-Oblit-1Backbone=DeepSeek-R1-Llama-8B2026.02 | 45.5 | |
| ArmoRM-UFDPO Training Data=UltraFeedback, Preference Pair Construction=ArmoRM2025.02 | 45 | |
| ArmoRM-OPDPO Training Data=On-Policy, Preference Pair Construction=ArmoRM2025.02 | 44.4 | |
| REWARDAGENT_LLAMA-UFDPO Training Data=UltraFeedback, Preference Pair Construction=REWARDAGENT2025.02 | 44.1 | |
| LLaMA3 AR 8BArchitecture=Autoregressive, Parameters=8B, Number of shots=02026.01 | 44 | |
| Mistral 7BArchitecture=Autoregressive, Parameters=7B, Number of shots=02026.01 | 42.2 | |
| Original UFDPO Training Data=Original UltraFeedback2025.02 | 42 | |
| DAPO (No Constraint)Backbone=Qwen2.5-7B2026.01 | 42 | |
| GRPO (No Constraint)Backbone=Qwen2.5-7B2026.01 | 41.2 | |
| GSPO (No Constraint)Backbone=Qwen2.5-7B2026.01 | 40.1 | |
| Zephyr-7b-sft-fullDPO Training Data=None (Base Model)2025.02 | 39.5 |