Utility Evaluation on Just-Eval
4.83Just-Eval Average ScoreNo Defense
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| No DefenseBase Model=Qwen32026.02 | 4.83 | — | — | — | — | — | |
| Self-ReminderBase Model=Qwen32026.02 | 4.81 | — | — | — | — | — | |
| SSDBase Model=Qwen32026.02 | 4.78 | — | — | — | — | — | |
| Self-ExamBase Model=Qwen32026.02 | 4.77 | — | — | — | — | — | |
| No DefenseBase Model=Llama32026.02 | 4.75 | — | — | — | — | — | |
| NGSDBase Model=Qwen32026.02 | 4.73 | — | — | — | — | — | |
| NGSDBase Model=Llama32026.02 | 4.47 | — | — | — | — | — | |
| SafeDecodingBase Model=Llama32026.02 | 4.45 | — | — | — | — | — | |
| SafeDecodingBase Model=Qwen32026.02 | 4.31 | — | — | — | — | — | |
| No DefenseBase Model=Vicuna2026.02 | 4.29 | — | — | — | — | — | |
| SSDBase Model=Llama32026.02 | 4.23 | — | — | — | — | — | |
| Self-ReminderBase Model=Vicuna2026.02 | 4.22 | — | — | — | — | — | |
| NGSDBase Model=Vicuna2026.02 | 4.07 | — | — | — | — | — | |
| No DefenseModel=Llama2-7B, ATGR=1.00×2025.10 | 4.063 | 4.039 | 4.698 | 4.286 | 3.231 | 4.059 | |
| SSDBase Model=Vicuna2026.02 | 4.04 | — | — | — | — | — | |
| SafeDecodingBase Model=Vicuna2026.02 | 4.01 | — | — | — | — | — | |
| No DefenseBackbone=Vicuna-13B2025.10 | 4.007 | — | — | — | — | — | |
| SAIDModel=Llama2-7B, ATGR=1.04×2025.10 | 3.997 | 3.958 | 4.621 | 4.259 | 3.161 | 3.987 | |
| Self-ReminderBase Model=Llama32026.02 | 3.99 | — | — | — | — | — | |
| Self-ExamBase Model=Llama32026.02 | 3.99 | — | — | — | — | — | |
| Self-ExaminationBackbone=Vicuna-13B2025.10 | 3.99 | — | — | — | — | — | |
| SAIDBackbone=Vicuna-13B2025.10 | 3.974 | — | — | — | — | — | |
| SafeDecodingBackbone=Vicuna-13B2025.10 | 3.926 | — | — | — | — | — | |
| SafeDecodingModel=Llama2-7B, ATGR=1.06×2025.10 | 3.871 | 3.734 | 4.605 | 4.193 | 3.069 | 3.756 | |
| Self-ExamBase Model=Vicuna2026.02 | 3.87 | — | — | — | — | — | |
| ParaphraseModel=Llama2-7B, ATGR=1.40×2025.10 | 3.852 | 3.744 | 4.472 | 4.183 | 3.032 | 3.831 | |
| ICDBackbone=Vicuna-13B2025.10 | 3.85 | — | — | — | — | — | |
| No DefenseModel=Vicuna-7B, ATGR=1.00×2025.10 | 3.842 | 4.208 | 4.547 | 4.159 | 3.03 | 3.267 | |
| ParaphraseBackbone=Vicuna-13B2025.10 | 3.84 | — | — | — | — | — | |
| Self-ExaminationModel=Vicuna-7B, ATGR=1.34×2025.10 | 3.828 | 4.149 | 4.549 | 4.168 | 3.023 | 3.25 | |
| ICDModel=Vicuna-7B, ATGR=1.05×2025.10 | 3.811 | 4.145 | 4.583 | 4.219 | 2.903 | 3.204 | |
| PPLModel=Llama2-7B, ATGR=0.90×2025.10 | 3.769 | 3.687 | 4.421 | 4.11 | 2.965 | 3.665 | |
| SAIDModel=Vicuna-7B, ATGR=1.32×2025.10 | 3.758 | 4.056 | 4.517 | 4.081 | 2.927 | 3.209 | |
| SafeDecodingModel=Vicuna-7B, ATGR=1.06×2025.10 | 3.744 | 3.93 | 4.554 | 4.213 | 2.851 | 3.172 | |
| PPLBackbone=Vicuna-13B2025.10 | 3.718 | — | — | — | — | — | |
| ParaphraseModel=Vicuna-7B, ATGR=2.99×2025.10 | 3.714 | 3.905 | 4.427 | 4.15 | 2.913 | 3.174 | |
| ICDModel=Llama2-7B, ATGR=0.99×2025.10 | 3.714 | 3.875 | 4.589 | 4.196 | 2.696 | 3.214 | |
| No DefenseModel=Guanaco, ATGR=1.00×2025.10 | 3.662 | 3.959 | 4.347 | 3.912 | 2.881 | 3.212 | |
| PPLModel=Vicuna-7B, ATGR=1.02×2025.10 | 3.592 | 3.815 | 4.3 | 4.033 | 2.795 | 3.016 | |
| RetokenizationBackbone=Vicuna-13B2025.10 | 3.578 | — | — | — | — | — | |
| ICDModel=Guanaco, ATGR=1.16×2025.10 | 3.42 | 3.566 | 4.118 | 3.881 | 2.618 | 2.92 | |
| SAIDModel=Guanaco, ATGR=1.24×2025.10 | 3.389 | 3.517 | 4.103 | 3.82 | 2.597 | 2.909 | |
| ParaphraseModel=Guanaco, ATGR=2.22×2025.10 | 3.381 | 3.472 | 3.996 | 3.839 | 2.66 | 2.935 | |
| SafeDecodingModel=Guanaco, ATGR=1.85×2025.10 | 3.346 | 3.215 | 4.135 | 3.999 | 2.399 | 2.984 | |
| PPLModel=Guanaco, ATGR=1.55×2025.10 | 3.322 | 3.438 | 3.931 | 3.837 | 2.592 | 2.809 | |
| RetokenizationModel=Vicuna-7B, ATGR=1.06×2025.10 | 3.318 | 3.252 | 4.09 | 3.807 | 2.51 | 2.931 | |
| RetokenizationModel=Llama2-7B, ATGR=0.99×2025.10 | 3.265 | 2.845 | 4.095 | 3.709 | 2.405 | 3.272 | |
| RetokenizationModel=Guanaco, ATGR=0.96×2025.10 | 3.08 | 2.924 | 3.744 | 3.669 | 2.379 | 2.682 | |
| Self-ExaminationModel=Guanaco, ATGR=3.16×2025.10 | 2.913 | 2.819 | 3.59 | 3.623 | 2.192 | 2.339 | |
| Self-ExaminationModel=Llama2-7B, ATGR=1.48×2025.10 | 2.076 | 1.481 | 2.961 | 3.422 | 1.149 | 1.368 |