Detoxification on RealToxicityPrompts challenging
0.062Max ToxicityRAD
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| RADBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=5002026.01 | 0.062 | 0.055 | 35.79 | 0.005 | — | |
| RADBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=3002026.01 | 0.07 | 0.058 | 19.46 | 0.006 | — | |
| RADModel=Gemma 2-2B, Temperature=0.1, Beta (β)=5002026.01 | 0.089 | 0.075 | 30.3 | 0.019 | — | |
| RADModel=Gemma 2-2B, Temperature=0.1, Beta (β)=3002026.01 | 0.1 | 0.083 | 17.99 | 0.029 | — | |
| DeSteinBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=32026.01 | 0.116 | 0.087 | 8.72 | 0.062 | — | |
| RADBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=1002026.01 | 0.134 | 0.102 | 7.21 | 0.046 | — | |
| RADModel=Gemma 2-2B, Temperature=0.1, Beta (β)=1002026.01 | 0.149 | 0.115 | 5.99 | 0.069 | — | |
| TIDEModel=Gemma 2-2B, Temperature=0.12026.01 | 0.151 | 0.113 | 4.57 | 0.002 | — | |
| TIDEBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=32026.01 | 0.156 | 0.122 | 5.53 | 0.003 | 3.17 | |
| RADBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=752026.01 | 0.159 | 0.122 | 6.25 | 0.066 | — | |
| RADModel=Gemma 2-2B, Temperature=0.1, Beta (β)=752026.01 | 0.167 | 0.129 | 5.01 | 0.085 | — | |
| RADModel=Gemma 2-2B, Temperature=0.1, Beta (β)=502026.01 | 0.206 | 0.161 | 4.31 | 0.121 | — | |
| RADBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=502026.01 | 0.209 | 0.157 | 5.5 | 0.12 | — | |
| SASAModel=Gemma 2-2B, Temperature=0.1, Beta (β)=5002026.01 | 0.283 | 0.214 | 11.77 | 0.252 | — | |
| GeDiBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=32026.01 | 0.297 | 0.245 | 7.46 | 0.25 | — | |
| Toxification ReversalBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=32026.01 | 0.334 | 0.271 | 6.63 | 0.321 | — | |
| RADModel=Gemma 2-2B, Temperature=0.1, Beta (β)=102026.01 | 0.38 | 0.303 | 3.53 | 0.353 | — | |
| SASAModel=Gemma 2-2B, Temperature=0.1, Beta (β)=3002026.01 | 0.397 | 0.311 | 6.17 | 0.415 | — | |
| RADBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=102026.01 | 0.419 | 0.336 | 4.39 | 0.401 | — | |
| SASAModel=Gemma 2-2B, Temperature=0.1, Beta (β)=1002026.01 | 0.522 | 0.428 | 3.47 | 0.561 | — | |
| DExpertsBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=32026.01 | 0.527 | 0.46 | 4.18 | 0.562 | — | |
| SASAModel=Gemma 2-2B, Temperature=0.1, Beta (β)=752026.01 | 0.533 | 0.438 | 3.41 | 0.575 | — | |
| SASAModel=Gemma 2-2B, Temperature=0.1, Beta (β)=502026.01 | 0.545 | 0.448 | 3.39 | 0.589 | — | |
| Base modelModel=Gemma 2-2B, Temperature=0.12026.01 | 0.547 | 0.458 | 3.36 | 0.588 | — | |
| SASAModel=Gemma 2-2B, Temperature=0.1, Beta (β)=102026.01 | 0.556 | 0.458 | 3.39 | 0.598 | — | |
| SASABackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=5002026.01 | 0.557 | 0.455 | 4.17 | 0.596 | — | |
| SASABackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=3002026.01 | 0.57 | 0.468 | 4.18 | 0.61 | — | |
| SASABackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=1002026.01 | 0.579 | 0.481 | 4.2 | 0.621 | — | |
| SASABackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=502026.01 | 0.58 | 0.482 | 4.18 | 0.624 | — | |
| SASABackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=752026.01 | 0.587 | 0.488 | 4.2 | 0.63 | — | |
| Base modelBackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=32026.01 | 0.591 | 0.495 | 4.13 | 0.639 | — | |
| SASABackbone=GPT-2 Large (774M), Temperature=0.1, Number of trials=3, beta (hyperparameter)=102026.01 | 0.592 | 0.492 | 4.2 | 0.638 | — |