Token-forcing loss optimization on Random targets Held-out (val)
0.27Qwen-2.5-7B Lossclaude_v82
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| claude_v82FLOPs budget=10^172026.03 | 0.27 | 4.49 | 2.33 | 1.41 | 0.77 | 1.85 | |
| claude_v79FLOPs budget=10^172026.03 | 0.38 | 6.08 | 2.88 | 1.62 | 0.84 | 2.36 | |
| claude_v63FLOPs budget=10^172026.03 | 0.41 | 4.7 | 3.76 | 0.92 | 0.9 | 2.14 | |
| claude_v60FLOPs budget=10^172026.03 | 0.52 | 4.32 | 4.3 | 0.84 | 0.85 | 2.17 | |
| claude_v13FLOPs budget=10^172026.03 | 0.54 | 4.25 | 3.05 | 1.41 | 0.59 | 1.97 | |
| claude_v56FLOPs budget=10^172026.03 | 0.62 | 4.41 | 3.71 | 0.74 | 0.79 | 2.05 | |
| claude_v15FLOPs budget=10^172026.03 | 0.63 | 4.39 | 3.11 | 0.81 | 0.49 | 1.88 | |
| claude_v50FLOPs budget=10^172026.03 | 0.67 | 4.61 | 4.12 | 0.7 | 0.87 | 2.19 | |
| claude_v53FLOPs budget=10^172026.03 | 0.72 | 4.17 | 3.3 | 0.67 | 0.4 | 1.85 | |
| claude_v6FLOPs budget=10^172026.03 | 1.23 | 10.56 | 5.8 | 2.16 | 1.58 | 4.27 | |
| I-GCGHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 2.24 | 3.16 | 3.27 | 1.86 | 2.01 | 2.51 | |
| I-GCG-LSGMHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 2.47 | 3.35 | 4.38 | 2.05 | 2.61 | 2.97 | |
| I-GCGFLOPs budget=10^172026.03 | 4.04 | 3.69 | 4.89 | 2.05 | 2.44 | 3.43 | |
| I-GCG-LSGMFLOPs budget=10^172026.03 | 4.05 | 3.41 | 4.38 | 2.15 | 2.15 | 3.23 | |
| claude_v3FLOPs budget=10^172026.03 | 4.21 | 3.97 | 4.62 | 2.31 | 2.67 | 3.55 | |
| MACHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 4.36 | 3.66 | 2.44 | 0.87 | 2.36 | 2.74 | |
| TAOFLOPs budget=10^172026.03 | 5.16 | 3.84 | 2.93 | 1.51 | 2.88 | 3.26 | |
| ADCHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 5.24 | 10.89 | 7.02 | 1.27 | 5.74 | 6.03 | |
| Mask-GCGHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 5.31 | 4.06 | 11.55 | 1.58 | 3.37 | 5.18 | |
| GCGHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 5.32 | 3.05 | 3.55 | 1.68 | 2.72 | 3.26 | |
| MC-GCGHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 5.34 | 3.34 | 3.12 | 1.35 | 2.84 | 3.2 | |
| SM-GCGHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 5.4 | 4.09 | 7.31 | 1.82 | 2.6 | 4.24 | |
| TAOHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 5.55 | 3.57 | 3.36 | 1.66 | 3.15 | 3.46 | |
| AttnGCGHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 5.91 | 3.74 | 3.66 | 1.63 | 3.45 | 3.68 | |
| AttnGCGFLOPs budget=10^172026.03 | 6.11 | 3.96 | 3.59 | 1.76 | 3.37 | 3.76 | |
| MACFLOPs budget=10^172026.03 | 6.18 | 3.42 | 4.54 | 1.94 | 3.17 | 3.85 | |
| Mask-GCGFLOPs budget=10^172026.03 | 6.4 | 3.79 | 12.34 | 2 | 3.56 | 5.62 | |
| MC-GCGFLOPs budget=10^172026.03 | 6.58 | 3.56 | 4.23 | 1.87 | 3.17 | 3.88 | |
| SM-GCGFLOPs budget=10^172026.03 | 6.65 | 4.14 | 12.62 | 2.3 | 2.83 | 5.71 | |
| Probe SamplingFLOPs budget=10^172026.03 | 6.68 | 4.12 | 4.82 | 1.92 | 2.96 | 4.1 | |
| PGDFLOPs budget=10^172026.03 | 7.12 | 3.54 | 6.04 | 1.88 | 3.64 | 4.44 | |
| GCGFLOPs budget=10^172026.03 | 7.62 | 4.15 | 5.04 | 1.78 | 3.55 | 4.43 | |
| I-GCG-LILAFLOPs budget=10^172026.03 | 8.05 | 3.95 | 6.6 | 2.07 | 3.95 | 4.92 | |
| MAGICFLOPs budget=10^172026.03 | 8.12 | 5.39 | 4.86 | 2.17 | 5.35 | 5.18 | |
| MAGICHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 8.14 | 4.75 | 2.89 | 1.53 | 5.52 | 4.56 | |
| DeGCGFLOPs budget=10^172026.03 | 8.43 | 6.41 | 4.74 | 2.27 | 4.82 | 5.33 | |
| ADCFLOPs budget=10^172026.03 | 8.62 | 6.63 | 4.25 | 0.27 | 2.57 | 4.47 | |
| DeGCGHyperparameter tuning=Optuna (100 trials), FLOPs budget=10^172026.03 | 9.12 | 5.94 | 4.62 | 1.32 | 5.83 | 5.37 | |
| ACGFLOPs budget=10^172026.03 | 9.73 | 6.3 | 6.69 | 3.84 | 5.58 | 6.43 | |
| GCG++FLOPs budget=10^172026.03 | 10.12 | 6.04 | 7.78 | 2.54 | 7.65 | 6.83 | |
| Faster-GCGFLOPs budget=10^172026.03 | 10.65 | 12.38 | 15.1 | 3.93 | 12.09 | 10.83 | |
| LLSFLOPs budget=10^172026.03 | 10.76 | 8.66 | 14.76 | 9.97 | 8.43 | 10.51 | |
| GBDAFLOPs budget=10^172026.03 | 11.17 | 12.15 | 13.36 | 7.08 | 11.31 | 11.01 | |
| TGCGFLOPs budget=10^172026.03 | 11.63 | 9.69 | 13.56 | 6.02 | 8.87 | 9.95 | |
| AutoPromptFLOPs budget=10^172026.03 | 11.67 | 7.55 | 13.87 | 6.47 | 6.66 | 9.24 | |
| Slot-GCGFLOPs budget=10^172026.03 | 11.8 | 9.8 | 14.25 | 11.39 | 8.96 | 11.24 | |
| PEZFLOPs budget=10^172026.03 | 11.98 | 10.86 | 17.12 | 3.66 | 12.45 | 11.21 | |
| UATFLOPs budget=10^172026.03 | 12.01 | 8.12 | 8.99 | 4.73 | 7.29 | 8.23 | |
| PRSFLOPs budget=10^172026.03 | 12.03 | 9.82 | 17.45 | 12.74 | 9.62 | 12.33 | |
| ARCAFLOPs budget=10^172026.03 | 12.26 | 9.51 | 3.75 | 1.28 | 8.72 | 7.11 | |
| RAILSFLOPs budget=10^172026.03 | 12.71 | 11.11 | 17.37 | 13.34 | 10.52 | 13.01 | |
| BEASTFLOPs budget=10^172026.03 | 12.74 | 11.34 | 17.98 | 14.91 | 10.94 | 13.58 | |
| BONFLOPs budget=10^172026.03 | 15.39 | 13.59 | 20.62 | 16.63 | 12.42 | 15.73 | |
| REINFORCE-GCGFLOPs budget=10^172026.03 | 16.12 | 13.75 | 21.18 | 16.15 | 12.67 | 15.97 | |
| Reg-RelaxFLOPs budget=10^172026.03 | 17.75 | 13.53 | 20.8 | 16.03 | 14.27 | 16.48 | |
| COLD-AttackFLOPs budget=10^172026.03 | 18.11 | 14.61 | 24.88 | 18.14 | 13.08 | 17.77 |