Jailbreak Defense on GPTFuzz, TAP, GCG, AutoDAN, and Template benchmark leaderboard · SOTA2 Research

Jailbreak Defense on GPTFuzz, TAP, GCG, AutoDAN, and Template

24.98GPTFuzz ASRSFT

Evaluation Results

Method	Links
SFT2024.12		24.98	40	17.39	25.79	29.04	27.44	55.92
DARE2024.12		26.16	42.38	19.02	25.5	28.91	28.4	54.56
SFT2024.12		32.11	44.76	23.91	36.96	38.52	35.25	47.71
DARE2024.12		32.51	41.43	23.91	36.1	38.58	34.51	48.05
SafeDecoding2024.12		37.66	65.71	36.68	28.37	47.83	43.25	39.71
RESTA2024.12		54.51	76.19	89.13	86.82	78.24	76.98	5.98
RESTAd2024.12		55.1	75.71	88.86	84.24	77.49	76.28	6.68
IRR2024.12		59.56	82.86	85.05	91.12	79.68	79.65	3.31
IRRd2024.12		60.36	84.29	86.96	93.12	82.1	82.1	0.86
SFT2024.12		66.2	46.19	34.78	64.47	69.42	56.21	26.75
DARE2024.12		66.2	48.57	34.24	65.33	69.88	56.84	26.12
RESTA2024.12		66.3	81.9	68.21	93.12	83.47	78.6	4.36
RESTAd2024.12		66.8	82.86	69.57	91.69	83.63	78.91	4.05
IRR2024.12		71.46	85.24	67.93	91.69	89.28	81.12	1.84
Safe LoRA2024.12		74.73	96.19	89.13	65.04	89.58	82.93	0.03
Safe LoRA2024.12		74.73	96.19	89.13	65.04	89.58	82.93	0.03
IRRd2024.12		75.12	87.62	71.2	93.12	90.59	83.53	-0.57
Safe LORA2024.12		77.6	96.67	60.87	80.8	80.82	79.35	3.61
SafeDecoding2024.12		78.39	85.24	92.12	52.44	52.44	76.35	6.05
SafeDecoding2024.12		81.27	66.19	62.77	76.22	77.79	72.85	10.11
RESTAd2024.12		89.89	77.62	79.35	99.14	95.95	88.39	-5.43
RESTA2024.12		91.28	78.57	79.89	98.85	96.01	88.92	-5.96
IRR2024.12		91.58	78.1	62.23	99.43	95.23	85.31	-2.35
IRRd2024.12		92.17	81.9	62.23	100	95.95	86.45	-3.49