Adversarial Attack on GLUE

3.56SST-2 SpeedupBaseline (PABEE-ALBERT-base)

Evaluation Results

Method
Baseline (PABEE-ALBERT-base)2023.05	3.56	0	1.91	3.45	3.07	0.25	2.41	1.2	3.4	0	1.85	5.05	2.28	1.4	2.29	1.6
A2T2023.05	3.51	0	1.67	5.18	3.13	0	2.33	2.9	3.29	0.2	1.71	11.91	2.06	5.9	2.1	5.5
HotFlip2023.05	3.3	0.11	1.62	7.86	2.57	0.49	2.27	3.1	3.02	0	1.74	9.75	1.99	4.5	2.02	5.1
PWWS2023.05	3.01	0.23	1.6	4.12	2.35	0.49	2.04	4.5	3.15	0	1.61	11.91	1.83	6.2	1.81	5.8
TextBugger2023.05	2.97	0.11	1.6	3.74	2.53	0.98	1.83	9.2	2.88	0.1	1.6	12.27	1.75	8.9	1.74	8.2
TextFooler2023.05	2.96	0.34	1.6	3.93	2.53	0.98	1.85	10.3	2.93	0.5	1.62	11.19	1.72	11	1.75	11
Baseline (PABEE-ALBERT-base)2023.05	2.67	0	1.63	9.11	2.03	1.47	1.97	3	2.58	0.5	1.57	11.55	1.86	4.7	2.29	1.6
A2T2023.05	2.56	1.03	1.43	15.72	1.8	9.31	1.88	7.7	2.44	1.1	1.45	22.74	1.67	14.9	2.1	5.5
HotFlip2023.05	2.46	1.38	1.4	17.26	1.78	7.6	1.83	9.2	2.25	0.3	1.46	20.94	1.62	13.3	2.02	5.1
PWWS2023.05	2.23	2.41	1.4	11.6	1.53	13.48	1.63	15	2.34	1.5	1.41	20.58	1.5	15.4	1.81	5.8
TextBugger2023.05	2.2	2.06	1.41	11.12	1.46	16.42	1.48	22.3	2.18	2.3	1.37	26.35	1.45	20.2	1.74	8.2
TextFooler2023.05	2.12	2.64	1.41	11.12	1.47	19.12	1.47	25.8	2.19	2.2	1.37	28.88	1.41	26	1.75	11
A2T2023.05	2	2.98	1.07	72.2	1.66	3.43	1.78	7.6	3.47	0.1	1.14	71.12	1.66	13.2	1.63	15
Baseline (PABEE-BERT-base)2023.05	1.98	2.41	1.19	35.57	2.01	2.45	1.84	4.7	3.45	0.1	1.42	28.16	1.76	7.5	1.75	6.1
HotFlip2023.05	1.8	5.5	1.05	80.44	1.79	1.96	1.68	11.4	3.29	0.1	1.36	40.43	1.6	13.9	1.61	15.3
SAME-Char2023.05	1.77	14.22	1.01	95.49	1.38	40.93	1.42	41.8	1.71	27.2	1.2	59.93	1.1	79.7	1.11	80.4
SAME-Word2023.05	1.68	17.2	1.06	84.95	1.32	41.67	1.41	40.9	1.65	27.1	1.34	33.21	1.11	77.3	1.08	82.2
Baseline (PABEE-BERT-base)2023.05	1.66	9.52	1.19	35.57	1.66	9.56	1.58	11	2.6	0.4	1.21	55.23	1.5	16.1	1.75	6.1
PWWS2023.05	1.66	10.44	1.04	83.7	1.5	4.41	1.53	17.8	3.07	0	1.34	39.35	1.49	18.3	1.5	16
TextFooler2023.05	1.63	11.12	1.04	86.39	1.48	6.62	1.45	21.1	2.91	0.3	1.45	27.8	1.44	23.1	1.42	26.2
A2T2023.05	1.62	12.39	1.07	72.2	1.37	21.32	1.53	15	2.54	0.9	1.29	40.43	1.41	29	1.63	15
TextBugger2023.05	1.62	13.53	1.04	86.39	1.46	10.29	1.45	25	2.78	1	1.43	31.05	1.45	22	1.44	23.5
HotFlip2023.05	1.49	22.13	1.05	80.44	1.47	17.4	1.44	20.9	2.45	1.1	1.28	37.91	1.36	31.3	1.61	15.3
PWWS2023.05	1.41	28.44	1.04	83.7	1.28	24.02	1.33	32.1	2.31	2.3	1.26	44.77	1.28	40.4	1.5	16
SAME-Char2023.05	1.37	42.66	1.01	97.99	1.11	75.25	1.3	49.8	1.44	47.9	1.09	76.9	1.06	86.1	1.11	80.4
TextBugger2023.05	1.37	32.11	1.04	86.39	1.25	30.64	1.25	44	2.21	2.8	1.21	53.79	1.27	42.6	1.44	23.5
TextFooler2023.05	1.37	32.11	1.05	82.07	1.26	28.19	1.26	41.7	2.16	3.7	1.27	44.4	1.22	52.5	1.42	26.2
SAME-Word2023.05	1.26	53.1	1.05	84.47	1.1	73.28	1.28	50.6	1.39	48.7	1.13	67.15	1.06	85.1	1.08	82.2
SAME-Char2023.05	1.21	61.24	1	99.9	1.13	67.16	1.45	25	1.95	11.2	1.11	71.84	1.02	93.5	1.06	85.9
SAME-Char2023.05	1.14	69.95	1	99.9	1.05	85.78	1.15	63.8	1.58	32.9	1.08	74.01	1.03	92.7	1.03	91.8
SAME-Word2023.05	1.08	82.11	1	100	1.1	69.85	1.53	17.8	1.62	24.9	1.14	71.12	1.03	93	1.03	92.7
SAME-Word2023.05	1.05	88.19	1	100	1.04	86.76	1.1	76	1.35	49.6	1.09	75.45	1.01	96.2	1.01	97.8