Faithfulness Measurement on MHC benchmark leaderboard · SOTA2 Research

Faithfulness Measurement on MHC

72.3BLEURandom

Evaluation Results

Method	Links
Random2024.05		72.3	61.7	61.4	61.5	78.7	95.8	0.023
Random2024.05		71.5	62.5	62.3	62.3	81	97.2	0.012
Random2024.05		70.6	59.5	60.3	59.6	78.8	95.3	0.027
Captum2024.05		68.8	60.3	60.6	60.4	80.6	80.6	0.145
Attention2024.05		68.5	58.1	58.1	58	77.5	95	0.026
Last-Attention2024.05		68.4	57	57.3	57	77.3	94.7	0.028
Integrated-Gradient2024.05		67.2	55.9	55.5	55.6	76.2	94.1	0.03
Attention2024.05		66.5	53.6	53.8	53.1	74.5	91.6	0.056
Attention2024.05		66.4	53.1	53	52.9	73.9	91.6	0.053
Last-Attention2024.05		66.2	52.6	52.6	52.5	72.4	90.5	0.063
Last-Attention2024.05		66	52.9	52.5	52.6	73.6	90.7	0.023
Integrated-Gradient2024.05		64.6	50	49.6	49.8	68.7	83.6	0.117
Captum2024.05		64	49.7	49.3	49.4	66.3	76	0.189
Integrated-Gradient2024.05		61	47	47.9	47.9	68.8	68.8	0.358
Captum2024.05		57.9	43.8	43.2	43.3	62.7	81.1	0.12
JoPA2024.05		57.9	40.8	40.7	40.7	64.3	59.4	0.504
JoPA2024.05		57.5	40.3	40.5	40.3	60.2	70.1	0.246
JoPA2024.05		57.5	43.1	42.5	42.7	62	78.3	0.141