List your company

Faithfulness Measurement on MHC

72.3BLEURandom
56.90860.90464.968.896May 30, 2024

Evaluation results

MethodLinks
RandomModel=LlaMA-2 (7B-Chat)2024.0572.361.761.461.578.795.80.023
RandomModel=Vicuna (7B)2024.0571.562.562.362.38197.20.012
RandomModel=Llama-2 (7B-Chat)2024.0570.659.560.359.678.895.30.027
CaptumModel=Llama-2 (7B-Chat)2024.0568.860.360.660.480.680.60.145
AttentionModel=Vicuna (7B)2024.0568.558.158.15877.5950.026
Last-AttentionModel=Vicuna (7B)2024.0568.45757.35777.394.70.028
Integrated-GradientModel=Vicuna (7B)2024.0567.255.955.555.676.294.10.03
AttentionModel=LlaMA-2 (7B-Chat)2024.0566.553.653.853.174.591.60.056
AttentionModel=Llama-2 (7B-Chat)2024.0566.453.15352.973.991.60.053
Last-AttentionModel=Llama-2 (7B-Chat)2024.0566.252.652.652.572.490.50.063
Last-AttentionModel=LlaMA-2 (7B-Chat)2024.056652.952.552.673.690.70.023
Integrated-GradientModel=LlaMA-2 (7B-Chat)2024.0564.65049.649.868.783.60.117
CaptumModel=LlaMA-2 (7B-Chat)2024.056449.749.349.466.3760.189
Integrated-GradientModel=Llama-2 (7B-Chat)2024.05614747.947.968.868.80.358
CaptumModel=Vicuna (7B)2024.0557.943.843.243.362.781.10.12
JoPAModel=Llama-2 (7B-Chat)2024.0557.940.840.740.764.359.40.504
JoPAModel=LlaMA-2 (7B-Chat)2024.0557.540.340.540.360.270.10.246
JoPAModel=Vicuna (7B)2024.0557.543.142.542.76278.30.141