List your company

Faithfulness Measurement on Alpaca

0.601BLEURandom
0.426280.471640.5170.56236May 30, 2024

Evaluation results

MethodLinks
RandomModel=LlaMA-2 (7B-Chat)2024.050.6010.5270.5330.5220.8250.8420.134
RandomModel=Vicuna (7B)2024.050.5870.5410.5520.5350.7860.8910.079
AttentionModel=LlaMA-2 (7B-Chat)2024.050.5470.4470.460.4480.7210.7910.17
Integrated-GradientModel=LlaMA-2 (7B-Chat)2024.050.5410.4240.4350.4240.7250.7260.242
Integrated-GradientModel=Vicuna (7B)2024.050.5410.4950.5030.4880.7440.8350.135
Last-AttentionModel=LlaMA-2 (7B-Chat)2024.050.5330.4230.4520.4320.6720.770.202
CaptumModel=LlaMA-2 (7B-Chat)2024.050.5150.4090.4210.4090.680.6020.417
AttentionModel=Vicuna (7B)2024.050.4860.4470.4690.4410.6830.8310.14
JoPAModel=LlaMA-2 (7B-Chat)2024.050.4840.3880.3860.3790.6420.5490.504
Last-AttentionModel=Vicuna (7B)2024.050.4750.4230.4360.4150.6720.840.113
CaptumModel=Vicuna (7B)2024.050.4660.4350.4270.4180.6490.6540.347
JoPAModel=Vicuna (7B)2024.050.4330.3950.390.3770.6390.5890.459