Explanation Generation on Amazon (test)
77FMR (Faithfulness Rate)PETER+
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PETER+Feature Usage=Yes2021.05 | 77 | 31 | 1.2 | 46 | 19.75 | 3.06 | 34.71 | 23.99 | 26.35 | 9.04 | 6.23 | 6.71 | |
| NETEFeature Usage=Yes2021.05 | 71 | 19 | 1.93 | 57 | 18.76 | 2.46 | 33.87 | 21.43 | 24.81 | 7.58 | 4.77 | 5.46 | |
| NRTFeature Usage=No2021.05 | 12 | 7 | 2.93 | 17 | 12.93 | 0.96 | 21.03 | 13.57 | 15.56 | 2.71 | 1.84 | 2.05 | |
| Att2SeqFeature Usage=No2021.05 | 12 | 20 | 2.74 | 33 | 12.56 | 0.95 | 20.79 | 13.31 | 15.35 | 2.62 | 1.78 | 1.99 | |
| PETERFeature Usage=No2021.05 | 12 | 21 | 1.75 | 29 | 12.77 | 1.17 | 19.81 | 13.8 | 15.23 | 2.8 | 2.08 | 2.2 | |
| TransformerFeature Usage=No2021.05 | 10 | 1 | 3.26 | 0 | 9.71 | 0.59 | 19.68 | 11.94 | 14.11 | 2.1 | 1.39 | 1.55 | |
| ACMLMFeature Usage=Yes2021.05 | 10 | 31 | 2.07 | 96 | 9.52 | 0.22 | 11.65 | 10.39 | 9.69 | 0.71 | 0.81 | 0.64 |