Question Answering on TruthfulQA (Factuality and Diversity Metrics)
29.62FactualityTypical + DFD (MPT-7B)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Typical + DFD (MPT-7B)Backbone=MPT-7B, Decoding Strategy=Typical, Integration of DFD=True2025.03 | 29.62 | 76.35 | 84.45 | 84.15 | 13.98 | |
| Top-k + DFD (MPT-7B)Backbone=MPT-7B, Decoding Strategy=Top-k, Integration of DFD=True2025.03 | 28.4 | 76.04 | 84.95 | 84.8 | 13.37 | |
| Nucleus + DFD (MPT-7B)Backbone=MPT-7B, Decoding Strategy=Nucleus, Integration of DFD=True2025.03 | 28.4 | 75.66 | 83.82 | 83.61 | 14.75 | |
| Top-k (MPT-7B)Backbone=MPT-7B, Decoding Strategy=Top-k, Integration of DFD=False2025.03 | 27.25 | 74.52 | 83.59 | 83.86 | 14.5 | |
| Nucleus (MPT-7B)Backbone=MPT-7B, Decoding Strategy=Nucleus, Integration of DFD=False2025.03 | 26.68 | 74.82 | 82.61 | 82.32 | 15.88 | |
| Temperature + DFD (MPT-7B)Backbone=MPT-7B, Decoding Strategy=Temperature, Integration of DFD=True2025.03 | 25.83 | 80.72 | 89.02 | 88.02 | 8.84 | |
| Typical (MPT-7B)Backbone=MPT-7B, Decoding Strategy=Typical, Integration of DFD=False2025.03 | 25.7 | 75.77 | 83.38 | 83.04 | 15.29 | |
| Temperature (MPT-7B)Backbone=MPT-7B, Decoding Strategy=Temperature, Integration of DFD=False2025.03 | 24.24 | 79.96 | 88.5 | 87.68 | 9.26 |