Natural Language Inference Distribution Estimation on ChaosNLI 1.0 (test)
0KL Div (Dist)ChaosNLI HJD
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| ChaosNLI HJDExplanation Source=Human2024.12 | 0 | 0 | 0 | 0.077 | 0.974 | 60.9 | 0.06 | 0.922 | 65.3 | 1 | |
| Mixtral MJD + MNLI Label-Guided model explanationsBackbone=Mixtral, Explanation Source=Model, Selection Strategy=MNLI Label-Guided2024.12 | 0.237 | 0.223 | 0.253 | 0.095 | 1.028 | 53.3 | 0.085 | 0.997 | 55.9 | 0.726 | |
| Mixtral MJD + VariErr Label-Guided model explanationsBackbone=Mixtral, Explanation Source=Model, Selection Strategy=VariErr Label-Guided2024.12 | 0.238 | 0.224 | 0.255 | 0.097 | 1.032 | 51.3 | 0.089 | 1.01 | 55.7 | 0.642 | |
| Mixtral MJD + human explanationsBackbone=Mixtral, Explanation Source=Human2024.12 | 0.239 | 0.225 | 0.257 | 0.109 | 1.075 | 51.9 | 0.085 | 0.998 | 55.7 | 0.656 | |
| Mixtral MJD + Label-Free model explanationsBackbone=Mixtral, Explanation Source=Model, Selection Strategy=Label-Free2024.12 | 0.361 | 0.299 | 0.343 | 0.222 | 1.407 | 29.6 | 0.237 | 1.452 | 30.2 | 0.483 | |
| Mixtral MJDBackbone=Mixtral, Explanation Source=None2024.12 | 0.437 | 0.293 | 0.324 | 0.129 | 1.13 | 43.2 | 0.125 | 1.118 | 47.2 | 0.593 | |
| MNLI dist.Explanation Source=None2024.12 | 1.242 | 0.281 | 0.295 | 0.1 | 1.042 | 55.5 | 0.093 | 1.02 | 60.7 | 0.795 | |
| VariErr dist.Explanation Source=None2024.12 | 3.604 | 0.282 | 0.296 | 0.179 | 1.279 | 52.2 | 0.173 | 1.261 | 59.4 | 0.688 |