Natural Language Inference Distribution Estimation on ChaosNLI 1.0 (dev)
0.073KL Divergence (BERT FT)ChaosNLI HJD
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ChaosNLI HJDExplanation Source=Human2024.12 | 0.073 | 0.967 | 0.645 | 0.062 | 0.933 | 0.696 | — | — | — | |
| Mixtral MJD + MNLI Label-Guided model explanationsBackbone=Mixtral, Explanation Source=Model, Selection Strategy=MNLI Label-Guided2024.12 | 0.097 | 1.041 | 0.53 | 0.086 | 1.006 | 0.575 | — | — | — | |
| MNLI dist.Explanation Source=None2024.12 | 0.104 | 1.062 | 0.569 | 0.101 | 1.052 | 0.625 | — | — | — | |
| Mixtral MJD + VariErr Label-Guided model explanationsBackbone=Mixtral, Explanation Source=Model, Selection Strategy=VariErr Label-Guided2024.12 | 0.108 | 1.073 | 0.519 | 0.091 | 1.021 | 0.569 | — | — | — | |
| Mixtral MJD + human explanationsBackbone=Mixtral, Explanation Source=Human2024.12 | 0.121 | 1.112 | 0.525 | 0.086 | 1.007 | 0.575 | — | — | — | |
| Mixtral MJDBackbone=Mixtral, Explanation Source=None2024.12 | 0.131 | 1.14 | 0.427 | 0.121 | 1.112 | 0.497 | — | — | — | |
| VariErr dist.Explanation Source=None2024.12 | 0.177 | 1.279 | 0.552 | 0.166 | 1.246 | 0.616 | — | — | — | |
| Mixtral MJD + Label-Free model explanationsBackbone=Mixtral, Explanation Source=Model, Selection Strategy=Label-Free2024.12 | 0.233 | 1.447 | 0.298 | 0.241 | 1.472 | 0.274 | — | — | — | |
| ChaosNLI HJDLLM=Baseline, Fine-tuning Evaluator Model=BERT2024.12 | — | — | — | — | — | — | 0.073 | 0.967 | 0.645 | |
| ChaosNLI HJDLLM=Baseline, Fine-tuning Evaluator Model=RoBERTa2024.12 | — | — | — | — | — | — | 0.062 | 0.933 | 0.696 | |
| GPT-4o VariErr Label-GuidedLLM=GPT-4o, Explanation Type=VariErr Label-Guided, Fine-tuning Evaluator Model=BERT2024.12 | — | — | — | — | — | — | 0.092 | 1.026 | 0.554 | |
| GPT-4o VariErr Label-GuidedLLM=GPT-4o, Explanation Type=VariErr Label-Guided, Fine-tuning Evaluator Model=RoBERTa2024.12 | — | — | — | — | — | — | 0.088 | 1.013 | 0.618 | |
| Llama3 Label-FreeLLM=Llama3, Explanation Type=Label-Free, Fine-tuning Evaluator Model=BERT2024.12 | — | — | — | — | — | — | 0.106 | 1.066 | 0.539 | |
| Llama3 Label-FreeLLM=Llama3, Explanation Type=Label-Free, Fine-tuning Evaluator Model=RoBERTa2024.12 | — | — | — | — | — | — | 0.103 | 1.059 | 0.581 |