Model Calibration on Multi-domain aggregate
23.2Agreementagreement-based consistency
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| agreement-based consistencyBackbone=LLaMA-7B, Sampling Strategy=n=40, T=0.42024.02 | 23.2 | — | — | — | |
| agreement-based consistencyBackbone=Mistral-7B-instruct, Sampling Strategy=n=40, T=0.42024.02 | 21.6 | — | — | — | |
| agreement-based consistencyBackbone=LLaMA-13B, Sampling Strategy=n=40, T=0.42024.02 | 20.4 | — | — | — | |
| agreement-based consistencyBackbone=Mistral-7B, Sampling Strategy=n=40, T=0.42024.02 | 18.3 | — | — | — | |
| agreement-based consistencyBackbone=LLaMA-70B, Sampling Strategy=n=40, T=0.42024.02 | 15.4 | — | — | — | |
| entropy-based consistencyBackbone=LLaMA-7B, Sampling Strategy=n=40, T=0.42024.02 | — | 0.241 | — | — | |
| entropy-based consistencyBackbone=LLaMA-13B, Sampling Strategy=n=40, T=0.42024.02 | — | 0.222 | — | — | |
| entropy-based consistencyBackbone=LLaMA-70B, Sampling Strategy=n=40, T=0.42024.02 | — | 0.182 | — | — | |
| entropy-based consistencyBackbone=Mistral-7B, Sampling Strategy=n=40, T=0.42024.02 | — | 0.205 | — | — | |
| entropy-based consistencyBackbone=Mistral-7B-instruct, Sampling Strategy=n=40, T=0.42024.02 | — | 0.22 | — | — | |
| first-second-distance-based (FSD)Backbone=LLaMA-7B, Sampling Strategy=n=40, T=0.42024.02 | — | — | 0.235 | — | |
| first-second-distance-based (FSD)Backbone=LLaMA-13B, Sampling Strategy=n=40, T=0.42024.02 | — | — | 0.211 | — | |
| first-second-distance-based (FSD)Backbone=LLaMA-70B, Sampling Strategy=n=40, T=0.42024.02 | — | — | 0.165 | — | |
| first-second-distance-based (FSD)Backbone=Mistral-7B, Sampling Strategy=n=40, T=0.42024.02 | — | — | 0.191 | — | |
| first-second-distance-based (FSD)Backbone=Mistral-7B-instruct, Sampling Strategy=n=40, T=0.42024.02 | — | — | 0.215 | — | |
| logitBackbone=LLaMA-7B2024.02 | — | — | — | 0.474 | |
| logitBackbone=LLaMA-13B2024.02 | — | — | — | 0.389 | |
| logitBackbone=LLaMA-70B2024.02 | — | — | — | 0.252 | |
| logitBackbone=Mistral-7B2024.02 | — | — | — | 0.324 | |
| logitBackbone=Mistral-7B-instruct2024.02 | — | — | — | 0.384 |