# Natural Language Inference on MSciNLI (ECE and BS) Source: [Natural Language Inference on MSciNLI (ECE and BS)]() Collection index: [llms.txt]() Data updated: 2026-06-05T09:01:23.155Z Evaluation of Natural Language Inference performance on the MSciNLI dataset, measured by Expected Calibration Error (ECE) and Brier Score (BS). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Natural Language Inference | | Dataset (including any stated split) | MSciNLI | | Metric | ECE | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ECE](): Lower is better; unit: PERCENT. - [BS](): Lower is better; unit: PERCENT. ## Recorded results | Method | ECE | Result date | Report paper | | --- | --- | --- | --- | | CaliDist (As.) | 2.08 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Sa.) | 2.88 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Pr.) | 4.98 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (As.) | 5.75 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Sa.) | 6.23 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Pr.) | 6.51 | Not recorded | 6a2258a086173ad9c1468bcb | | Temperature Scaling | 12.1 | Not recorded | 6a2258a086173ad9c1468bcb | | Vanilla | 25.76 | Not recorded | 6a2258a086173ad9c1468bcb | | Entropy | 28.05 | Not recorded | 6a2258a086173ad9c1468bcb | | FSD | 29.09 | Not recorded | 6a2258a086173ad9c1468bcb | | Consistency | 34.3 | Not recorded | 6a2258a086173ad9c1468bcb | | Temperature Scaling | 35.46 | Not recorded | 6a2258a086173ad9c1468bcb | | Entropy | 38.23 | Not recorded | 6a2258a086173ad9c1468bcb | | Vanilla | 38.6 | Not recorded | 6a2258a086173ad9c1468bcb | | FSD | 38.87 | Not recorded | 6a2258a086173ad9c1468bcb | | Consistency | 40.14 | Not recorded | 6a2258a086173ad9c1468bcb | ### CaliDist (As.) ECE: 2.08. Result ID: 6a228fdcfbc373592c20ce14. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"QWEN-3 8B"}. This record does not assert independent reproduction. ### CaliDist (Sa.) ECE: 2.88. Result ID: 6a228fdcfbc373592c20ce16. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"QWEN-3 8B"}. This record does not assert independent reproduction. ### CaliDist (Pr.) ECE: 4.98. Result ID: 6a228fdcfbc373592c20ce0d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"LLAMA-3.1 8B"}. This record does not assert independent reproduction. ### CaliDist (As.) ECE: 5.75. Result ID: 6a228fdcfbc373592c20ce0c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"LLAMA-3.1 8B"}. This record does not assert independent reproduction. ### CaliDist (Sa.) ECE: 6.23. Result ID: 6a228fdcfbc373592c20ce0e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"LLAMA-3.1 8B"}. This record does not assert independent reproduction. ### CaliDist (Pr.) ECE: 6.51. Result ID: 6a228fdcfbc373592c20ce15. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"QWEN-3 8B"}. This record does not assert independent reproduction. ### Temperature Scaling ECE: 12.1. Result ID: 6a228fdcfbc373592c20ce07. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"LLAMA-3.1 8B"}. This record does not assert independent reproduction. ### Vanilla ECE: 25.76. Result ID: 6a228fdcfbc373592c20ce08. [Reporting paper]() Recorded evaluation configuration: {"LLM":"LLAMA-3.1 8B"}. This record does not assert independent reproduction. ### Entropy ECE: 28.05. Result ID: 6a228fdcfbc373592c20ce0a. [Reporting paper]() Recorded evaluation configuration: {"LLM":"LLAMA-3.1 8B"}. This record does not assert independent reproduction. ### FSD ECE: 29.09. Result ID: 6a228fdcfbc373592c20ce0b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"LLAMA-3.1 8B"}. This record does not assert independent reproduction. ### Consistency ECE: 34.3. Result ID: 6a228fdcfbc373592c20ce09. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"LLAMA-3.1 8B"}. This record does not assert independent reproduction. ### Temperature Scaling ECE: 35.46. Result ID: 6a228fdcfbc373592c20ce0f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"QWEN-3 8B"}. This record does not assert independent reproduction. ### Entropy ECE: 38.23. Result ID: 6a228fdcfbc373592c20ce12. [Reporting paper]() Recorded evaluation configuration: {"LLM":"QWEN-3 8B"}. This record does not assert independent reproduction. ### Vanilla ECE: 38.6. Result ID: 6a228fdcfbc373592c20ce10. [Reporting paper]() Recorded evaluation configuration: {"LLM":"QWEN-3 8B"}. This record does not assert independent reproduction. ### FSD ECE: 38.87. Result ID: 6a228fdcfbc373592c20ce13. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"QWEN-3 8B"}. This record does not assert independent reproduction. ### Consistency ECE: 40.14. Result ID: 6a228fdcfbc373592c20ce11. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"QWEN-3 8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.