# Classification on MNLI (ECE, BS) Source: [Classification on MNLI (ECE, BS)]() Collection index: [llms.txt]() Data updated: 2026-06-05T09:01:22.430Z Evaluation of classification performance on the Multi-Genre Natural Language Inference (MNLI) dataset, reporting Expected Calibration Error (ECE) and Batch Score (BS). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Classification | | Dataset (including any stated split) | MNLI | | Metric | ECE | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ECE](): Lower is better; unit: PERCENT. - [BS](): Higher is better; unit: SCALAR. ## Recorded results | Method | ECE | Result date | Report paper | | --- | --- | --- | --- | | Temperature Scaling | 1.85 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (As.) | 3.02 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Pr.) | 3.4 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Sa.) | 3.68 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Pr.) | 3.71 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Sa.) | 5.77 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (As.) | 6.45 | Not recorded | 6a2258a086173ad9c1468bcb | | Vanilla | 8.05 | Not recorded | 6a2258a086173ad9c1468bcb | | Consistency | 9.7 | Not recorded | 6a2258a086173ad9c1468bcb | | FSD | 15.16 | Not recorded | 6a2258a086173ad9c1468bcb | | Entropy | 28.09 | Not recorded | 6a2258a086173ad9c1468bcb | | Temperature Scaling | 38.47 | Not recorded | 6a2258a086173ad9c1468bcb | | Entropy | 39.75 | Not recorded | 6a2258a086173ad9c1468bcb | | FSD | 39.8 | Not recorded | 6a2258a086173ad9c1468bcb | | Consistency | 40.21 | Not recorded | 6a2258a086173ad9c1468bcb | | Vanilla | 40.22 | Not recorded | 6a2258a086173ad9c1468bcb | ### Temperature Scaling ECE: 1.85. Result ID: 6a228fdcfbc373592c20cd13. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### CaliDist (As.) ECE: 3.02. Result ID: 6a228fdcfbc373592c20cd18. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### CaliDist (Pr.) ECE: 3.4. Result ID: 6a228fdcfbc373592c20cd19. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### CaliDist (Sa.) ECE: 3.68. Result ID: 6a228fdcfbc373592c20cd1a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### CaliDist (Pr.) ECE: 3.71. Result ID: 6a228fdcfbc373592c20cd21. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### CaliDist (Sa.) ECE: 5.77. Result ID: 6a228fdcfbc373592c20cd22. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### CaliDist (As.) ECE: 6.45. Result ID: 6a228fdcfbc373592c20cd20. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### Vanilla ECE: 8.05. Result ID: 6a228fdcfbc373592c20cd14. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### Consistency ECE: 9.7. Result ID: 6a228fdcfbc373592c20cd15. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### FSD ECE: 15.16. Result ID: 6a228fdcfbc373592c20cd17. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### Entropy ECE: 28.09. Result ID: 6a228fdcfbc373592c20cd16. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### Temperature Scaling ECE: 38.47. Result ID: 6a228fdcfbc373592c20cd1b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### Entropy ECE: 39.75. Result ID: 6a228fdcfbc373592c20cd1e. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### FSD ECE: 39.8. Result ID: 6a228fdcfbc373592c20cd1f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### Consistency ECE: 40.21. Result ID: 6a228fdcfbc373592c20cd1d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### Vanilla ECE: 40.22. Result ID: 6a228fdcfbc373592c20cd1c. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.