# Classification on AQUA Source: [Classification on AQUA]() Collection index: [llms.txt]() Data updated: 2026-06-05T09:01:22.594Z Evaluation of Classification performance on the AQUA dataset, reporting Expected Calibration Error (ECE) and Brier Score (BS). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Classification | | Dataset (including any stated split) | AQUA | | Metric | ECE | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ECE](): Lower is better; unit: PERCENT. - [Brier Score (BS)](): Lower is better; unit: SCALAR. ## Recorded results | Method | ECE | Result date | Report paper | | --- | --- | --- | --- | | CaliDist (As.) | 2.96 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Sa.) | 4.49 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Pr.) | 5.91 | Not recorded | 6a2258a086173ad9c1468bcb | | Consistency | 5.93 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (As.) | 5.97 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Sa.) | 6.15 | Not recorded | 6a2258a086173ad9c1468bcb | | CaliDist (Pr.) | 9.64 | Not recorded | 6a2258a086173ad9c1468bcb | | Temperature Scaling | 10.07 | Not recorded | 6a2258a086173ad9c1468bcb | | Temperature Scaling | 12.25 | Not recorded | 6a2258a086173ad9c1468bcb | | Consistency | 13.8 | Not recorded | 6a2258a086173ad9c1468bcb | | FSD | 14.19 | Not recorded | 6a2258a086173ad9c1468bcb | | FSD | 15.85 | Not recorded | 6a2258a086173ad9c1468bcb | | Vanilla | 18.93 | Not recorded | 6a2258a086173ad9c1468bcb | | Vanilla | 19.69 | Not recorded | 6a2258a086173ad9c1468bcb | | Entropy | 23.46 | Not recorded | 6a2258a086173ad9c1468bcb | | Entropy | 26.33 | Not recorded | 6a2258a086173ad9c1468bcb | ### CaliDist (As.) ECE: 2.96. Result ID: 6a228fdcfbc373592c20cd70. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### CaliDist (Sa.) ECE: 4.49. Result ID: 6a228fdcfbc373592c20cd72. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### CaliDist (Pr.) ECE: 5.91. Result ID: 6a228fdcfbc373592c20cd71. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### Consistency ECE: 5.93. Result ID: 6a228fdcfbc373592c20cd65. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### CaliDist (As.) ECE: 5.97. Result ID: 6a228fdcfbc373592c20cd68. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### CaliDist (Sa.) ECE: 6.15. Result ID: 6a228fdcfbc373592c20cd6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### CaliDist (Pr.) ECE: 9.64. Result ID: 6a228fdcfbc373592c20cd69. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### Temperature Scaling ECE: 10.07. Result ID: 6a228fdcfbc373592c20cd6b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### Temperature Scaling ECE: 12.25. Result ID: 6a228fdcfbc373592c20cd63. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### Consistency ECE: 13.8. Result ID: 6a228fdcfbc373592c20cd6d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### FSD ECE: 14.19. Result ID: 6a228fdcfbc373592c20cd67. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### FSD ECE: 15.85. Result ID: 6a228fdcfbc373592c20cd6f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### Vanilla ECE: 18.93. Result ID: 6a228fdcfbc373592c20cd64. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. ### Vanilla ECE: 19.69. Result ID: 6a228fdcfbc373592c20cd6c. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### Entropy ECE: 23.46. Result ID: 6a228fdcfbc373592c20cd6e. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Gemma-3 (4B)"}. This record does not assert independent reproduction. ### Entropy ECE: 26.33. Result ID: 6a228fdcfbc373592c20cd66. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Phi-4-mini (3.8B)"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.