# Medical Image Classification on DermaMNIST (m=5 synthetic annotators, K=7, test) Source: [Medical Image Classification on DermaMNIST (m=5 synthetic annotators, K=7, test)]() Collection index: [llms.txt]() Data updated: 2026-03-25T13:47:34.980Z Evaluation of Medical Image Classification performance on a specific configuration of the DermaMNIST dataset derived from 5 synthetic annotators (K=7 agreement structure), reporting calibration and uncertainty metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Medical Image Classification | | Dataset (including any stated split) | DermaMNIST m=5 synthetic annotators, K=7, mean agreement 64.7% (test) | | Metric | ECE | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 26 of 26 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ECE](): Lower is better; unit: PERCENT. - [T Metric](): Higher is better; unit: SCALAR. - [aECE](): Lower is better; unit: PERCENT. - [cwECE](): Lower is better; unit: PERCENT. - [Reliability Score (Br)](): Higher is better; unit: SCALAR. - [NLL](): Lower is better; unit: SCALAR. ## Recorded results | Method | ECE | Result date | Report paper | | --- | --- | --- | --- | | IR-Soft | 2.06 | Not recorded | 69c36c3d0636e5bf4afc664b | | IR-Soft | 2.2 | Not recorded | 69c36c3d0636e5bf4afc664b | | Dirichlet-Soft | 3.18 | Not recorded | 69c36c3d0636e5bf4afc664b | | MCTS S=1 | 3.28 | Not recorded | 69c36c3d0636e5bf4afc664b | | SLTS | 3.29 | Not recorded | 69c36c3d0636e5bf4afc664b | | SoftPlatt | 3.41 | Not recorded | 69c36c3d0636e5bf4afc664b | | Oracle TS | 3.51 | Not recorded | 69c36c3d0636e5bf4afc664b | | VS | 3.81 | Not recorded | 69c36c3d0636e5bf4afc664b | | Dirichlet-Soft | 3.94 | Not recorded | 69c36c3d0636e5bf4afc664b | | SoftPlatt | 4.05 | Not recorded | 69c36c3d0636e5bf4afc664b | | VS | 4.58 | Not recorded | 69c36c3d0636e5bf4afc664b | | Oracle TS | 4.87 | Not recorded | 69c36c3d0636e5bf4afc664b | | LS-TS | 5.05 | Not recorded | 69c36c3d0636e5bf4afc664b | | MCTS S=1 | 5.07 | Not recorded | 69c36c3d0636e5bf4afc664b | | SLTS | 5.13 | Not recorded | 69c36c3d0636e5bf4afc664b | | LS-TS | 7.36 | Not recorded | 69c36c3d0636e5bf4afc664b | | TS | 22.25 | Not recorded | 69c36c3d0636e5bf4afc664b | | ATS | 22.82 | Not recorded | 69c36c3d0636e5bf4afc664b | | Dirichlet-Hard | 23.02 | Not recorded | 69c36c3d0636e5bf4afc664b | | Platt | 23.1 | Not recorded | 69c36c3d0636e5bf4afc664b | | Platt | 24.18 | Not recorded | 69c36c3d0636e5bf4afc664b | | Dirichlet-Hard | 24.55 | Not recorded | 69c36c3d0636e5bf4afc664b | | TS | 24.65 | Not recorded | 69c36c3d0636e5bf4afc664b | | ATS | 25.35 | Not recorded | 69c36c3d0636e5bf4afc664b | | Uncalibrated | 34.56 | Not recorded | 69c36c3d0636e5bf4afc664b | | Uncalibrated | 37.55 | Not recorded | 69c36c3d0636e5bf4afc664b | ### IR-Soft ECE: 2.06. Result ID: 69c3e77347f34ce924ad8839. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### IR-Soft ECE: 2.2. Result ID: 69c3e77347f34ce924ad882c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### Dirichlet-Soft ECE: 3.18. Result ID: 69c3e77347f34ce924ad883a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### MCTS S=1 ECE: 3.28. Result ID: 69c3e77347f34ce924ad8836. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### SLTS ECE: 3.29. Result ID: 69c3e77347f34ce924ad8835. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### SoftPlatt ECE: 3.41. Result ID: 69c3e77347f34ce924ad8837. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### Oracle TS ECE: 3.51. Result ID: 69c3e77347f34ce924ad883c. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### VS ECE: 3.81. Result ID: 69c3e77347f34ce924ad8838. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### Dirichlet-Soft ECE: 3.94. Result ID: 69c3e77347f34ce924ad882d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### SoftPlatt ECE: 4.05. Result ID: 69c3e77347f34ce924ad882a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### VS ECE: 4.58. Result ID: 69c3e77347f34ce924ad882b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### Oracle TS ECE: 4.87. Result ID: 69c3e77347f34ce924ad882f. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### LS-TS ECE: 5.05. Result ID: 69c3e77347f34ce924ad882e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### MCTS S=1 ECE: 5.07. Result ID: 69c3e77347f34ce924ad8829. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### SLTS ECE: 5.13. Result ID: 69c3e77347f34ce924ad8828. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### LS-TS ECE: 7.36. Result ID: 69c3e77347f34ce924ad883b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### TS ECE: 22.25. Result ID: 69c3e77347f34ce924ad8824. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### ATS ECE: 22.82. Result ID: 69c3e77347f34ce924ad8825. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### Dirichlet-Hard ECE: 23.02. Result ID: 69c3e77347f34ce924ad8827. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### Platt ECE: 23.1. Result ID: 69c3e77347f34ce924ad8826. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### Platt ECE: 24.18. Result ID: 69c3e77347f34ce924ad8833. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### Dirichlet-Hard ECE: 24.55. Result ID: 69c3e77347f34ce924ad8834. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### TS ECE: 24.65. Result ID: 69c3e77347f34ce924ad8831. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### ATS ECE: 25.35. Result ID: 69c3e77347f34ce924ad8832. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. ### Uncalibrated ECE: 34.56. Result ID: 69c3e77347f34ce924ad8823. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"ResNet-18"}. This record does not assert independent reproduction. ### Uncalibrated ECE: 37.55. Result ID: 69c3e77347f34ce924ad8830. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"ViT-S/16"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.