Calibration on HMDA (test)
1.29Oracle ECEPlatt
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Platt(Base,Post)=(PU,PU), Base Predictor=NaiveBayes2026.05 | 1.29 | 14.66 | 1.46 | 1.73 | 19.85 | 1.7 | 77.68 | |
| WLMCBase Predictor=LogisticRegression, (Base,Post)=(PN,PU)2026.05 | 1.48 | 10.06 | 0.26 | 2.3 | 17.22 | 0.72 | 82.68 | |
| WLMCBase Predictor=LogisticRegression, (Base,Post)=(PN,Pconf)2026.05 | 1.49 | 10.06 | 0.26 | 3.98 | 6.38 | 1.03 | 82.68 | |
| ERMBase Predictor=LogisticRegression, (Base,Post)=(PN,PN)2026.05 | 1.5 | 10.06 | 0.26 | 1.49 | 10.06 | 0.26 | 82.68 | |
| WLMCBase Predictor=LogisticRegression, (Base,Post)=(PN,PN)2026.05 | 1.5 | 10.06 | 0.26 | 1.5 | 10.06 | 0.26 | 82.68 | |
| WLMCBase Predictor=LogisticRegression, (Base,Post)=(PN,UU)2026.05 | 1.5 | 10.06 | 0.26 | 1.5 | 11.52 | 0.32 | 82.68 | |
| TempBase Predictor=LogisticRegression, (Base,Post)=(PN,UU)2026.05 | 1.54 | 10.08 | 0.28 | 1.57 | 11.69 | 0.34 | 82.68 | |
| PlattBase Predictor=LogisticRegression, (Base,Post)=(PN,UU)2026.05 | 1.57 | 10.05 | 0.28 | 1.57 | 11.62 | 0.35 | 82.68 | |
| WLMCBase Predictor=LogisticRegression, (Base,Post)=(UU,UU)2026.05 | 1.63 | 14.36 | 0.56 | 1.64 | 13.51 | 0.58 | 75.4 | |
| PlattBase predictor=MLP, Scenario=(PU,PU)2026.05 | 1.74 | 12.28 | 1.35 | 1.87 | 19.81 | 1.58 | 77.09 | |
| WLMCBase Predictor=LogisticRegression, (Base,Post)=(PU,PU)2026.05 | 1.76 | 14.23 | 0.71 | 2.2 | 23.48 | 0.99 | 75.43 | |
| TempBase Predictor=LogisticRegression, (Base,Post)=(PN,PU)2026.05 | 2 | 10.23 | 0.4 | 2.89 | 17.55 | 0.89 | 82.68 | |
| WLMC(Base,Post)=(PU,PU), Base Predictor=NaiveBayes2026.05 | 2.24 | 12.47 | 0.79 | 2.45 | 19.56 | 1.05 | 76.57 | |
| WLMCBase Predictor=DecisionTree, (Base, Post) setting=(UU,UU)2026.05 | 2.36 | 8.28 | 0.42 | 2.48 | 11.39 | 0.46 | 85.6 | |
| PlattBase predictor=MLP, Scenario=(UU,UU)2026.05 | 2.4 | 8.92 | 0.38 | 2.53 | 12.21 | 0.45 | 81.46 | |
| PlattBase Predictor=LogisticRegression, (Base,Post)=(PN,PU)2026.05 | 2.44 | 10.4 | 0.42 | 3.22 | 17.8 | 0.76 | 82.69 | |
| PlattBase Predictor=LogisticRegression, (Base,Post)=(Pconf,Pconf)2026.05 | 2.44 | 17.76 | 1.29 | 1.37 | 5.94 | 1.7 | 75.44 | |
| ERMBase Predictor=LogisticRegression, (Base,Post)=(UU,UU)2026.05 | 2.53 | 14.14 | 1.76 | 2.65 | 13.54 | 1.81 | 75.4 | |
| TempBase Predictor=LogisticRegression, (Base,Post)=(UU,UU)2026.05 | 2.55 | 14.41 | 1.79 | 2.63 | 14.46 | 1.84 | 75.4 | |
| PlattBase Predictor=LogisticRegression, (Base,Post)=(UU,UU)2026.05 | 2.55 | 14.05 | 1.8 | 2.66 | 13.85 | 1.86 | 75.39 | |
| WLMCBase Predictor=DecisionTree, (Base, Post) setting=(PN,PU)2026.05 | 2.63 | 6.93 | 0.46 | 2.74 | 15.45 | 0.69 | 86.03 | |
| WLMCBase Predictor=DecisionTree, (Base, Post) setting=(PN,UU)2026.05 | 2.64 | 6.95 | 0.4 | 2.7 | 9.86 | 0.43 | 86.03 | |
| WLMCBase Predictor=DecisionTree, (Base, Post) setting=(PN,PN)2026.05 | 2.67 | 6.99 | 0.41 | 2.67 | 6.99 | 0.41 | 86.03 | |
| ERMBase Predictor=DecisionTree, (Base, Post) setting=(UU,UU)2026.05 | 2.67 | 8.51 | 0.62 | 2.76 | 11.42 | 0.67 | 85.6 | |
| ERMBase Predictor=DecisionTree, (Base, Post) setting=(PN,PN)2026.05 | 2.73 | 7.04 | 0.44 | 2.73 | 7.04 | 0.44 | 86.02 | |
| Platt(Base,Post)=(Pconf,Pconf), Base Predictor=NaiveBayes2026.05 | 2.78 | 16.11 | 0.51 | 0.95 | 5.36 | 0.22 | 78.11 | |
| TempBase Predictor=LogisticRegression, (Base,Post)=(Pconf,Pconf)2026.05 | 2.79 | 18.25 | 1.49 | 1.11 | 6.16 | 1.65 | 75.44 | |
| PlattBase predictor=MLP, Scenario=(PN,PN)2026.05 | 2.81 | 9.03 | 0.82 | 2.81 | 9.03 | 0.82 | 80.15 | |
| PlattBase Predictor=LogisticRegression, (Base,Post)=(PU,PU)2026.05 | 2.85 | 15.02 | 1.86 | 3.01 | 21.79 | 1.89 | 75.41 | |
| Platt(Base,Post)=(UU,UU), Base Predictor=NaiveBayes2026.05 | 2.91 | 12.95 | 0.87 | 3.01 | 13.99 | 0.91 | 80.99 | |
| TempBase Predictor=LogisticRegression, (Base,Post)=(PU,PU)2026.05 | 2.93 | 14.66 | 1.85 | 3.13 | 22.03 | 1.87 | 75.41 | |
| Platt(Base,Post)=(PN,UU), Base Predictor=NaiveBayes2026.05 | 2.97 | 12.91 | 0.66 | 3.1 | 14.09 | 0.69 | 81.65 | |
| ERMBase Predictor=LogisticRegression, (Base,Post)=(PU,PU)2026.05 | 2.99 | 14.96 | 1.88 | 3.19 | 22.16 | 1.91 | 75.41 | |
| WLMC(Base,Post)=(PN,Pconf), Base Predictor=NaiveBayes2026.05 | 3.15 | 11.47 | 0.91 | 2.07 | 7.63 | 0.47 | 80.79 | |
| Platt(Base,Post)=(PN,PU), Base Predictor=NaiveBayes2026.05 | 3.16 | 13.11 | 0.67 | 3.75 | 20.01 | 0.97 | 81.64 | |
| WLMC(Base,Post)=(PN,PU), Base Predictor=NaiveBayes2026.05 | 3.47 | 10.09 | 0.8 | 3.83 | 16.76 | 1.02 | 80.83 | |
| TempBase predictor=MLP, Scenario=(PU,PU)2026.05 | 3.48 | 10.97 | 1.7 | 3.75 | 18.17 | 1.93 | 76.89 | |
| PlattBase Predictor=DecisionTree, (Base, Post) setting=(PN,PU)2026.05 | 3.55 | 8.56 | 0.65 | 3.67 | 17.46 | 0.76 | 86 | |
| PlattBase predictor=MLP, Scenario=(PN,UU)2026.05 | 3.58 | 9.56 | 0.78 | 3.63 | 11.99 | 0.79 | 80.52 | |
| PlattBase predictor=MLP, Scenario=(PN,PU)2026.05 | 3.6 | 10.16 | 0.87 | 3.65 | 17.22 | 1.04 | 80.53 | |
| WLMCBase Predictor=LogisticRegression, (Base,Post)=(Pconf,Pconf)2026.05 | 3.61 | 18.44 | 1.22 | 1.65 | 6.57 | 0.81 | 75.44 | |
| TempBase Predictor=DecisionTree, (Base, Post) setting=(PN,PU)2026.05 | 3.64 | 8.79 | 0.65 | 3.75 | 17.97 | 0.75 | 86.03 | |
| WLMCBase predictor=MLP, Scenario=(UU,UU)2026.05 | 3.69 | 9.47 | 0.52 | 3.79 | 12.07 | 0.54 | 81.36 | |
| Platt(Base,Post)=(PN,PN), Base Predictor=NaiveBayes2026.05 | 3.72 | 11.27 | 0.98 | 3.72 | 11.27 | 0.98 | 81.7 | |
| TempBase predictor=MLP, Scenario=(UU,UU)2026.05 | 3.85 | 8.65 | 0.45 | 3.97 | 13.04 | 0.52 | 81.31 | |
| WLMCBase predictor=MLP, Scenario=(PU,PU)2026.05 | 3.9 | 13.87 | 1.17 | 4.24 | 18.03 | 1.23 | 76.89 | |
| WLMCBase Predictor=DecisionTree, (Base, Post) setting=(PU,PU)2026.05 | 3.92 | 8.87 | 0.72 | 4.03 | 17.84 | 0.86 | 84.53 | |
| PlattBase Predictor=DecisionTree, (Base, Post) setting=(PN,UU)2026.05 | 4.11 | 8.98 | 0.8 | 4.05 | 12.19 | 0.8 | 86.02 | |
| WLMCBase predictor=MLP, Scenario=(PN,Pconf)2026.05 | 4.17 | 12.8 | 1.15 | 4.33 | 12.72 | 1.11 | 79.76 | |
| TempBase Predictor=DecisionTree, (Base, Post) setting=(PN,UU)2026.05 | 4.24 | 9.6 | 0.76 | 4.19 | 12.08 | 0.81 | 86.03 | |
| ERMBase Predictor=DecisionTree, (Base, Post) setting=(PU,PU)2026.05 | 4.36 | 9.23 | 0.77 | 4.45 | 17.77 | 0.96 | 84.5 | |
| WLMCBase Predictor=DecisionTree, (Base, Post) setting=(PN,Pconf)2026.05 | 4.38 | 8.29 | 1.7 | 3.92 | 6.45 | 0.64 | 86.03 | |
| PlattBase predictor=MLP, Scenario=(PN,Pconf)2026.05 | 4.4 | 11.53 | 1.05 | 2.14 | 4.4 | 0.57 | 80.18 | |
| WLMCBase predictor=MLP, Scenario=(PN,PN)2026.05 | 4.42 | 13.39 | 1 | 4.43 | 13.39 | 1 | 79.83 | |
| PlattBase Predictor=DecisionTree, (Base, Post) setting=(Pconf,Pconf)2026.05 | 4.43 | 12.8 | 1 | 0.36 | 1.87 | 0.1 | 86.02 | |
| PlattBase predictor=MLP, Scenario=(Pconf,Pconf)2026.05 | 4.48 | 13.35 | 0.91 | 1.92 | 4.28 | 0.48 | 78.65 | |
| WLMC(Base,Post)=(Pconf,Pconf), Base Predictor=NaiveBayes2026.05 | 4.5 | 15.12 | 0.52 | 4.38 | 13.03 | 0.56 | 77.09 | |
| WLMC(Base,Post)=(UU,UU), Base Predictor=NaiveBayes2026.05 | 4.52 | 11.83 | 1.1 | 4.64 | 12.4 | 1.14 | 80.12 | |
| Platt(Base,Post)=(PN,Pconf), Base Predictor=NaiveBayes2026.05 | 4.58 | 14.71 | 0.95 | 1.37 | 5.73 | 0.36 | 80.81 | |
| WLMCBase predictor=MLP, Scenario=(PN,UU)2026.05 | 4.6 | 14 | 1.15 | 4.74 | 14.54 | 1.21 | 79.84 | |
| PlattBase Predictor=DecisionTree, (Base, Post) setting=(PN,PN)2026.05 | 4.67 | 7.97 | 0.98 | 4.67 | 7.97 | 0.98 | 85.75 | |
| TempBase predictor=MLP, Scenario=(Pconf,Pconf)2026.05 | 4.84 | 12.23 | 0.94 | 3.52 | 5.69 | 0.61 | 78.64 | |
| WLMC(Base,Post)=(PN,PN), Base Predictor=NaiveBayes2026.05 | 4.95 | 12.36 | 1.07 | 4.95 | 12.36 | 1.07 | 80.8 | |
| WLMC(Base,Post)=(PN,UU), Base Predictor=NaiveBayes2026.05 | 4.97 | 11.99 | 1.08 | 5.04 | 12.98 | 1.13 | 80.8 | |
| WLMCBase predictor=MLP, Scenario=(PN,PU)2026.05 | 5.06 | 12.5 | 1.26 | 4.99 | 18.08 | 1.55 | 79.83 | |
| PlattBase Predictor=LogisticRegression, (Base,Post)=(PN,PN)2026.05 | 5.13 | 13.39 | 0.79 | 5.13 | 13.4 | 0.79 | 82.61 | |
| WLMCBase predictor=MLP, Scenario=(Pconf,Pconf)2026.05 | 5.24 | 13.47 | 1.24 | 4.72 | 9.84 | 1.03 | 78.69 | |
| PlattBase Predictor=DecisionTree, (Base, Post) setting=(PU,PU)2026.05 | 5.27 | 14.41 | 1.6 | 5.39 | 20.01 | 1.68 | 84.19 | |
| TempBase Predictor=DecisionTree, (Base, Post) setting=(PU,PU)2026.05 | 5.39 | 13.91 | 1.59 | 5.34 | 19.58 | 1.69 | 84.51 | |
| TempBase Predictor=DecisionTree, (Base, Post) setting=(UU,UU)2026.05 | 5.53 | 10.41 | 0.92 | 5.5 | 13.31 | 0.97 | 85.6 | |
| PlattBase Predictor=DecisionTree, (Base, Post) setting=(UU,UU)2026.05 | 5.56 | 11.04 | 0.96 | 5.56 | 14.09 | 1.01 | 85.25 | |
| TempBase predictor=MLP, Scenario=(PN,UU)2026.05 | 5.94 | 10.96 | 0.97 | 5.89 | 12.94 | 0.98 | 79.83 | |
| TempBase Predictor=LogisticRegression, (Base,Post)=(PN,Pconf)2026.05 | 5.98 | 12.24 | 1.23 | 4.75 | 6 | 1.05 | 82.68 | |
| TempBase predictor=MLP, Scenario=(PN,PU)2026.05 | 5.99 | 11.36 | 1.06 | 6.04 | 17.72 | 1.23 | 79.83 | |
| TempBase predictor=MLP, Scenario=(PN,Pconf)2026.05 | 6 | 11.25 | 1.2 | 5.4 | 8.7 | 0.8 | 79.83 | |
| ERMBase predictor=MLP, Scenario=(UU,UU)2026.05 | 6.39 | 10.96 | 1.32 | 6.51 | 13.54 | 1.35 | 81.31 | |
| PlattBase Predictor=DecisionTree, (Base, Post) setting=(PN,Pconf)2026.05 | 7.05 | 11.9 | 1.6 | 3.24 | 4.91 | 0.79 | 85.31 | |
| PlattBase Predictor=LogisticRegression, (Base,Post)=(PN,Pconf)2026.05 | 7.44 | 14.81 | 1.44 | 4.76 | 6.39 | 1.19 | 80.79 | |
| WLMCBase Predictor=DecisionTree, (Base, Post) setting=(Pconf,Pconf)2026.05 | 7.45 | 16.86 | 0.96 | 4.34 | 6.71 | 0.47 | 82.99 | |
| TempBase Predictor=DecisionTree, (Base, Post) setting=(PN,Pconf)2026.05 | 7.56 | 11.56 | 1.38 | 3.46 | 4.74 | 0.98 | 86.03 | |
| Temp(Base,Post)=(PN,PU), Base Predictor=NaiveBayes2026.05 | 7.58 | 14.09 | 0.81 | 7.66 | 19.34 | 1.08 | 80.8 | |
| Temp(Base,Post)=(PN,UU), Base Predictor=NaiveBayes2026.05 | 7.59 | 14.05 | 0.79 | 7.62 | 14.39 | 0.81 | 80.8 | |
| Temp(Base,Post)=(PN,Pconf), Base Predictor=NaiveBayes2026.05 | 8.25 | 13.23 | 1.28 | 8.54 | 14.56 | 0.75 | 80.8 | |
| ERMBase Predictor=DecisionTree, (Base, Post) setting=(Pconf,Pconf)2026.05 | 8.61 | 18.53 | 1.24 | 5.34 | 7.6 | 0.7 | 82.01 | |
| TempBase Predictor=DecisionTree, (Base, Post) setting=(Pconf,Pconf)2026.05 | 9.11 | 19.03 | 1.37 | 4.56 | 6.7 | 0.74 | 82.01 | |
| Temp(Base,Post)=(UU,UU), Base Predictor=NaiveBayes2026.05 | 9.29 | 14.7 | 2.73 | 9.41 | 15.4 | 2.78 | 75.04 | |
| Temp(Base,Post)=(Pconf,Pconf), Base Predictor=NaiveBayes2026.05 | 9.54 | 15.26 | 1.41 | 10.88 | 17.93 | 1.51 | 75.19 | |
| ERMBase Predictor=LogisticRegression, (Base,Post)=(Pconf,Pconf)2026.05 | 10.51 | 26.46 | 4.32 | 7.78 | 13.52 | 3.4 | 75.44 | |
| TempBase predictor=MLP, Scenario=(PN,PN)2026.05 | 13.35 | 19.01 | 5.11 | 13.35 | 19.01 | 5.11 | 79.83 | |
| ERM(Base,Post)=(PN,PN), Base Predictor=NaiveBayes2026.05 | 13.44 | 19.83 | 4.19 | 13.44 | 19.83 | 4.19 | 80.8 | |
| ERM(Base,Post)=(UU,UU), Base Predictor=NaiveBayes2026.05 | 14.93 | 20.64 | 5.92 | 15.04 | 22.34 | 6.04 | 80.12 | |
| ERMBase predictor=MLP, Scenario=(Pconf,Pconf)2026.05 | 15.3 | 22.6 | 6.52 | 14.28 | 16.7 | 6.15 | 78.64 | |
| ERM(Base,Post)=(Pconf,Pconf), Base Predictor=NaiveBayes2026.05 | 15.62 | 25.92 | 3.11 | 14.97 | 25.29 | 2.76 | 75.19 | |
| Temp(Base,Post)=(PU,PU), Base Predictor=NaiveBayes2026.05 | 15.9 | 23.53 | 8.34 | 15.97 | 25.86 | 8.46 | 55.76 | |
| ERMBase predictor=MLP, Scenario=(PN,PN)2026.05 | 16.28 | 22.03 | 6.79 | 16.28 | 22.03 | 6.79 | 79.83 | |
| ERMBase predictor=MLP, Scenario=(PU,PU)2026.05 | 18.78 | 27.77 | 8.63 | 18.81 | 29.43 | 8.77 | 76.89 | |
| ERM(Base,Post)=(PU,PU), Base Predictor=NaiveBayes2026.05 | 19.68 | 29.75 | 9.86 | 19.74 | 31.53 | 9.99 | 76.11 | |
| Temp(Base,Post)=(PN,PN), Base Predictor=NaiveBayes2026.05 | 19.69 | 21.4 | 12.58 | 19.69 | 21.4 | 12.58 | 80.8 | |
| TempBase Predictor=LogisticRegression, (Base,Post)=(PN,PN)2026.05 | 21.49 | 27.68 | 12.99 | 21.48 | 27.68 | 12.99 | 82.68 | |
| TempBase Predictor=DecisionTree, (Base, Post) setting=(PN,PN)2026.05 | 23.19 | 25.87 | 13.61 | 23.19 | 25.87 | 13.61 | 86.03 |