Classification on HMDA (test)
1.49Oracle ECEPlatt
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| PlattBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,UU)2026.05 | 1.49 | 6.12 | 0.23 | 1.78 | 10.96 | 0.31 | 86.67 | |
| TempBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,UU)2026.05 | 1.98 | 6.65 | 0.22 | 2.14 | 10.33 | 0.31 | 86.6 | |
| PlattBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,PU)2026.05 | 2.05 | 6 | 0.37 | 2.82 | 18.93 | 0.7 | 86.67 | |
| TempBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,PU)2026.05 | 2.46 | 6.66 | 0.34 | 3.06 | 18.34 | 0.69 | 86.6 | |
| WLMCBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,UU), step size=0.05, max updates=502026.05 | 2.97 | 9.2 | 0.37 | 2.99 | 11.22 | 0.42 | 86.6 | |
| WLMCBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,PN), step size=0.05, max updates=502026.05 | 3.09 | 9.38 | 0.39 | 3.09 | 9.38 | 0.39 | 86.6 | |
| WLMCBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,PU), step size=0.05, max updates=502026.05 | 3.11 | 9.25 | 0.48 | 3.71 | 20.54 | 0.82 | 86.6 | |
| PlattBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,PN)2026.05 | 3.42 | 6.67 | 0.53 | 3.42 | 6.67 | 0.53 | 86.62 | |
| ERMBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,PN)2026.05 | 3.97 | 10.21 | 0.88 | 3.97 | 10.21 | 0.88 | 86.6 | |
| WLMCBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,Pconf), step size=0.05, max updates=502026.05 | 3.97 | 10.21 | 0.88 | 0.41 | 1.07 | 0.15 | 86.6 | |
| WLMCBase Predictor=RandomForest, Distribution Shift (Base, Post)=(UU,UU), step size=0.05, max updates=502026.05 | 3.99 | 10.51 | 0.51 | 4.08 | 11.65 | 0.55 | 80.36 | |
| TempBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,Pconf)2026.05 | 4.42 | 10.61 | 0.99 | 0.24 | 0.93 | 0.05 | 86.6 | |
| PlattBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,Pconf)2026.05 | 4.46 | 10.76 | 0.99 | 0.23 | 0.93 | 0.05 | 86.6 | |
| WLMCBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PU,PU), step size=0.05, max updates=502026.05 | 4.98 | 19.12 | 0.9 | 4.86 | 25.52 | 1.04 | 77.55 | |
| PlattBase Predictor=RandomForest, Distribution Shift (Base, Post)=(Pconf,Pconf)2026.05 | 5.18 | 14.22 | 0.7 | 1.19 | 2.66 | 0.16 | 84.98 | |
| PlattBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PU,PU)2026.05 | 5.9 | 16.28 | 2.29 | 6.01 | 19.61 | 2.49 | 76.33 | |
| PlattBase Predictor=RandomForest, Distribution Shift (Base, Post)=(UU,UU)2026.05 | 7.17 | 9.09 | 2.02 | 7.23 | 9.99 | 2.03 | 81.44 | |
| WLMCBase Predictor=RandomForest, Distribution Shift (Base, Post)=(Pconf,Pconf), step size=0.05, max updates=502026.05 | 8.09 | 17.13 | 0.93 | 3.41 | 8.48 | 0.53 | 81.83 | |
| Platt(Base,Post)=(PN,PN), Base Predictor=SVM2026.05 | 9.83 | 21.02 | 4.44 | 9.83 | 21.02 | 4.44 | 74.38 | |
| TempBase Predictor=RandomForest, Distribution Shift (Base, Post)=(Pconf,Pconf)2026.05 | 10.79 | 22.38 | 1.91 | 6.07 | 8.4 | 1.17 | 77.62 | |
| Platt(Base,Post)=(PN,PU), Base Predictor=SVM2026.05 | 11.51 | 22.25 | 5.85 | 11.41 | 21.94 | 6.01 | 73.32 | |
| TempBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PU,PU)2026.05 | 11.97 | 17.91 | 3.52 | 12 | 22.08 | 3.46 | 76.98 | |
| ERMBase Predictor=RandomForest, Distribution Shift (Base, Post)=(Pconf,Pconf)2026.05 | 12.06 | 22.07 | 2.2 | 7 | 9.6 | 1.22 | 77.62 | |
| Platt(Base,Post)=(PN,UU), Base Predictor=SVM2026.05 | 12.18 | 22.71 | 5.84 | 12.09 | 19.87 | 5.83 | 73.32 | |
| Platt(Base,Post)=(PN,Pconf), Base Predictor=SVM2026.05 | 12.5 | 23.65 | 5.35 | 10.39 | 14.26 | 5.1 | 73.32 | |
| WLMC(Base,Post)=(PN,Pconf), Base Predictor=SVM2026.05 | 13.96 | 25.38 | 6.65 | 13.04 | 19.96 | 5.99 | 72.92 | |
| WLMC(Base,Post)=(PN,PN), Base Predictor=SVM2026.05 | 14.12 | 25.73 | 6.1 | 14.12 | 25.74 | 6.1 | 73.1 | |
| WLMC(Base,Post)=(PN,PU), Base Predictor=SVM2026.05 | 14.24 | 25.04 | 6.44 | 14.42 | 27.07 | 6.61 | 73.1 | |
| WLMC(Base,Post)=(PN,UU), Base Predictor=SVM2026.05 | 14.44 | 26.15 | 6.19 | 14.54 | 26.68 | 6.19 | 72.89 | |
| ERMBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PU,PU)2026.05 | 17.6 | 31.69 | 6.18 | 17.31 | 35.62 | 6.06 | 76.98 | |
| ERMBase Predictor=RandomForest, Distribution Shift (Base, Post)=(UU,UU)2026.05 | 19.8 | 24.91 | 6.87 | 19.82 | 29.13 | 6.88 | 79.62 | |
| Platt(Base,Post)=(PU,PU), Base Predictor=SVM2026.05 | 23.33 | 33.63 | 12.7 | 23.58 | 36.29 | 12.85 | 66.08 | |
| Platt(Base,Post)=(Pconf,Pconf), Base Predictor=SVM2026.05 | 25.06 | 33.07 | 13.12 | 24.53 | 29.18 | 12.95 | 68.77 | |
| TempBase Predictor=RandomForest, Distribution Shift (Base, Post)=(PN,PN)2026.05 | 25.52 | 27.67 | 13.82 | 25.52 | 27.67 | 13.82 | 86.6 | |
| WLMC(Base,Post)=(Pconf,Pconf), Base Predictor=SVM2026.05 | 25.8 | 34.63 | 12.57 | 25.43 | 32.38 | 12.44 | 68.48 | |
| Temp(Base,Post)=(PN,PU), Base Predictor=SVM2026.05 | 26.21 | 30.78 | 11.87 | 26.2 | 31.13 | 12.08 | 64.75 | |
| Temp(Base,Post)=(PN,UU), Base Predictor=SVM2026.05 | 26.22 | 30.78 | 11.87 | 26.24 | 30 | 11.92 | 64.75 | |
| Temp(Base,Post)=(PN,Pconf), Base Predictor=SVM2026.05 | 26.26 | 30.72 | 12.2 | 27.29 | 30.93 | 12.34 | 64.75 | |
| Temp(Base,Post)=(Pconf,Pconf), Base Predictor=SVM2026.05 | 26.81 | 30.38 | 12.46 | 27.62 | 30.39 | 12.67 | 64.42 | |
| Temp(Base,Post)=(UU,UU), Base Predictor=SVM2026.05 | 26.84 | 30.38 | 11.71 | 26.92 | 30.3 | 11.87 | 64.68 | |
| Platt(Base,Post)=(UU,UU), Base Predictor=SVM2026.05 | 27.35 | 34.76 | 13.97 | 27.6 | 35.05 | 14.1 | 66.83 | |
| Temp(Base,Post)=(PU,PU), Base Predictor=SVM2026.05 | 27.71 | 31.12 | 13.3 | 27.79 | 32.04 | 13.33 | 56.52 | |
| WLMC(Base,Post)=(PU,PU), Base Predictor=SVM2026.05 | 28.09 | 38.71 | 14.79 | 28.15 | 40.49 | 14.84 | 64.04 | |
| TempBase Predictor=RandomForest, Distribution Shift (Base, Post)=(UU,UU)2026.05 | 28.64 | 33.61 | 10.78 | 28.65 | 33.34 | 10.77 | 79.62 | |
| ERM(Base,Post)=(PN,PN), Base Predictor=SVM2026.05 | 35.25 | 41.65 | 18.99 | 35.25 | 41.65 | 18.99 | 64.75 | |
| Temp(Base,Post)=(PN,PN), Base Predictor=SVM2026.05 | 35.25 | 41.65 | 18.99 | 35.25 | 41.65 | 18.99 | 64.75 | |
| ERM(Base,Post)=(UU,UU), Base Predictor=SVM2026.05 | 35.32 | 41.86 | 19.03 | 35.44 | 42.6 | 19.08 | 64.68 | |
| ERM(Base,Post)=(Pconf,Pconf), Base Predictor=SVM2026.05 | 35.58 | 40.91 | 19.48 | 35.83 | 41.08 | 19.48 | 64.42 | |
| WLMC(Base,Post)=(UU,UU), Base Predictor=SVM2026.05 | 36.03 | 44.85 | 17.48 | 36.17 | 45.36 | 17.63 | 65.72 | |
| ERM(Base,Post)=(PU,PU), Base Predictor=SVM2026.05 | 43.48 | 51.54 | 22.79 | 43.3 | 52.88 | 22.85 | 56.52 |