Reward Modeling on UltraFeedback (Top-50 Regression Metrics)
0.794R2 (Top-50)UARM
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| UARMMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.794 | 0.383 | 0.461 | |
| ClearMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.77 | 0.432 | 0.513 | |
| MCNFMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.769 | 0.503 | 0.513 | |
| SCCPMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.75 | 0.547 | 0.53 | |
| TorchNautMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.746 | 0.463 | 0.499 | |
| Packed EnsembleMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.71 | 0.491 | 0.514 | |
| ACIMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.678 | 0.523 | 0.534 | |
| DERMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.663 | 0.403 | 0.47 | |
| WCPMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.646 | 1.15 | 0.793 | |
| Deep EnsembleMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.632 | 0.473 | 0.507 | |
| CQRMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.623 | 0.406 | 0.51 | |
| SCPMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.609 | 1.345 | 0.815 | |
| MC-DropoutMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.607 | 0.949 | 0.717 | |
| NaiveMiscoverage rate (alpha)=0.1, Confidence threshold=50% most confident samples2026.06 | 0.563 | 1.481 | 0.832 |