# Mislabel detection on Weak Reference Labels Mislabels + subjective

Source: [Mislabel detection on Weak Reference Labels Mislabels + subjective](<https://www.sota2.com/research/sota/mislabel-detection-on-weak-reference-labels-mislabels-plus-subjective>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-01T07:34:16.474Z

Evaluation of model performance for Mislabel detection using the Weak Reference Labels Mislabels + subjective dataset, evaluated using AP and Precision at specific ranks (P@437, P@874).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Mislabel detection |
| Dataset (including any stated split) | Weak Reference Labels Mislabels + subjective |
| Metric | AP |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 9 of 9 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [AP](<https://www.sota2.com/research/sota/mislabel-detection-on-weak-reference-labels-mislabels-plus-subjective.md?metric=AP>): Higher is better; unit: PERCENT.

- [P@437](<https://www.sota2.com/research/sota/mislabel-detection-on-weak-reference-labels-mislabels-plus-subjective.md?metric=P%40437>): Higher is better; unit: PERCENT.

- [P@874](<https://www.sota2.com/research/sota/mislabel-detection-on-weak-reference-labels-mislabels-plus-subjective.md?metric=P%40874>): Higher is better; unit: PERCENT.

## Recorded results

| Method | AP | Result date | Report paper |
| --- | --- | --- | --- |
| 4PL Δℓi (proposed) | 76.2 | Not recorded | 6a1d1206dbd7150506f83f1a |
| top-10 disagreement | 73.8 | Not recorded | 6a1d1206dbd7150506f83f1a |
| XGBoost (4PL params) | 70.3 | Not recorded | 6a1d1206dbd7150506f83f1a |
| plain 2PL (low ai) | 70 | Not recorded | 6a1d1206dbd7150506f83f1a |
| 4PL, low di | 69.8 | Not recorded | 6a1d1206dbd7150506f83f1a |
| plain 4PL, single-stage (low di) | 69.8 | Not recorded | 6a1d1206dbd7150506f83f1a |
| GLAD | 66.1 | Not recorded | 6a1d1206dbd7150506f83f1a |
| low ri | 62.6 | Not recorded | 6a1d1206dbd7150506f83f1a |
| overall disagreement | 57.3 | Not recorded | 6a1d1206dbd7150506f83f1a |

### 4PL Δℓi (proposed)

AP: 76.2. Result ID: 6a1d347dfbc373592c1601af.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

Recorded evaluation configuration: {"Criterion":"Forced-ceiling likelihood ratio"}. This record does not assert independent reproduction.

### top-10 disagreement

AP: 73.8. Result ID: 6a1d347dfbc373592c1601b0.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### XGBoost (4PL params)

AP: 70.3. Result ID: 6a1d347dfbc373592c1601b1.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

Recorded evaluation configuration: {"Model":"XGBoost","Features":"4PL parameters"}. This record does not assert independent reproduction.

### plain 2PL (low ai)

AP: 70. Result ID: 6a1d347dfbc373592c1601b6.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

Recorded evaluation configuration: {"Model":"2PL","Selection":"low discrimination"}. This record does not assert independent reproduction.

### 4PL, low di

AP: 69.8. Result ID: 6a1d347dfbc373592c1601b2.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

Recorded evaluation configuration: {"Model":"4PL","Selection":"low difficulty"}. This record does not assert independent reproduction.

### plain 4PL, single-stage (low di)

AP: 69.8. Result ID: 6a1d347dfbc373592c1601b5.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

Recorded evaluation configuration: {"Model":"4PL","Protocol":"single-stage","Selection":"low difficulty"}. This record does not assert independent reproduction.

### GLAD

AP: 66.1. Result ID: 6a1d347dfbc373592c1601b7.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### low ri

AP: 62.6. Result ID: 6a1d347dfbc373592c1601b3.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

Recorded evaluation configuration: {"Selection":"low response probability"}. This record does not assert independent reproduction.

### overall disagreement

AP: 57.3. Result ID: 6a1d347dfbc373592c1601b4.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1206dbd7150506f83f1a.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.