# Annotator Modeling on aloi Source: [Annotator Modeling on aloi]() Collection index: [llms.txt]() Data updated: 2026-06-04T08:58:49.028Z Evaluation of Annotator Modeling performance on the aloi dataset, measured using AUROC. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Annotator Modeling | | Dataset (including any stated split) | aloi | | Metric | AUROC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [AUROC](): Higher is better; unit: PERCENT. ## Recorded results | Method | AUROC | Result date | Report paper | | --- | --- | --- | --- | | annot-mix | 88.7 | Not recorded | 68fa73141e209131cea5a5f9 | | annot-mix | 88.7 | Not recorded | 68fa73141e209131cea5a5f9 | | madl | 86.2 | Not recorded | 68fa73141e209131cea5a5f9 | | madl | 85.9 | Not recorded | 68fa73141e209131cea5a5f9 | | trace-reg | 70.8 | Not recorded | 68fa73141e209131cea5a5f9 | | trace-reg | 63.9 | Not recorded | 68fa73141e209131cea5a5f9 | | union-net | 56.2 | Not recorded | 68fa73141e209131cea5a5f9 | | union-net | 56.1 | Not recorded | 68fa73141e209131cea5a5f9 | | geo-reg-w | 55.6 | Not recorded | 68fa73141e209131cea5a5f9 | | geo-reg-f | 55.4 | Not recorded | 68fa73141e209131cea5a5f9 | | geo-reg-w | 55.3 | Not recorded | 68fa73141e209131cea5a5f9 | | geo-reg-f | 55.1 | Not recorded | 68fa73141e209131cea5a5f9 | | crowd-layer | 54.1 | Not recorded | 68fa73141e209131cea5a5f9 | | conal | 52.2 | Not recorded | 68fa73141e209131cea5a5f9 | | conal | 52.1 | Not recorded | 68fa73141e209131cea5a5f9 | | crowd-ar | 50.8 | Not recorded | 68fa73141e209131cea5a5f9 | | crowd-ar | 50.5 | Not recorded | 68fa73141e209131cea5a5f9 | | crowd-layer | 48.1 | Not recorded | 68fa73141e209131cea5a5f9 | ### annot-mix AUROC: 88.7. Result ID: 6a213e27fbc373592c1cf483. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Last Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### annot-mix AUROC: 88.7. Result ID: 6a213e27fbc373592c1cf48c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Best Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### madl AUROC: 86.2. Result ID: 6a213e27fbc373592c1cf488. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Best Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### madl AUROC: 85.9. Result ID: 6a213e27fbc373592c1cf47f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Last Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### trace-reg AUROC: 70.8. Result ID: 6a213e27fbc373592c1cf47c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Last Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### trace-reg AUROC: 63.9. Result ID: 6a213e27fbc373592c1cf485. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Best Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### union-net AUROC: 56.2. Result ID: 6a213e27fbc373592c1cf47e. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Last Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### union-net AUROC: 56.1. Result ID: 6a213e27fbc373592c1cf487. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Best Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### geo-reg-w AUROC: 55.6. Result ID: 6a213e27fbc373592c1cf481. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Last Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### geo-reg-f AUROC: 55.4. Result ID: 6a213e27fbc373592c1cf480. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Last Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### geo-reg-w AUROC: 55.3. Result ID: 6a213e27fbc373592c1cf48a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Best Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### geo-reg-f AUROC: 55.1. Result ID: 6a213e27fbc373592c1cf489. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Best Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### crowd-layer AUROC: 54.1. Result ID: 6a213e27fbc373592c1cf484. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Best Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### conal AUROC: 52.2. Result ID: 6a213e27fbc373592c1cf47d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Last Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### conal AUROC: 52.1. Result ID: 6a213e27fbc373592c1cf486. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Best Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### crowd-ar AUROC: 50.8. Result ID: 6a213e27fbc373592c1cf48b. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Best Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### crowd-ar AUROC: 50.5. Result ID: 6a213e27fbc373592c1cf482. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Last Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. ### crowd-layer AUROC: 48.1. Result ID: 6a213e27fbc373592c1cf47b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Epoch":"Last Epoch","False label fraction":"43.4"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.