# Aggregate Quality Recovery on UltraFeedback (kappa\_max = 10 Aggregate) Source: [Aggregate Quality Recovery on UltraFeedback (kappa\_max = 10 Aggregate)]() Collection index: [llms.txt]() Data updated: 2026-05-13T10:44:35.595Z Evaluation of model performance on the UltraFeedback dataset using the Aggregate Quality Recovery methodology, focusing on metrics such as Q-hat, 95% Confidence Intervals, and comparison against the true value Q\*. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Aggregate Quality Recovery | | Dataset (including any stated split) | UltraFeedback kappa\_max = 10 (Aggregate) | | Metric | Q-hat | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 8 of 8 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Q-hat](): Higher is better; unit: SCALAR. - [95% CI (Lower Bound)](): Higher is better; unit: SCALAR. - [Absolute Error \|Q-hat - Q\*\|](): Lower is better; unit: SCALAR. - [CI Coverage of Q\*](): Higher is better; unit: PERCENT. ## Recorded results | Method | Q-hat | Result date | Report paper | | --- | --- | --- | --- | | Enhanced (global sentiment) | 0.8467 | Not recorded | 6a0405dbdbc836b87d827881 | | Ground truth Q\* | 0.6249 | Not recorded | 6a0405dbdbc836b87d827881 | | Hierarchical-Informed | 0.5484 | Not recorded | 6a0405dbdbc836b87d827881 | | IPW | 0.379 | Not recorded | 6a0405dbdbc836b87d827881 | | Corrected-Global (3-param) | 0.3705 | Not recorded | 6a0405dbdbc836b87d827881 | | Basic (two-stage) | 0.3638 | Not recorded | 6a0405dbdbc836b87d827881 | | Hierarchical-Sentiment | 0.3004 | Not recorded | 6a0405dbdbc836b87d827881 | | Naive mean | 0.2617 | Not recorded | 6a0405dbdbc836b87d827881 | ### Enhanced (global sentiment) Q-hat: 0.8467. Result ID: 6a04553847f34ce924eaec6e. [Reporting paper]() Recorded evaluation configuration: {"kappa\_max":"10"}. This record does not assert independent reproduction. ### Ground truth Q\* Q-hat: 0.6249. Result ID: 6a04553847f34ce924eaec6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"kappa\_max":"10"}. This record does not assert independent reproduction. ### Hierarchical-Informed Q-hat: 0.5484. Result ID: 6a04553847f34ce924eaec70. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"kappa\_max":"10"}. This record does not assert independent reproduction. ### IPW Q-hat: 0.379. Result ID: 6a04553847f34ce924eaec6c. [Reporting paper]() Recorded evaluation configuration: {"kappa\_max":"10"}. This record does not assert independent reproduction. ### Corrected-Global (3-param) Q-hat: 0.3705. Result ID: 6a04553847f34ce924eaec71. [Reporting paper]() Recorded evaluation configuration: {"kappa\_max":"10"}. This record does not assert independent reproduction. ### Basic (two-stage) Q-hat: 0.3638. Result ID: 6a04553847f34ce924eaec6d. [Reporting paper]() Recorded evaluation configuration: {"kappa\_max":"10"}. This record does not assert independent reproduction. ### Hierarchical-Sentiment Q-hat: 0.3004. Result ID: 6a04553847f34ce924eaec6f. [Reporting paper]() Recorded evaluation configuration: {"kappa\_max":"10"}. This record does not assert independent reproduction. ### Naive mean Q-hat: 0.2617. Result ID: 6a04553847f34ce924eaec6b. [Reporting paper]() Recorded evaluation configuration: {"kappa\_max":"10"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.