# Active Preference Learning for Ordering Items In- and Out-of-sample Source: [Active Preference Learning for Ordering Items In- and Out-of-sample]() Collection index: [llms.txt]() Data updated: 2026-01-02T07:37:03.787Z Authors: Herman Bergstr\\"om, Emil Carlsson, Devdatt Dubhashi, Fredrik D. Johansson Published: 2024-05-05T21:44:03.000Z ## Abstract Learning an ordering of items based on pairwise comparisons is useful when items are difficult to rate consistently on an absolute scale, for example, when annotators have to make subjective assessments. When exhaustive comparison is infeasible, actively sampling item pairs can reduce the number of annotations necessary for learning an accurate ordering. However, many algorithms ignore shared structure between items, limiting their sample efficiency and precluding generalization to new items. It is also common to disregard how noise in comparisons varies between item pairs, despite it being informative of item similarity. In this work, we study active preference learning for ordering items with contextual attributes, both in- and out-of-sample. We give an upper bound on the expected ordering error of a logistic preference model as a function of which items have been compared. Next, we propose an active learning strategy that samples items to minimize this bound by accounting for aleatoric and epistemic uncertainty in comparisons. We evaluate the resulting algorithm, and a variant aimed at reducing model misspecification, in multiple realistic ordering tasks with comparisons made by human annotators. Our results demonstrate superior sample efficiency and generalization compared to non-contextual ranking approaches and active preference learning baselines. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Metalens image quality grading on Metaformer Landscape](): Metalens image quality grading; Metaformer Landscape; Kendall Correlation: 0.783. - [Metalens image quality grading on DRMI Animal](): Metalens image quality grading; DRMI Animal; Kendall Correlation: 0.426. - [Metalens image quality grading on Metaformer Animal](): Metalens image quality grading; Metaformer Animal; Kendall Correlation: 0.272. - [Metalens image quality grading on DRMI Landscape](): Metalens image quality grading; DRMI Landscape; Kendall's Tau: 0.423. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()