# Preference Evaluation on Mturk user study Overall Preferences 1.0 Source: [Preference Evaluation on Mturk user study Overall Preferences 1.0]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:54:31.821Z Overall preference evaluation results derived from the Mturk user study version 1.0, quantified by the observed preference fraction. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Preference Evaluation | | Dataset (including any stated split) | Mturk user study Overall Preferences 1.0 | | Metric | Overall Preference Fraction | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Overall Preference Fraction](): Higher is better; unit: PERCENT. ## Recorded results | Method | Overall Preference Fraction | Result date | Report paper | | --- | --- | --- | --- | | Policy R1 | 76 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy C | 75 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy R2 | 73 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy C | 70 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy E | 66 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy R1 | 66 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy R2 | 66 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy R2 | 66 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy B | 61 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy C | 61 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy R1 | 59 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy C | 57 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy D | 57 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy R2 | 56 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy E | 52 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy R2 | 51 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy B | 50 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy B | 50 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy D | 50 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy E | 50 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy D | 48 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy A | 43 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy D | 43 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy A | 39 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy E | 39 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy R1 | 39 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy A | 34 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy B | 30 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy R1 | 29 | Not recorded | 68fa69eb1e209131cea24e67 | | Policy A | 25 | Not recorded | 68fa69eb1e209131cea24e67 | ### Policy R1 Overall Preference Fraction: 76. Result ID: 698df35fffc3901de099ba7c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"B"}. This record does not assert independent reproduction. ### Policy C Overall Preference Fraction: 75. Result ID: 698df35fffc3901de099ba6f. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"A"}. This record does not assert independent reproduction. ### Policy R2 Overall Preference Fraction: 73. Result ID: 698df35fffc3901de099ba81. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"B"}. This record does not assert independent reproduction. ### Policy C Overall Preference Fraction: 70. Result ID: 698df35fffc3901de099ba70. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"B"}. This record does not assert independent reproduction. ### Policy E Overall Preference Fraction: 66. Result ID: 698df35fffc3901de099ba77. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"A"}. This record does not assert independent reproduction. ### Policy R1 Overall Preference Fraction: 66. Result ID: 698df35fffc3901de099ba7b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"A"}. This record does not assert independent reproduction. ### Policy R2 Overall Preference Fraction: 66. Result ID: 698df35fffc3901de099ba80. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"A"}. This record does not assert independent reproduction. ### Policy R2 Overall Preference Fraction: 66. Result ID: 698df35fffc3901de099ba82. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"C"}. This record does not assert independent reproduction. ### Policy B Overall Preference Fraction: 61. Result ID: 698df35fffc3901de099ba6b. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"A"}. This record does not assert independent reproduction. ### Policy C Overall Preference Fraction: 61. Result ID: 698df35fffc3901de099ba72. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"E"}. This record does not assert independent reproduction. ### Policy R1 Overall Preference Fraction: 59. Result ID: 698df35fffc3901de099ba7e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"D"}. This record does not assert independent reproduction. ### Policy C Overall Preference Fraction: 57. Result ID: 698df35fffc3901de099ba71. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"D"}. This record does not assert independent reproduction. ### Policy D Overall Preference Fraction: 57. Result ID: 698df35fffc3901de099ba73. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"A"}. This record does not assert independent reproduction. ### Policy R2 Overall Preference Fraction: 56. Result ID: 698df35fffc3901de099ba83. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"D"}. This record does not assert independent reproduction. ### Policy E Overall Preference Fraction: 52. Result ID: 698df35fffc3901de099ba7a. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"D"}. This record does not assert independent reproduction. ### Policy R2 Overall Preference Fraction: 51. Result ID: 698df35fffc3901de099ba84. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"E"}. This record does not assert independent reproduction. ### Policy B Overall Preference Fraction: 50. Result ID: 698df35fffc3901de099ba6d. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"D"}. This record does not assert independent reproduction. ### Policy B Overall Preference Fraction: 50. Result ID: 698df35fffc3901de099ba6e. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"E"}. This record does not assert independent reproduction. ### Policy D Overall Preference Fraction: 50. Result ID: 698df35fffc3901de099ba74. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"B"}. This record does not assert independent reproduction. ### Policy E Overall Preference Fraction: 50. Result ID: 698df35fffc3901de099ba78. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"B"}. This record does not assert independent reproduction. ### Policy D Overall Preference Fraction: 48. Result ID: 698df35fffc3901de099ba76. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"E"}. This record does not assert independent reproduction. ### Policy A Overall Preference Fraction: 43. Result ID: 698df35fffc3901de099ba69. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"D"}. This record does not assert independent reproduction. ### Policy D Overall Preference Fraction: 43. Result ID: 698df35fffc3901de099ba75. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"C"}. This record does not assert independent reproduction. ### Policy A Overall Preference Fraction: 39. Result ID: 698df35fffc3901de099ba67. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"B"}. This record does not assert independent reproduction. ### Policy E Overall Preference Fraction: 39. Result ID: 698df35fffc3901de099ba79. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"C"}. This record does not assert independent reproduction. ### Policy R1 Overall Preference Fraction: 39. Result ID: 698df35fffc3901de099ba7f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"E"}. This record does not assert independent reproduction. ### Policy A Overall Preference Fraction: 34. Result ID: 698df35fffc3901de099ba6a. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"E"}. This record does not assert independent reproduction. ### Policy B Overall Preference Fraction: 30. Result ID: 698df35fffc3901de099ba6c. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"C"}. This record does not assert independent reproduction. ### Policy R1 Overall Preference Fraction: 29. Result ID: 698df35fffc3901de099ba7d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reference Policy":"C"}. This record does not assert independent reproduction. ### Policy A Overall Preference Fraction: 25. Result ID: 698df35fffc3901de099ba68. [Reporting paper]() Recorded evaluation configuration: {"Reference Policy":"C"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.