# Honesty Alignment on Natural Questions (NQ) In-Domain Source: [Honesty Alignment on Natural Questions (NQ) In-Domain]() Collection index: [llms.txt]() Data updated: 2026-03-05T06:57:19.132Z Evaluation of model performance on the In-Domain subset of the Natural Questions (NQ) dataset, specifically measuring Honesty Alignment using AUROC. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Honesty Alignment | | Dataset (including any stated split) | Natural Questions (NQ) In-Domain | | Metric | AUROC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 33 of 33 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [AUROC](): Higher is better; unit: PERCENT. ## Recorded results | Method | AUROC | Result date | Report paper | | --- | --- | --- | --- | | EliCal | 85.16 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Cal-Only | 84.89 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | EliCal | 84.57 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Cal-Only | 83.95 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | EliCal | 82.38 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Consis-Sem | 80.68 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Consis-Sem | 80.5 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | EliCal | 80.46 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | EliCal | 79.22 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Cal-Only | 78.97 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Eli-Only | 77.86 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Consis-Sem | 77.77 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Eli-Only | 76.92 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | EliCal | 74.86 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Eli-Only | 74.21 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Cal-Only | 72.19 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Cal-Only | 69.62 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Verbal-10 | 68.82 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Consis-Lex | 68.65 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Cal-Only | 68.48 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | N-Prob | 66.11 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | N-Prob | 65.83 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Verbal-10 | 65.7 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Verbal-0 | 65.33 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Consis-Lex | 65.32 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Consis-Lex | 65.02 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | N-Prob | 64.25 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Verbal-0 | 64.02 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Verbal-0 | 61.72 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Prob | 61.44 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Prob | 56.79 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Verbal-10 | 56.08 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Prob | 55.53 | Not recorded | 68fa7c5b1e209131cea8dbc1 | ### EliCal AUROC: 85.16. Result ID: 69a928b36dbb7075b951f97c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Upper Bound","Sample Size":"560k"}. This record does not assert independent reproduction. ### Cal-Only AUROC: 84.89. Result ID: 69a928b36dbb7075b951f97b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Upper Bound","Sample Size":"560k"}. This record does not assert independent reproduction. ### EliCal AUROC: 84.57. Result ID: 69a928b36dbb7075b951f987. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Upper Bound","Sample Size":"560k"}. This record does not assert independent reproduction. ### Cal-Only AUROC: 83.95. Result ID: 69a928b36dbb7075b951f986. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Upper Bound","Sample Size":"560k"}. This record does not assert independent reproduction. ### EliCal AUROC: 82.38. Result ID: 69a928b36dbb7075b951f97a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Training-based","Sample Size":"1k"}. This record does not assert independent reproduction. ### Consis-Sem AUROC: 80.68. Result ID: 69a928b36dbb7075b951f977. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Consis-Sem AUROC: 80.5. Result ID: 69a928b36dbb7075b951f98d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Training-free"}. This record does not assert independent reproduction. ### EliCal AUROC: 80.46. Result ID: 69a928b36dbb7075b951f985. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Training-based","Sample Size":"1k"}. This record does not assert independent reproduction. ### EliCal AUROC: 79.22. Result ID: 69a928b36dbb7075b951f992. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Upper Bound","Sample Size":"560k"}. This record does not assert independent reproduction. ### Cal-Only AUROC: 78.97. Result ID: 69a928b36dbb7075b951f991. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Upper Bound","Sample Size":"560k"}. This record does not assert independent reproduction. ### Eli-Only AUROC: 77.86. Result ID: 69a928b36dbb7075b951f978. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Training-based"}. This record does not assert independent reproduction. ### Consis-Sem AUROC: 77.77. Result ID: 69a928b36dbb7075b951f982. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Eli-Only AUROC: 76.92. Result ID: 69a928b36dbb7075b951f983. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Training-based"}. This record does not assert independent reproduction. ### EliCal AUROC: 74.86. Result ID: 69a928b36dbb7075b951f990. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Training-based","Sample Size":"1k"}. This record does not assert independent reproduction. ### Eli-Only AUROC: 74.21. Result ID: 69a928b36dbb7075b951f98e. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Training-based"}. This record does not assert independent reproduction. ### Cal-Only AUROC: 72.19. Result ID: 69a928b36dbb7075b951f979. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Training-based","Sample Size":"1k"}. This record does not assert independent reproduction. ### Cal-Only AUROC: 69.62. Result ID: 69a928b36dbb7075b951f984. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Training-based","Sample Size":"1k"}. This record does not assert independent reproduction. ### Verbal-10 AUROC: 68.82. Result ID: 69a928b36dbb7075b951f975. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Consis-Lex AUROC: 68.65. Result ID: 69a928b36dbb7075b951f981. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Cal-Only AUROC: 68.48. Result ID: 69a928b36dbb7075b951f98f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Training-based","Sample Size":"1k"}. This record does not assert independent reproduction. ### N-Prob AUROC: 66.11. Result ID: 69a928b36dbb7075b951f973. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Training-free"}. This record does not assert independent reproduction. ### N-Prob AUROC: 65.83. Result ID: 69a928b36dbb7075b951f97e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Verbal-10 AUROC: 65.7. Result ID: 69a928b36dbb7075b951f980. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Verbal-0 AUROC: 65.33. Result ID: 69a928b36dbb7075b951f97f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Consis-Lex AUROC: 65.32. Result ID: 69a928b36dbb7075b951f98c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Consis-Lex AUROC: 65.02. Result ID: 69a928b36dbb7075b951f976. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Training-free"}. This record does not assert independent reproduction. ### N-Prob AUROC: 64.25. Result ID: 69a928b36dbb7075b951f989. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Verbal-0 AUROC: 64.02. Result ID: 69a928b36dbb7075b951f974. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Verbal-0 AUROC: 61.72. Result ID: 69a928b36dbb7075b951f98a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Prob AUROC: 61.44. Result ID: 69a928b36dbb7075b951f97d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-14B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Prob AUROC: 56.79. Result ID: 69a928b36dbb7075b951f972. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-7B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Verbal-10 AUROC: 56.08. Result ID: 69a928b36dbb7075b951f98b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Training-free"}. This record does not assert independent reproduction. ### Prob AUROC: 55.53. Result ID: 69a928b36dbb7075b951f988. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-8B","Regime":"Training-free"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.