# Uncertainty Estimation on SQW (SearchQA) Source: [Uncertainty Estimation on SQW (SearchQA)]() Collection index: [llms.txt]() Data updated: 2026-05-01T08:40:13.452Z Evaluation of model performance on the SQW (SearchQA) dataset focusing on Uncertainty Estimation metrics such as AUROC and AUPRC. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Uncertainty Estimation | | Dataset (including any stated split) | SQW (SearchQA) | | Metric | AUROC | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [AUROC](): Higher is better; unit: SCALAR. - [AUPRC](): Higher is better; unit: SCALAR. ## Recorded results | Method | AUROC | Result date | Report paper | | --- | --- | --- | --- | | Proposed | 0.83 | Not recorded | 69f4339755a3c32af89fd154 | | Geometry-Calibrated Conformal Abstention | 0.81 | Not recorded | 69f4339755a3c32af89fd154 | | SAR | 0.77 | Not recorded | 69f4339755a3c32af89fd154 | | Semantic | 0.77 | Not recorded | 69f4339755a3c32af89fd154 | | Perplexity | 0.76 | Not recorded | 69f4339755a3c32af89fd154 | | Eigen | 0.76 | Not recorded | 69f4339755a3c32af89fd154 | | SAR | 0.74 | Not recorded | 69f4339755a3c32af89fd154 | | Semantic | 0.74 | Not recorded | 69f4339755a3c32af89fd154 | | ATRMD | 0.71 | Not recorded | 69f4339755a3c32af89fd154 | | Perplexity | 0.7 | Not recorded | 69f4339755a3c32af89fd154 | | Eigen | 0.7 | Not recorded | 69f4339755a3c32af89fd154 | | Focus | 0.55 | Not recorded | 69f4339755a3c32af89fd154 | | ATRMD | 0.53 | Not recorded | 69f4339755a3c32af89fd154 | | P(True) | 0.49 | Not recorded | 69f4339755a3c32af89fd154 | | Focus | 0.49 | Not recorded | 69f4339755a3c32af89fd154 | | Attention | 0.47 | Not recorded | 69f4339755a3c32af89fd154 | | P(True) | 0.47 | Not recorded | 69f4339755a3c32af89fd154 | | Attention | 0.45 | Not recorded | 69f4339755a3c32af89fd154 | ### Proposed AUROC: 0.83. Result ID: 69f466de47f34ce924da6b33. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA-3-8B-Instruct"}. This record does not assert independent reproduction. ### Geometry-Calibrated Conformal Abstention AUROC: 0.81. Result ID: 69f466de47f34ce924da6afd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B-Instruct"}. This record does not assert independent reproduction. ### SAR AUROC: 0.77. Result ID: 69f466de47f34ce924da6af6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B-Instruct"}. This record does not assert independent reproduction. ### Semantic AUROC: 0.77. Result ID: 69f466de47f34ce924da6af7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B-Instruct"}. This record does not assert independent reproduction. ### Perplexity AUROC: 0.76. Result ID: 69f466de47f34ce924da6af5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B-Instruct"}. This record does not assert independent reproduction. ### Eigen AUROC: 0.76. Result ID: 69f466de47f34ce924da6afa. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B-Instruct"}. This record does not assert independent reproduction. ### SAR AUROC: 0.74. Result ID: 69f466de47f34ce924da6b2c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA-3-8B-Instruct"}. This record does not assert independent reproduction. ### Semantic AUROC: 0.74. Result ID: 69f466de47f34ce924da6b2d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA-3-8B-Instruct"}. This record does not assert independent reproduction. ### ATRMD AUROC: 0.71. Result ID: 69f466de47f34ce924da6b31. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA-3-8B-Instruct"}. This record does not assert independent reproduction. ### Perplexity AUROC: 0.7. Result ID: 69f466de47f34ce924da6b2b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA-3-8B-Instruct"}. This record does not assert independent reproduction. ### Eigen AUROC: 0.7. Result ID: 69f466de47f34ce924da6b30. [Reporting paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA-3-8B-Instruct"}. This record does not assert independent reproduction. ### Focus AUROC: 0.55. Result ID: 69f466de47f34ce924da6af8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B-Instruct"}. This record does not assert independent reproduction. ### ATRMD AUROC: 0.53. Result ID: 69f466de47f34ce924da6afb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B-Instruct"}. This record does not assert independent reproduction. ### P(True) AUROC: 0.49. Result ID: 69f466de47f34ce924da6afc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B-Instruct"}. This record does not assert independent reproduction. ### Focus AUROC: 0.49. Result ID: 69f466de47f34ce924da6b2e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA-3-8B-Instruct"}. This record does not assert independent reproduction. ### Attention AUROC: 0.47. Result ID: 69f466de47f34ce924da6b2f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA-3-8B-Instruct"}. This record does not assert independent reproduction. ### P(True) AUROC: 0.47. Result ID: 69f466de47f34ce924da6b32. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA-3-8B-Instruct"}. This record does not assert independent reproduction. ### Attention AUROC: 0.45. Result ID: 69f466de47f34ce924da6af9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.