# Question Answering Calibration on OOD Evaluation (Squad, WQ, CWQ, MSQ, PopQA) Source: [Question Answering Calibration on OOD Evaluation (Squad, WQ, CWQ, MSQ, PopQA)]() Collection index: [llms.txt]() Data updated: 2026-03-05T06:57:19.024Z Evaluation of model calibration performance across a suite of Out-of-Distribution (OOD) Question Answering datasets, including Squad, WQ, CWQ, MSQ, and PopQA, reporting average calibration metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering Calibration | | Dataset (including any stated split) | OOD Evaluation (Squad, WQ, CWQ, MSQ, PopQA) | | Metric | Squad Calibration Score | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 11 of 11 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Squad Calibration Score](): Lower is better; unit: PERCENT. - [WQ Calibration Score](): Lower is better; unit: PERCENT. - [CWQ Calibration Score](): Lower is better; unit: PERCENT. - [MSQ Calibration Score](): Lower is better; unit: PERCENT. - [PopQA Calibration Score](): Lower is better; unit: PERCENT. - [Average Calibration Score (OOD)](): Lower is better; unit: PERCENT. ## Recorded results | Method | Squad Calibration Score | Result date | Report paper | | --- | --- | --- | --- | | Thermometer | 5 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | DACA | 9 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | EliCal(1k) | 10 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Consis-Lex | 14 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Cal-Only(1k) | 17 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Prob | 31 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Consis-Sem | 31 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Elicitation | 31 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Verbal-10 | 35 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | Verbal-0 | 42 | Not recorded | 68fa7c5b1e209131cea8dbc1 | | N-Prob | 46 | Not recorded | 68fa7c5b1e209131cea8dbc1 | ### Thermometer Squad Calibration Score: 5. Result ID: 69a928b36dbb7075b951f953. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### DACA Squad Calibration Score: 9. Result ID: 69a928b36dbb7075b951f954. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### EliCal(1k) Squad Calibration Score: 10. Result ID: 69a928b36dbb7075b951f957. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","LoRA rank":"8","LoRA alpha":"16","Batch size":"16"}. This record does not assert independent reproduction. ### Consis-Lex Squad Calibration Score: 14. Result ID: 69a928b36dbb7075b951f951. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Cal-Only(1k) Squad Calibration Score: 17. Result ID: 69a928b36dbb7075b951f956. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","LoRA rank":"8","LoRA alpha":"16","Batch size":"16"}. This record does not assert independent reproduction. ### Prob Squad Calibration Score: 31. Result ID: 69a928b36dbb7075b951f94d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Consis-Sem Squad Calibration Score: 31. Result ID: 69a928b36dbb7075b951f952. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Elicitation Squad Calibration Score: 31. Result ID: 69a928b36dbb7075b951f955. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","LoRA rank":"8","LoRA alpha":"16","Batch size":"16"}. This record does not assert independent reproduction. ### Verbal-10 Squad Calibration Score: 35. Result ID: 69a928b36dbb7075b951f950. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Verbal-0 Squad Calibration Score: 42. Result ID: 69a928b36dbb7075b951f94f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### N-Prob Squad Calibration Score: 46. Result ID: 69a928b36dbb7075b951f94e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.