# Question Answering on HotpotQA (test) (ACC, AUROC, Brier, ECE Metrics) Source: [Question Answering on HotpotQA (test) (ACC, AUROC, Brier, ECE Metrics)]() Collection index: [llms.txt]() Data updated: 2026-04-02T09:28:57.780Z Evaluation of model performance on the test split of the HotpotQA dataset for Question Answering, focusing on Accuracy, AUROC, Brier score, and Expected Calibration Error (ECE). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering | | Dataset (including any stated split) | HotpotQA (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [AUROC](): Higher is better; unit: PERCENT. - [Brier Score](): Lower is better; unit: PERCENT. - [ECE](): Lower is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | GPT-4.1 | 68.6 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-4.1 + Verbalized Distrib. | 68.2 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-5-mini + Verbalized Conf. | 68.2 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-4.1 + Verbalized Conf. | 67.9 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-4.1 + Verbalized Top-k | 67.3 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-5-mini | 66.9 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-5-mini + Verbalized Probability Distribution | 66.8 | Not recorded | 691d4f16d6b0dd334292d799 | | GPT-5-mini + Verbalized Top-k | 65.9 | Not recorded | 691d4f16d6b0dd334292d799 | | DeepSeek-V3 + Verbalized Distrib. | 59.4 | Not recorded | 691d4f16d6b0dd334292d799 | | DeepSeek-V3 + Verbalized Conf. | 59.1 | Not recorded | 691d4f16d6b0dd334292d799 | | DeepSeek-V3 | 58.3 | Not recorded | 691d4f16d6b0dd334292d799 | | DeepSeek-V3 + Verbalized Top-k | 57.2 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + Verbalized Top-k | 43.9 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + Verbalized Distrib. | 43.8 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct | 43.7 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + Logit | 43.7 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + p(True) | 43.7 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-30B-A3B-Instruct + Verbalized Conf. | 42.6 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + Verbalized Distrib. | 33.6 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Thinking + Verbalized Top-k | 33 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + Verbalized Top-k | 31.9 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Thinking + Verbalized Conf. | 31.7 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Thinking + Verbalized Probability Distribution | 31.7 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct | 30.3 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + Logit | 30.3 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + p(True) | 30.3 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Thinking | 29.9 | Not recorded | 691d4f16d6b0dd334292d799 | | Qwen3-4B-Instruct + Verbalized Conf. | 29 | Not recorded | 691d4f16d6b0dd334292d799 | ### GPT-4.1 Accuracy: 68.6. Result ID: 69ce36d647f34ce924b7b062. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1","Inference Strategy":"Base"}. This record does not assert independent reproduction. ### GPT-4.1 + Verbalized Distrib. Accuracy: 68.2. Result ID: 69ce36d647f34ce924b7b065. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1","Inference Strategy":"+ Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### GPT-5-mini + Verbalized Conf. Accuracy: 68.2. Result ID: 69ce36d647f34ce924b7b08b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"GPT-5-mini","Confidence Estimation Strategy":"Verbalized Conf."}. This record does not assert independent reproduction. ### GPT-4.1 + Verbalized Conf. Accuracy: 67.9. Result ID: 69ce36d647f34ce924b7b063. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1","Inference Strategy":"+ Verbalized Conf."}. This record does not assert independent reproduction. ### GPT-4.1 + Verbalized Top-k Accuracy: 67.3. Result ID: 69ce36d647f34ce924b7b064. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1","Inference Strategy":"+ Verbalized Top-k"}. This record does not assert independent reproduction. ### GPT-5-mini Accuracy: 66.9. Result ID: 69ce36d647f34ce924b7b08a. [Reporting paper]() Recorded evaluation configuration: {"Backbone Model":"GPT-5-mini","Confidence Estimation Strategy":"Base"}. This record does not assert independent reproduction. ### GPT-5-mini + Verbalized Probability Distribution Accuracy: 66.8. Result ID: 69ce36d647f34ce924b7b08d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"GPT-5-mini","Confidence Estimation Strategy":"Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### GPT-5-mini + Verbalized Top-k Accuracy: 65.9. Result ID: 69ce36d647f34ce924b7b08c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"GPT-5-mini","Confidence Estimation Strategy":"Verbalized Top-k"}. This record does not assert independent reproduction. ### DeepSeek-V3 + Verbalized Distrib. Accuracy: 59.4. Result ID: 69ce36d647f34ce924b7b069. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3","Inference Strategy":"+ Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### DeepSeek-V3 + Verbalized Conf. Accuracy: 59.1. Result ID: 69ce36d647f34ce924b7b067. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3","Inference Strategy":"+ Verbalized Conf."}. This record does not assert independent reproduction. ### DeepSeek-V3 Accuracy: 58.3. Result ID: 69ce36d647f34ce924b7b066. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3","Inference Strategy":"Base"}. This record does not assert independent reproduction. ### DeepSeek-V3 + Verbalized Top-k Accuracy: 57.2. Result ID: 69ce36d647f34ce924b7b068. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3","Inference Strategy":"+ Verbalized Top-k"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + Verbalized Top-k Accuracy: 43.9. Result ID: 69ce36d647f34ce924b7b060. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ Verbalized Top-k"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + Verbalized Distrib. Accuracy: 43.8. Result ID: 69ce36d647f34ce924b7b061. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct Accuracy: 43.7. Result ID: 69ce36d647f34ce924b7b05c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"Base"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + Logit Accuracy: 43.7. Result ID: 69ce36d647f34ce924b7b05d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ Logit"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + p(True) Accuracy: 43.7. Result ID: 69ce36d647f34ce924b7b05e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ p(True)"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct + Verbalized Conf. Accuracy: 42.6. Result ID: 69ce36d647f34ce924b7b05f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B-Instruct","Inference Strategy":"+ Verbalized Conf."}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Verbalized Distrib. Accuracy: 33.6. Result ID: 69ce36d647f34ce924b7b05b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking + Verbalized Top-k Accuracy: 33. Result ID: 69ce36d647f34ce924b7b088. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen3-4B-Thinking","Confidence Estimation Strategy":"Verbalized Top-k"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Verbalized Top-k Accuracy: 31.9. Result ID: 69ce36d647f34ce924b7b05a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ Verbalized Top-k"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking + Verbalized Conf. Accuracy: 31.7. Result ID: 69ce36d647f34ce924b7b087. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen3-4B-Thinking","Confidence Estimation Strategy":"Verbalized Conf."}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking + Verbalized Probability Distribution Accuracy: 31.7. Result ID: 69ce36d647f34ce924b7b089. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen3-4B-Thinking","Confidence Estimation Strategy":"Verbalized Probability Distribution"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Accuracy: 30.3. Result ID: 69ce36d647f34ce924b7b056. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"Base"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Logit Accuracy: 30.3. Result ID: 69ce36d647f34ce924b7b057. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ Logit"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + p(True) Accuracy: 30.3. Result ID: 69ce36d647f34ce924b7b058. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ p(True)"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking Accuracy: 29.9. Result ID: 69ce36d647f34ce924b7b086. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen3-4B-Thinking","Confidence Estimation Strategy":"Base"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct + Verbalized Conf. Accuracy: 29. Result ID: 69ce36d647f34ce924b7b059. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct","Inference Strategy":"+ Verbalized Conf."}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.