# Question Answering on ARC Challenge (Mean Improvement and Statistical Significance) Source: [Question Answering on ARC Challenge (Mean Improvement and Statistical Significance)]() Collection index: [llms.txt]() Data updated: 2026-05-18T08:38:17.211Z Evaluation of model performance on the ARC Challenge dataset for Question Answering, focusing on statistical significance metrics such as mean improvement, 95% confidence intervals, and p-values. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering | | Dataset (including any stated split) | ARC Challenge | | Metric | Mean Improvement | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 9 of 9 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Mean Improvement](): Higher is better; unit: SCALAR. - [95% Confidence Interval](): Higher is better; unit: SCALAR. - [P-Value](): Lower is better; unit: SCALAR. ## Recorded results | Method | Mean Improvement | Result date | Report paper | | --- | --- | --- | --- | | PASf | 0.066 | Not recorded | 68fa7bc71e209131cea8a68d | | iPASa | 0.051 | Not recorded | 68fa7bc71e209131cea8a68d | | iPASwo | 0.039 | Not recorded | 68fa7bc71e209131cea8a68d | | iPASa | 0 | Not recorded | 68fa7bc71e209131cea8a68d | | PASf | 0 | Not recorded | 68fa7bc71e209131cea8a68d | | iPASwo | -0.003 | Not recorded | 68fa7bc71e209131cea8a68d | | PASf | -0.004 | Not recorded | 68fa7bc71e209131cea8a68d | | iPASa | -0.005 | Not recorded | 68fa7bc71e209131cea8a68d | | iPASwo | -0.01 | Not recorded | 68fa7bc71e209131cea8a68d | ### PASf Mean Improvement: 0.066. Result ID: 6a0acfd547f34ce924f0f901. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek-R1-Distill-8B"}. This record does not assert independent reproduction. ### iPASa Mean Improvement: 0.051. Result ID: 6a0acfd547f34ce924f0f8ff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek-R1-Distill-8B"}. This record does not assert independent reproduction. ### iPASwo Mean Improvement: 0.039. Result ID: 6a0acfd547f34ce924f0f900. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek-R1-Distill-8B"}. This record does not assert independent reproduction. ### iPASa Mean Improvement: 0. Result ID: 6a0acfd547f34ce924f0f902. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### PASf Mean Improvement: 0. Result ID: 6a0acfd547f34ce924f0f904. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### iPASwo Mean Improvement: -0.003. Result ID: 6a0acfd547f34ce924f0f8fd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Nous-Hermes-2-Mistral-7B-DPO"}. This record does not assert independent reproduction. ### PASf Mean Improvement: -0.004. Result ID: 6a0acfd547f34ce924f0f8fe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Nous-Hermes-2-Mistral-7B-DPO"}. This record does not assert independent reproduction. ### iPASa Mean Improvement: -0.005. Result ID: 6a0acfd547f34ce924f0f8fc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Nous-Hermes-2-Mistral-7B-DPO"}. This record does not assert independent reproduction. ### iPASwo Mean Improvement: -0.01. Result ID: 6a0acfd547f34ce924f0f903. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.