# Short-form Question Answering on PopQA Source: [Short-form Question Answering on PopQA]() Collection index: [llms.txt]() Data updated: 2026-07-09T07:52:02.089Z Evaluation of model performance on the PopQA dataset for Short-form Question Answering, measured using Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Short-form Question Answering | | Dataset (including any stated split) | PopQA | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 11 of 11 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | CO-LMLM-360M-FW | 50.6 | Not recorded | 6a4f2b3b32fb937314875e42 | | CO-LMLM-135M | 47.3 | Not recorded | 6a4f2b3b32fb937314875e42 | | CO-LMLM-360M | 46.5 | Not recorded | 6a4f2b3b32fb937314875e42 | | REL-LMLM-135M | 27.2 | Not recorded | 6a4f2b3b32fb937314875e42 | | REL-LMLM-360M | 26.6 | Not recorded | 6a4f2b3b32fb937314875e42 | | HF/SMOLLM2-1.7B\* | 24 | Not recorded | 6a4f2b3b32fb937314875e42 | | HF/SMOLLM2-360M\* | 22.7 | Not recorded | 6a4f2b3b32fb937314875e42 | | HF/SMOLLM2-135M\* | 18.9 | Not recorded | 6a4f2b3b32fb937314875e42 | | STANDARD-360M-FW | 18.9 | Not recorded | 6a4f2b3b32fb937314875e42 | | STANDARD-360M | 16.2 | Not recorded | 6a4f2b3b32fb937314875e42 | | STANDARD-135M | 15.2 | Not recorded | 6a4f2b3b32fb937314875e42 | ### CO-LMLM-360M-FW Accuracy: 50.6. Result ID: 6a4f531460b88bb339e550f0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"360M","Training Corpus":"FW"}. This record does not assert independent reproduction. ### CO-LMLM-135M Accuracy: 47.3. Result ID: 6a4f531460b88bb339e550ea. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"135M"}. This record does not assert independent reproduction. ### CO-LMLM-360M Accuracy: 46.5. Result ID: 6a4f531460b88bb339e550ef. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"360M"}. This record does not assert independent reproduction. ### REL-LMLM-135M Accuracy: 27.2. Result ID: 6a4f531460b88bb339e550e9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"135M"}. This record does not assert independent reproduction. ### REL-LMLM-360M Accuracy: 26.6. Result ID: 6a4f531460b88bb339e550ee. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"360M"}. This record does not assert independent reproduction. ### HF/SMOLLM2-1.7B\* Accuracy: 24. Result ID: 6a4f531460b88bb339e550f1. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"1.7B","Training Token Count":"11T"}. This record does not assert independent reproduction. ### HF/SMOLLM2-360M\* Accuracy: 22.7. Result ID: 6a4f531460b88bb339e550eb. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"360M","Training Token Count":"4T"}. This record does not assert independent reproduction. ### HF/SMOLLM2-135M\* Accuracy: 18.9. Result ID: 6a4f531460b88bb339e550e7. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"135M","Training Token Count":"2T"}. This record does not assert independent reproduction. ### STANDARD-360M-FW Accuracy: 18.9. Result ID: 6a4f531460b88bb339e550ed. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"360M","Training Corpus":"FW"}. This record does not assert independent reproduction. ### STANDARD-360M Accuracy: 16.2. Result ID: 6a4f531460b88bb339e550ec. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"360M"}. This record does not assert independent reproduction. ### STANDARD-135M Accuracy: 15.2. Result ID: 6a4f531460b88bb339e550e8. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"135M"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.