# Short Answer Question Answering on Natural Questions (NQ) (test) Source: [Short Answer Question Answering on Natural Questions (NQ) (test)]() Collection index: [llms.txt]() Data updated: 2026-06-12T07:24:43.301Z Evaluation of model performance on the test split of the Natural Questions (NQ) dataset, focusing on Short Answer Question Answering using Precision, Recall, and F1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Short Answer Question Answering | | Dataset (including any stated split) | Natural Questions (NQ) (test) | | Metric | F1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 Score](): Higher is better; unit: PERCENT. - [Precision](): Higher is better; unit: PERCENT. - [Recall](): Higher is better; unit: PERCENT. - [Rank](): Lower is better; unit: SCALAR. ## Recorded results | Method | F1 Score | Result date | Report paper | | --- | --- | --- | --- | | AGAR | 67.4 | Not recorded | 6a2b92b7d33dfb23e783b14d | | RikiNet | 61.3 | Not recorded | 68fa6a211e209131cea262e2 | | Cluster-Former | 60.9 | Not recorded | 68fa6a211e209131cea262e2 | | VEA-Con | 59.9 | Not recorded | 6a2b92b7d33dfb23e783b14d | | Uniform | 59.6 | Not recorded | 6a2b92b7d33dfb23e783b14d | | Random | 59.3 | Not recorded | 6a2b92b7d33dfb23e783b14d | | Plain VQA | 59.1 | Not recorded | 6a2b92b7d33dfb23e783b14d | | VEA-Br | 59 | Not recorded | 6a2b92b7d33dfb23e783b14d | | BigBird-ETC-large | 57.9 | Not recorded | 68fa6a211e209131cea262e2 | | ETC | 57.86 | Not recorded | 68fa689c1e209131cea1d06b | | AGLA | 53 | Not recorded | 6a2b92b7d33dfb23e783b14d | | VAR | 52.4 | Not recorded | 6a2b92b7d33dfb23e783b14d | | BERT joint | 52.1 | Not recorded | 68fa66ce1e209131cea127dc | | DocumentQA | 35.1 | Not recorded | 68fa66ce1e209131cea127dc | | CGR | 33.6 | Not recorded | 6a2b92b7d33dfb23e783b14d | | DecAtt + DocReader | 31.5 | Not recorded | 68fa66ce1e209131cea127dc | ### AGAR F1 Score: 67.4. Result ID: 6a2bb42ffbc373592c29b149. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"InternVL3.5-8B"}. This record does not assert independent reproduction. ### RikiNet F1 Score: 61.3. Result ID: 698e12edffc3901de09a9646. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Cluster-Former F1 Score: 60.9. Result ID: 698e12edffc3901de09a9647. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### VEA-Con F1 Score: 59.9. Result ID: 6a2bb42ffbc373592c29b145. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"InternVL3.5-8B"}. This record does not assert independent reproduction. ### Uniform F1 Score: 59.6. Result ID: 6a2bb42ffbc373592c29b143. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"InternVL3.5-8B"}. This record does not assert independent reproduction. ### Random F1 Score: 59.3. Result ID: 6a2bb42ffbc373592c29b142. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"InternVL3.5-8B"}. This record does not assert independent reproduction. ### Plain VQA F1 Score: 59.1. Result ID: 6a2bb42ffbc373592c29b141. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"InternVL3.5-8B"}. This record does not assert independent reproduction. ### VEA-Br F1 Score: 59. Result ID: 6a2bb42ffbc373592c29b144. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"InternVL3.5-8B"}. This record does not assert independent reproduction. ### BigBird-ETC-large F1 Score: 57.9. Result ID: 698e12edffc3901de09a9645. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### ETC F1 Score: 57.86. Result ID: 698d78a68b517eb43321be7e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### AGLA F1 Score: 53. Result ID: 6a2bb42ffbc373592c29b148. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"InternVL3.5-8B"}. This record does not assert independent reproduction. ### VAR F1 Score: 52.4. Result ID: 6a2bb42ffbc373592c29b147. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"InternVL3.5-8B"}. This record does not assert independent reproduction. ### BERT joint F1 Score: 52.1. Result ID: 698d092a8b517eb4331e8671. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"pre-training":"SQuAD 1.1 fine-tuned"}. This record does not assert independent reproduction. ### DocumentQA F1 Score: 35.1. Result ID: 698d092a8b517eb4331e866f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### CGR F1 Score: 33.6. Result ID: 6a2bb42ffbc373592c29b146. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"InternVL3.5-8B"}. This record does not assert independent reproduction. ### DecAtt + DocReader F1 Score: 31.5. Result ID: 698d092a8b517eb4331e8670. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.