# End-to-end Generation on NQ (test) Source: [End-to-end Generation on NQ (test)]() Collection index: [llms.txt]() Data updated: 2026-05-07T08:27:49.964Z Evaluation of model performance on the test split of the Natural Questions (NQ) dataset using end-to-end generation metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | End-to-end Generation | | Dataset (including any stated split) | NQ (test) | | Metric | F1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 Score](): Higher is better; unit: PERCENT. - [Percentage Change](): Higher is better; unit: PERCENT. ## Recorded results | Method | F1 Score | Result date | Report paper | | --- | --- | --- | --- | | RankT5 | 15.818 | Not recorded | 69fc1cac5497b3b670c88615 | | QLM | 15.741 | Not recorded | 69fc1cac5497b3b670c88615 | | RankT5 | 15.735 | Not recorded | 69fc1cac5497b3b670c88615 | | ColBERT | 15.733 | Not recorded | 69fc1cac5497b3b670c88615 | | Cross-Encoder | 15.724 | Not recorded | 69fc1cac5497b3b670c88615 | | Cross-Encoder | 15.681 | Not recorded | 69fc1cac5497b3b670c88615 | | ColBERT | 15.635 | Not recorded | 69fc1cac5497b3b670c88615 | | QLM | 15.462 | Not recorded | 69fc1cac5497b3b670c88615 | | RankGPT | 15.291 | Not recorded | 69fc1cac5497b3b670c88615 | | BM25 | 15.228 | Not recorded | 69fc1cac5497b3b670c88615 | | YesNo | 15.192 | Not recorded | 69fc1cac5497b3b670c88615 | | RankGPT | 13.886 | Not recorded | 69fc1cac5497b3b670c88615 | | BM25 | 13.789 | Not recorded | 69fc1cac5497b3b670c88615 | | YesNo | 12.974 | Not recorded | 69fc1cac5497b3b670c88615 | ### RankT5 F1 Score: 15.818. Result ID: 69fc4cf947f34ce924deff97. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"true","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### QLM F1 Score: 15.741. Result ID: 69fc4cf947f34ce924deff8f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"true","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### RankT5 F1 Score: 15.735. Result ID: 69fc4cf947f34ce924deff96. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"false","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### ColBERT F1 Score: 15.733. Result ID: 69fc4cf947f34ce924deff93. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"true","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### Cross-Encoder F1 Score: 15.724. Result ID: 69fc4cf947f34ce924deff95. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"true","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### Cross-Encoder F1 Score: 15.681. Result ID: 69fc4cf947f34ce924deff94. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"false","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### ColBERT F1 Score: 15.635. Result ID: 69fc4cf947f34ce924deff92. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"false","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### QLM F1 Score: 15.462. Result ID: 69fc4cf947f34ce924deff8e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"false","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### RankGPT F1 Score: 15.291. Result ID: 69fc4cf947f34ce924deff91. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"true","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### BM25 F1 Score: 15.228. Result ID: 69fc4cf947f34ce924deff8b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"true","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### YesNo F1 Score: 15.192. Result ID: 69fc4cf947f34ce924deff8d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"true","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### RankGPT F1 Score: 13.886. Result ID: 69fc4cf947f34ce924deff90. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"false","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### BM25 F1 Score: 13.789. Result ID: 69fc4cf947f34ce924deff8a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"false","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. ### YesNo F1 Score: 12.974. Result ID: 69fc4cf947f34ce924deff8c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"CAR":"false","Generator":"Qwen2.5-7B-Instruct","Initial Retriever":"BM25"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.