# Distractor Generation on MCQ (test) Source: [Distractor Generation on MCQ (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:54:45.503Z Evaluation of Distractor Generation performance on the test split of the MCQ dataset, measured using ranking and retrieval metrics (P@1, R@1, F1@3, MRR, NDCG@3). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Distractor Generation | | Dataset (including any stated split) | MCQ (test) | | Metric | P@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [P@1](): Higher is better; unit: PERCENT. - [R@1](): Higher is better; unit: PERCENT. - [F1@3](): Higher is better; unit: PERCENT. - [MRR](): Higher is better; unit: PERCENT. - [NDCG@3](): Higher is better; unit: PERCENT. ## Recorded results | Method | P@1 | Result date | Report paper | | --- | --- | --- | --- | | Text2text (RAP-T5) | 22.39 | Not recorded | 68fa73be1e209131cea5e347 | | KAG (T5) (with only answer triplet) | 22 | Not recorded | 68fa73be1e209131cea5e347 | | KAG (RAP-T5) (with only answer triplet) | 21.62 | Not recorded | 68fa73be1e209131cea5e347 | | KAG (BART) | 20.07 | Not recorded | 68fa73be1e209131cea5e347 | | KAG (T5) | 20.07 | Not recorded | 68fa73be1e209131cea5e347 | | KAG (RAP-T5) | 20.07 | Not recorded | 68fa73be1e209131cea5e347 | | Wang et al., 2023 (BART w/ c.a.) | 19.69 | Not recorded | 68fa73be1e209131cea5e347 | | ChatGPT | 18.91 | Not recorded | 68fa73be1e209131cea5e347 | | Wang et al., 2023 (T5) | 18.53 | Not recorded | 68fa73be1e209131cea5e347 | | Text2text (RAP-BART) | 18.14 | Not recorded | 68fa73be1e209131cea5e347 | | KAG (RAP-BART) | 16.6 | Not recorded | 68fa73be1e209131cea5e347 | | Wang et al., 2023 (T5 w/ c.a.) | 16.6 | Not recorded | 68fa73be1e209131cea5e347 | | Wang et al., 2023 (BART) | 14.28 | Not recorded | 68fa73be1e209131cea5e347 | | KAG (BART) (with only answer triplet) | 14.28 | Not recorded | 68fa73be1e209131cea5e347 | | KAG (RAP-BART) (with only answer triplet) | 13.12 | Not recorded | 68fa73be1e209131cea5e347 | | Chiang et al., 2022 | 10.81 | Not recorded | 68fa73be1e209131cea5e347 | | Ren and Zhu, 2021 | 10.58 | Not recorded | 68fa73be1e209131cea5e347 | ### Text2text (RAP-T5) P@1: 22.39. Result ID: 69931b233a6c31484913c3ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"T5","Methodology":"RAP","Framework":"Text2text"}. This record does not assert independent reproduction. ### KAG (T5) (with only answer triplet) P@1: 22. Result ID: 69931b233a6c31484913c3b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"T5","Methodology":"KAG","Constraint":"only answer triplet"}. This record does not assert independent reproduction. ### KAG (RAP-T5) (with only answer triplet) P@1: 21.62. Result ID: 69931b233a6c31484913c3b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"T5","Methodology":"KAG + RAP","Constraint":"only answer triplet"}. This record does not assert independent reproduction. ### KAG (BART) P@1: 20.07. Result ID: 69931b233a6c31484913c3a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BART","Methodology":"KAG"}. This record does not assert independent reproduction. ### KAG (T5) P@1: 20.07. Result ID: 69931b233a6c31484913c3af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"T5","Methodology":"KAG"}. This record does not assert independent reproduction. ### KAG (RAP-T5) P@1: 20.07. Result ID: 69931b233a6c31484913c3b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"T5","Methodology":"KAG + RAP"}. This record does not assert independent reproduction. ### Wang et al., 2023 (BART w/ c.a.) P@1: 19.69. Result ID: 69931b233a6c31484913c3a7. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"BART","candidate augmentation":true}. This record does not assert independent reproduction. ### ChatGPT P@1: 18.91. Result ID: 69931b233a6c31484913c3b3. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Wang et al., 2023 (T5) P@1: 18.53. Result ID: 69931b233a6c31484913c3ac. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"T5"}. This record does not assert independent reproduction. ### Text2text (RAP-BART) P@1: 18.14. Result ID: 69931b233a6c31484913c3a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BART","Methodology":"RAP","Framework":"Text2text"}. This record does not assert independent reproduction. ### KAG (RAP-BART) P@1: 16.6. Result ID: 69931b233a6c31484913c3a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BART","Methodology":"KAG + RAP"}. This record does not assert independent reproduction. ### Wang et al., 2023 (T5 w/ c.a.) P@1: 16.6. Result ID: 69931b233a6c31484913c3ae. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"T5","candidate augmentation":true}. This record does not assert independent reproduction. ### Wang et al., 2023 (BART) P@1: 14.28. Result ID: 69931b233a6c31484913c3a5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"BART"}. This record does not assert independent reproduction. ### KAG (BART) (with only answer triplet) P@1: 14.28. Result ID: 69931b233a6c31484913c3aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BART","Methodology":"KAG","Constraint":"only answer triplet"}. This record does not assert independent reproduction. ### KAG (RAP-BART) (with only answer triplet) P@1: 13.12. Result ID: 69931b233a6c31484913c3ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BART","Methodology":"KAG + RAP","Constraint":"only answer triplet"}. This record does not assert independent reproduction. ### Chiang et al., 2022 P@1: 10.81. Result ID: 69931b233a6c31484913c3a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Ren and Zhu, 2021 P@1: 10.58. Result ID: 69931b233a6c31484913c3a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.