Distractor Generation on MCQ (test)
22.39P@1Text2text (RAP-T5)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Text2text (RAP-T5)Backbone=T5, Methodology=RAP, Framework=Text2text2024.06 | 22.39 | 7.46 | 14.8 | 29.02 | 30.72 | |
| KAG (T5) (with only answer triplet)Backbone=T5, Methodology=KAG, Constraint=only answer triplet2024.06 | 22 | 7.35 | 15.18 | 28.7 | 30.61 | |
| KAG (RAP-T5) (with only answer triplet)Backbone=T5, Methodology=KAG + RAP, Constraint=only answer triplet2024.06 | 21.62 | 7.2 | 14.41 | 27.22 | 28.6 | |
| KAG (BART)Backbone=BART, Methodology=KAG2024.06 | 20.07 | 6.69 | 14.41 | 27.02 | 29.37 | |
| KAG (T5)Backbone=T5, Methodology=KAG2024.06 | 20.07 | 6.69 | 16.47 | 28.57 | 30.99 | |
| KAG (RAP-T5)Backbone=T5, Methodology=KAG + RAP2024.06 | 20.07 | 6.69 | 14.92 | 26.44 | 28.23 | |
| Wang et al., 2023 (BART w/ c.a.)Backbone=BART, candidate augmentation=true2024.06 | 19.69 | 6.56 | 13.12 | 25.03 | 26.26 | |
| ChatGPT2024.06 | 18.91 | 6.3 | 13.38 | 25.86 | 27.97 | |
| Wang et al., 2023 (T5)Backbone=T52024.06 | 18.53 | 6.17 | 11.45 | 23.61 | 25.08 | |
| Text2text (RAP-BART)Backbone=BART, Methodology=RAP, Framework=Text2text2024.06 | 18.14 | 6.04 | 12.35 | 24.06 | 25.78 | |
| KAG (RAP-BART)Backbone=BART, Methodology=KAG + RAP2024.06 | 16.6 | 5.75 | 12.22 | 22.65 | 24.5 | |
| Wang et al., 2023 (T5 w/ c.a.)Backbone=T5, candidate augmentation=true2024.06 | 16.6 | 5.53 | 14.8 | 24.9 | 27.61 | |
| Wang et al., 2023 (BART)Backbone=BART2024.06 | 14.28 | 4.76 | 11.45 | 21.49 | 23.7 | |
| KAG (BART) (with only answer triplet)Backbone=BART, Methodology=KAG, Constraint=only answer triplet2024.06 | 14.28 | 4.76 | 12.74 | 21.55 | 24.13 | |
| KAG (RAP-BART) (with only answer triplet)Backbone=BART, Methodology=KAG + RAP, Constraint=only answer triplet2024.06 | 13.12 | 4.37 | 11.84 | 20.01 | 22.35 | |
| Chiang et al., 20222024.06 | 10.81 | 3.6 | 7.72 | 18.15 | 15.39 | |
| Ren and Zhu, 20212024.06 | 10.58 | — | 9.19 | 17.51 | — |