Distractor Generation on Sciq (test)
24.3Precision@1Text2text (RAP-T5)
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Text2text (RAP-T5)Backbone=T5, Methodology=RAP2024.06 | 24.3 | 8.09 | 16.06 | 29.98 | 31.49 | |
| Wang et al., 2023 Text2text (T5)Backbone=T52024.06 | 24.27 | 8.09 | 15.92 | 30.32 | 31.99 | |
| KAG (T5) (with only answer triplet)Backbone=T5, Methodology=KAG, Constraint=only answer triplet2024.06 | 23.7 | 7.91 | 16.5 | 30.41 | 32.39 | |
| KAG (RAP-T5) (with only answer triplet)Backbone=T5, Methodology=KAG + RAP, Constraint=only answer triplet2024.06 | 23 | 7.68 | 16.46 | 29.51 | 31.41 | |
| KAG (T5)Backbone=T5, Methodology=KAG2024.06 | 22.5 | 7.51 | 15.38 | 29.1 | 31.04 | |
| Wang et al., 2023 (T5 w/ c.a.)Backbone=T5, candidate augmentation=true2024.06 | 22.06 | 7.35 | 12.83 | 25.54 | 26.47 | |
| Wang et al., 2023 (BART w/ c.a.)Backbone=BART, candidate augmentation=true2024.06 | 21.26 | 7.08 | 14.98 | 27.63 | 29.28 | |
| Text2text (RAP-BART)Backbone=BART, Methodology=RAP2024.06 | 20.1 | 6.73 | 16.32 | 27.56 | 29.68 | |
| KAG (RAP-BART)Backbone=BART, Methodology=KAG + RAP2024.06 | 19.7 | 6.58 | 15.42 | 26.61 | 28.69 | |
| Wang et al., 2023 (BART)Backbone=BART2024.06 | 19.3 | 6.46 | 14.94 | 26.4 | 28.36 | |
| KAG (RAP-T5)Backbone=T5, Methodology=KAG + RAP2024.06 | 18.5 | 6.18 | 15.15 | 26 | 28.4 | |
| KAG (RAP-BART) (with only answer triplet)Backbone=BART, Methodology=KAG + RAP, Constraint=only answer triplet2024.06 | 18.35 | 6.15 | 15.52 | 25.81 | 28.22 | |
| KAG (BART) (with only answer triplet)Backbone=BART, Methodology=KAG, Constraint=only answer triplet2024.06 | 17.15 | 5.57 | 14.87 | 25 | 27.27 | |
| KAG (BART)Backbone=BART, Methodology=KAG2024.06 | 16.24 | 5.41 | 13.99 | 23.62 | 25.93 | |
| ChatGPT2024.06 | 15.17 | 5.16 | 10.61 | 19.39 | 20.68 |