Question Generation on SQuAD (test)
44.61BLEU-1partial copy + QA-based reranking
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| partial copy + QA-based rerankinglambda_1=1, lambda_2=0.22019.10 | 44.61 | 20.13 | — | — | 15.29 | 28.78 | 20.59 | — | — | — | — | — | |
| partial copylambda_1=12019.10 | 44.33 | 20 | — | — | 14.95 | 28.34 | 20.17 | — | — | — | — | — | |
| partial copylambda_1=0.52019.10 | 44.13 | 19.81 | — | — | 14.95 | 28.29 | 20.17 | — | — | — | — | — | |
| Du et al. (2017)2019.10 | 43.09 | 16.62 | — | — | 12.28 | 25.96 | 17.5 | — | — | — | — | — | |
| QA-based rerankinglambda_2=0.22019.10 | 42.64 | 19.43 | — | — | 14.72 | 27.57 | 19.77 | — | — | — | — | — | |
| QA-based rerankinglambda_2=0.52019.10 | 42.5 | 19.38 | — | — | 14.56 | 27.42 | 19.62 | — | — | — | — | — | |
| QA-based rerankinglambda_2=0.82019.10 | 42.43 | 19.34 | — | — | 14.48 | 27.34 | 19.54 | — | — | — | — | — | |
| partial copylambda_1=22019.10 | 42.34 | 20.16 | — | — | 14.16 | 26.95 | 19.15 | — | — | — | — | — | |
| Song et al. (2018)source=our re-running2019.10 | 42.15 | 19.15 | — | — | 14.56 | 27.21 | 19.53 | — | — | — | — | — | |
| PEGASUSDecoding Setting=Single2022.03 | 22.17 | — | — | — | — | — | — | — | — | — | — | — | |
| FROSTDecoding Setting=Single2022.03 | 21.04 | — | — | — | — | — | — | — | — | — | — | — | |
| Composition (FROST++)Decoding Setting=Diverse, Backbone=FROST++2022.03 | 18.77 | — | — | — | — | — | — | 26.6 | 74.89 | — | — | — | |
| Composition (FROST)Decoding Setting=Diverse, Backbone=FROST2022.03 | 17.16 | — | — | — | — | — | — | 27.04 | 61.68 | — | — | — | |
| MCP+GSADecoding Setting=Single2022.03 | 16.85 | — | — | — | — | — | — | — | — | — | — | — | |
| Mixture Selector (Cho et al., 2019)Decoding Setting=Diverse2022.03 | 15.67 | — | — | — | — | — | — | 22.45 | 59.82 | — | — | — | |
| Mixture Selector (Wang et al., 2020)Decoding Setting=Diverse2022.03 | 15.34 | — | — | — | — | — | — | 21.15 | 54.18 | — | — | — | |
| Mixture DecoderDecoding Setting=Diverse2022.03 | 15.17 | — | — | — | — | — | — | 21.97 | 58.73 | — | — | — | |
| Diverse BeamDecoding Setting=Diverse2022.03 | 13.38 | — | — | — | — | — | — | 18.3 | 74.8 | — | — | — | |
| NQG++Decoding Setting=Single2022.03 | 13.27 | — | — | — | — | — | — | — | — | — | — | — | |
| Nucleus (PEGASUS)Decoding Setting=Diverse, Backbone=PEGASUS2022.03 | 12.05 | — | — | — | — | — | — | 24.72 | 30.64 | — | — | — | |
| top-k SamplingDecoding Setting=Diverse2022.03 | 11.53 | — | — | — | — | — | — | 17.65 | 45.99 | — | — | — | |
| Nucleus (FROST)Decoding Setting=Diverse, Backbone=FROST2022.03 | 10.64 | — | — | — | — | — | — | 22.49 | 25.5 | — | — | — | |
| Composition (FROST)Decoding strategy=Diverse Decoding2022.03 | — | — | — | — | — | — | — | — | — | 16.62 | 26.07 | 62.47 | |
| Composition (FROST++)Decoding strategy=Diverse Decoding2022.03 | — | — | — | — | — | — | — | — | — | 17.28 | 25.03 | 75.81 | |
| Du et al.year=20172018.05 | — | 15.2 | — | 38 | 10.8 | — | — | — | — | — | — | — | |
| Du et al.2018.05 | — | 15.2 | — | 38 | 10.8 | — | — | — | — | — | — | — | |
| FROSTDecoding strategy=Single-best with Beam Search2022.03 | — | — | — | — | — | — | — | — | — | 19.98 | — | — | |
| Nucleus (FROST)Decoding strategy=Diverse Decoding2022.03 | — | — | — | — | — | — | — | — | — | 10.98 | 22.61 | 26.36 | |
| Nucleus (PEGASUS)Decoding strategy=Diverse Decoding2022.03 | — | — | — | — | — | — | — | — | — | 12.6 | 24.45 | 31.23 | |
| Our pointer QGlayers=1, mechanism=pointer2018.05 | — | 15.4 | 75.3 | 36.2 | 9.2 | — | — | — | — | — | — | — | |
| Our pointer QGlayers=2, mechanism=pointer2018.05 | — | 17.5 | 95.3 | 40.1 | 13.8 | — | — | — | — | — | — | — | |
| PEGASUSDecoding strategy=Single-best with Beam Search2022.03 | — | — | — | — | — | — | — | — | — | 21.52 | — | — | |
| Soft Layer-Specific Multi-Task Model (1-layer pointer QG)number of layers=1, pointer mechanism=true2018.05 | — | 15.4 | 75.3 | 36.2 | 9.2 | — | — | — | — | — | — | — | |
| Soft Layer-Specific Multi-Task Model (2-layer pointer QG)number of layers=2, pointer mechanism=true2018.05 | — | 17.5 | 95.3 | 40.1 | 13.8 | — | — | — | — | — | — | — | |
| Song et al. (2018)source=reported in paper2019.10 | — | 18.77 | — | — | 13.98 | — | — | — | — | — | — | — |