Story Ending Generation on ROCStories (test)
57.47BLEU-1ClarET
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ClarET2022.03 | 57.47 | — | 19.16 | — | — | — | — | — | — | — | — | — | — | — | |
| BART-largeSize=Large2022.03 | 54.22 | — | 18.07 | — | — | — | — | — | — | — | — | — | — | — | |
| COINSKnowledge Resource=Oracle2021.06 | 41.8 | — | 28.4 | — | — | — | 47.9 | 78.6 | — | — | — | — | — | — | |
| GPT2-SSize=Small2022.03 | 39.23 | — | 13.08 | — | — | — | — | — | — | — | — | — | — | — | |
| beam search + A*esquedecoding_base=beam search, heuristic=beam2021.12 | 34.4 | 2.14 | 16.8 | 2.94 | — | — | — | — | 35,030 | 43,120 | 2.72 | 2.62 | 2.61 | 2.63 | |
| beam search + A*esquedecoding_base=beam search, heuristic=sample2021.12 | 34.4 | 2.16 | 16.7 | 2.92 | — | — | — | — | 35,410 | 43,640 | 2.71 | 2.59 | 2.52 | 2.57 | |
| beam search + A*esquedecoding_base=beam search, heuristic=greedy2021.12 | 34.3 | 2.11 | 16.7 | 2.94 | — | — | — | — | 34,940 | 43,020 | 2.71 | 2.56 | 2.5 | 2.57 | |
| beam searchdecoding_base=beam search2021.12 | 33.7 | 2.24 | 16.5 | 2.81 | — | — | — | — | 34,090 | 41,910 | 2.5 | 2.46 | 2.27 | 2.32 | |
| top-k sample + A*esquedecoding_base=top-k sample, heuristic=beam2021.12 | 32.2 | 3.75 | 14.4 | 2.84 | — | — | — | — | 48,270 | 56,360 | 2.49 | 2.39 | 2.4 | 2.34 | |
| top-k sample + A*esquedecoding_base=top-k sample, heuristic=greedy2021.12 | 32.1 | 3.68 | 14.3 | 2.88 | — | — | — | — | 48,440 | 56,630 | 2.57 | 2.48 | 2.49 | 2.47 | |
| top-k sample + A*esquedecoding_base=top-k sample, heuristic=sample2021.12 | 32 | 3.7 | 14.2 | 2.84 | — | — | — | — | 48,040 | 56,150 | 2.55 | 2.47 | 2.48 | 2.44 | |
| top-k sampledecoding_base=top-k sample2021.12 | 31.4 | 4.01 | 13.9 | 2.69 | — | — | — | — | 48,360 | 56,620 | 2.38 | 2.23 | 2.3 | 2.15 | |
| COINS OracleInference Rule type=specific (SR), Setting=Oracle (Test-only)2021.06 | 30.75 | — | 22.76 | — | — | 32.5 | — | — | — | — | — | — | — | — | |
| COINSKnowledge Resource=GR2021.06 | 27.4 | — | 12.3 | — | — | — | 42.8 | 72.4 | — | — | — | — | — | — | |
| IE+MSAAttention Type=CA2018.08 | 26.82 | 8.79 | 9.36 | 1.66 | 124 | — | — | — | — | — | — | — | — | — | |
| COINS OracleInference Rule type=general (GR), Setting=Oracle (Test-only)2021.06 | 26.37 | — | 17.01 | — | — | 27.38 | — | — | — | — | — | — | — | — | |
| GRF2021.06 | 26.1 | — | 11 | — | — | — | 37.8 | 62.2 | — | — | — | — | — | — | |
| HLSTM+MSAAttention Type=GA2018.08 | 25.88 | 15.75 | 8.09 | 1.7 | 106 | — | — | — | — | — | — | — | — | — | |
| IE+MSAAttention Type=GA2018.08 | 25.66 | 9.72 | 8.54 | 1.68 | 126 | — | — | — | — | — | — | — | — | — | |
| GPT2-GLUCOSE2021.06 | 25.6 | — | 10.2 | — | — | — | 36.1 | 60.9 | — | — | — | — | — | — | |
| GPT2021.06 | 25.5 | — | 10.2 | — | — | — | 30.4 | 50.5 | — | — | — | — | — | — | |
| GPT2-OMCS2021.06 | 25.5 | — | 10.4 | — | — | — | 35.2 | 58.9 | — | — | — | — | — | — | |
| HLSTM+MSAAttention Type=CA2018.08 | 25.14 | 12.53 | 8.25 | 1.72 | 102 | — | — | — | — | — | — | — | — | — | |
| IE2018.08 | 25.14 | 11.04 | 8.13 | 1.84 | 110 | — | — | — | — | — | — | — | — | — | |
| HLSTM+Copy2018.08 | 24.69 | 19.93 | 7.83 | 1.66 | 90 | — | — | — | — | — | — | — | — | — | |
| MGCN-DP2022.03 | 24.6 | — | 8.6 | — | — | — | — | — | — | — | — | — | — | — | |
| HLSTM2018.08 | 24.59 | 17.26 | 7.71 | 1.57 | 84 | — | — | — | — | — | — | — | — | — | |
| Human2021.06 | 24.53 | — | 12.1 | — | — | 20.2 | — | — | — | — | — | — | — | — | |
| IE+MSA2022.03 | 24.4 | — | 7.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Plan&Write2022.03 | 24.4 | — | 8.4 | — | — | — | — | — | — | — | — | — | — | — | |
| T-CVAE2022.03 | 24.3 | — | 7.7 | — | — | — | — | — | — | — | — | — | — | — | |
| COINSInference Rule type=general (GR)2021.06 | 22.82 | 6.9 | 10.52 | — | — | 19.4 | — | — | — | — | — | — | — | — | |
| COINSInference Rule type=specific (SR)2021.06 | 22.53 | 6.7 | 10.1 | — | — | 18.9 | — | — | — | — | — | — | — | — | |
| GRFKnowledge Source=ConceptNet (CN)2021.06 | 20.8 | 12.18 | 8.2 | — | — | 17.6 | — | — | — | — | — | — | — | — | |
| IE+GA2021.06 | 20.8 | — | 6.4 | — | — | — | 14 | 28 | — | — | — | — | — | — | |
| Seq2Seq2021.06 | 19.1 | — | 5.5 | — | — | — | 18.1 | 36 | — | — | — | — | — | — | |
| Seq2Seq2018.08 | 18.64 | 18.97 | 4.1 | 1.74 | 70 | — | — | — | — | — | — | — | — | — | |
| Seq2Seq2022.03 | 18.5 | — | 5.9 | — | — | — | — | — | — | — | — | — | — | — | |
| GLUCOSE-GPT-2Knowledge Source=GLUCOSE2021.06 | 17.9 | 12.7 | 7.8 | — | — | 17.5 | — | — | — | — | — | — | — | — | |
| GCN2022.03 | 17.6 | — | 6.2 | — | — | — | — | — | — | — | — | — | — | — | |
| KEKnowledge Sources=ConceptNet (CN), ATOMIC (AT)2021.06 | 17.55 | 12.61 | 7.6 | — | — | 17.9 | — | — | — | — | — | — | — | — | |
| Transformer2022.03 | 17.4 | — | 6 | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-22021.06 | 16.66 | 11.56 | 6.8 | — | — | 17.2 | — | — | — | — | — | — | — | — |