Repetition Reduction on WikiText-103 (test)
23.82PPLSimCTG
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SimCTGdecoding=greedy2023.06 | 23.82 | 0.4091 | 0.5166 | 0.2865 | 0.6736 | 0.6333 | 0.05 | 0.05 | — | — | — | — | |
| MLEdecoding=greedy2023.06 | 24.23 | 0.3963 | 0.5282 | 0.2997 | 0.6921 | 0.6518 | 0.04 | 0.03 | — | — | — | — | |
| Lowestdecoding=greedy, sample construction strategy=Lowest2023.06 | 25.62 | 0.4136 | 0.4607 | 0.2514 | 0.4614 | 0.3574 | 0.25 | 0.43 | — | — | — | — | |
| Quarkdecoding=greedy2023.06 | 26.22 | 0.4157 | 0.4564 | 0.2507 | 0.3989 | 0.3062 | 0.35 | 0.74 | — | — | — | — | |
| Lowerdecoding=greedy, sample construction strategy=Lower2023.06 | 28.48 | 0.4044 | 0.4552 | 0.2509 | 0.3729 | 0.2512 | 0.39 | 0.75 | — | — | — | — | |
| Unlikelihooddecoding=greedy2023.06 | 28.57 | 0.3841 | 0.5123 | 0.2857 | 0.2412 | 0.1335 | 0.61 | 0.69 | — | — | — | — | |
| Randomdecoding=greedy, sample construction strategy=Random2023.06 | 29.4 | 0.4022 | 0.4561 | 0.2528 | 0.3502 | 0.2265 | 0.43 | 0.79 | — | — | — | — | |
| CLICKdecoding=greedy, sample construction strategy=Main2023.06 | 31.8 | 0.3883 | 0.4387 | 0.2473 | 0.2023 | 0.0743 | 0.72 | 0.93 | — | — | — | — | |
| CLICKComparison Baseline=Unlikelihood2023.06 | — | — | — | — | — | — | — | — | 0.35 | 0.36 | 0.39 | — | |
| CLICKComparison Baseline=SimCTG2023.06 | — | — | — | — | — | — | — | — | 0.52 | 0.6 | 0.55 | — | |
| Inter-annotator agreement (κ)2023.06 | — | — | — | — | — | — | — | — | 0.29 | 0.25 | 0.36 | — | |
| SimCTGComparison Baseline=CLICK2023.06 | — | — | — | — | — | — | — | — | 0.11 | 0.07 | 0.09 | — | |
| UnlikelihoodComparison Baseline=CLICK2023.06 | — | — | — | — | — | — | — | — | 0.19 | 0.24 | 0.25 | — |