Paraphrase Generation on QQP (test)
41.74BLEU-2DIFFSEQ + Ours
Evaluation results
| Method | Links | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DIFFSEQ + OursStep=20002023.05 | 41.74 | — | — | — | 26.27 | 40.56 | 61.88 | 28.01 | — | — | — | — | — | — | — | — | — | — | — | |
| Distance PenaltyStep=2000, Base Model=DIFFSEQ2023.05 | 41.74 | — | — | — | 26.27 | 40.56 | 61.88 | 28.01 | 0 | 84.72 | 96.03 | — | — | — | — | — | — | — | — | |
| Respace + OursStep=202023.05 | 41.43 | — | — | — | 25.81 | 39.88 | 61.62 | 35.57 | — | — | — | — | — | — | — | — | — | — | — | |
| Adaptive Decay SamplingStep=20, Base Model=DIFFSEQ2023.05 | 41.43 | — | — | — | 25.81 | 39.88 | 61.62 | 35.57 | 0 | 84.35 | 95.63 | — | — | — | — | — | — | — | — | |
| DIFFSEQStep=20002023.05 | 39.75 | — | — | — | 24.5 | 38.13 | 60.4 | 52.15 | — | — | — | — | — | — | — | — | — | — | — | |
| DIFFSEQStep=20002023.05 | 39.75 | — | — | — | 24.5 | 38.13 | 60.4 | 52.15 | 0.09 | 83.41 | 94.47 | — | — | — | — | — | — | — | — | |
| RespaceStep=202023.05 | 38.58 | — | — | — | 23.67 | 36.67 | 59.11 | 90.61 | — | — | — | — | — | — | — | — | — | — | — | |
| RespaceStep=20, Base Model=DIFFSEQ2023.05 | 38.58 | — | — | — | 23.67 | 36.67 | 59.11 | 90.61 | 0 | 82.16 | 90.63 | — | — | — | — | — | — | — | — | |
| CMLM2023.05 | 35.67 | — | — | — | 21.78 | 34.51 | 56.12 | 12.56 | — | — | — | — | — | — | — | — | — | — | — | |
| CMLMStep=102023.05 | 35.67 | — | — | — | 21.78 | 34.51 | 56.12 | 12.56 | 0.04 | 82.86 | 97.75 | — | — | — | — | — | — | — | — | |
| Ours (Hierarchical Contrastive Learning Framework)backbone=BERT, beam search size=42022.05 | 35.17 | — | — | — | 11.53 | — | — | — | — | — | — | 64.3 | 18.45 | 67.01 | 84.95 | 86.61 | — | — | — | |
| BART2023.05 | 33.94 | — | — | — | 20.94 | 33.29 | 54.8 | 8.34 | — | — | — | — | — | — | — | — | — | — | — | |
| BART2023.05 | 33.94 | — | — | — | 20.94 | 33.29 | 54.8 | 8.34 | 0.28 | 82.28 | 98.39 | — | — | — | — | — | — | — | — | |
| T5-CLAPSbackbone=T5, beam search size=42022.05 | 33.94 | — | — | — | 10.58 | — | — | — | — | — | — | 62.64 | 17.87 | 66.83 | 84.3 | 86.12 | — | — | — | |
| T5backbone=T5, beam search size=42022.05 | 33.01 | — | — | — | 10.19 | — | — | — | — | — | — | 61.72 | 17.3 | 66.79 | 84.08 | 86.01 | — | — | — | |
| DialoGPTbeam search size=42022.05 | 28.45 | — | — | — | 7.49 | — | — | — | — | — | — | 57.49 | 13.37 | 66.58 | 83.93 | 85.97 | — | — | — | |
| Seq2Seq-DUbeam search size=42022.05 | 27.81 | — | — | — | 7.63 | — | — | — | — | — | — | 56.13 | 13.34 | 66.79 | 83.02 | 85.9 | — | — | — | |
| BERT-GENbackbone=BERT, beam search size=42022.05 | 27.81 | — | — | — | 7.64 | — | — | — | — | — | — | 54.52 | 13.43 | 66.73 | 82.99 | 85.86 | — | — | — | |
| Group-wisebeam search size=42022.05 | 24.72 | — | — | — | 5.82 | — | — | — | — | — | — | 52.01 | 11.12 | 65.76 | 83.37 | 85.69 | — | — | — | |
| Transformerbeam search size=42022.05 | 23.39 | — | — | — | 5.9 | — | — | — | — | — | — | 48.32 | 10.86 | 65.23 | 82.74 | 85.31 | — | — | — | |
| CVAEbeam search size=42022.05 | 21.5 | — | — | — | 4.96 | — | — | — | — | — | — | 45.62 | 9.62 | 66.06 | 83.71 | 84.06 | — | — | — | |
| Seq2Seqbeam search size=42022.05 | 21.17 | — | — | — | 4.97 | — | — | — | — | — | — | 45.1 | 9.5 | 65.43 | 82.43 | 85.33 | — | — | — | |
| AEsampling=Top-1, oracle_sketches=false2022.03 | — | 28.99 | 60.11 | 11.17 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BTmPGsampling=Top-1, oracle_sketches=false2022.03 | — | 19.83 | 35.11 | 8.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Copysampling=Top-1, oracle_sketches=false2022.03 | — | 34.52 | 100 | 7.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CopyDescription=Directly copying input sentence2022.10 | — | 29.88 | 100 | 16.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DaggerSetting=Parallel Quora Generation2022.10 | — | 28.42 | 66.98 | 18.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DialoGPT2022.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.18 | 2.04 | 1.97 | |
| HRQ-VAEsampling=Top-1, oracle_sketches=false2022.03 | — | 33.11 | 40.35 | 18.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LBOWsampling=Top-1, oracle_sketches=false2022.03 | — | 23.51 | 42.08 | 10.39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LTSLSetting=Non-Parallel Quora Generation2022.10 | — | 29.25 | 71.25 | 19.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Oraclesampling=Top-1, oracle_sketches=false2022.03 | — | 50.47 | 36.84 | 33.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OursSetting=Non-Parallel Quora Generation, Backbone=T5-Base2022.10 | — | 31.47 | 45.43 | 23.78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours2022.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.51 | 2.45 | 2.43 | |
| R-RegressionSetting=Non-Parallel Quora Generation, Backbone=T5-Base2022.10 | — | 30.77 | 44.23 | 23.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RL-NNSetting=Parallel Quora Generation2022.10 | — | 20.98 | 40.52 | 14.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-TrainingSetting=Non-Parallel Quora Generation, Backbone=T5-Base2022.10 | — | 31.39 | 48.02 | 23.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Separatorsampling=Top-1, oracle_sketches=false2022.03 | — | 23.68 | 24.2 | 14.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Seq2Seq-DU2022.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.03 | 2.12 | 1.76 | |
| SOW/REAPsampling=Top-1, oracle_sketches=false2022.03 | — | 21.27 | 38.01 | 9.41 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T5-BaseSetting=Parallel Quora Generation, Backbone=T5-Base2022.10 | — | 30.83 | 44.77 | 23.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T5-CLAPS2022.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.24 | 2.16 | 2.19 | |
| tf-idfsampling=Top-1, oracle_sketches=false2022.03 | — | 24.05 | 62.49 | 6.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VAEsampling=Top-1, oracle_sketches=false2022.03 | — | 27.23 | 51.09 | 11.57 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VQ-VAEsampling=Top-1, oracle_sketches=false2022.03 | — | 16.31 | 21.13 | 8.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |