Paraphrase Evaluation on Quiz design dataset
0.5019Pearson CorrelationBLEURT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BLEURTEvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.05 | 0.5019 | 0.4816 | |
| MoverScoreEvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.05 | 0.4953 | 0.4292 | |
| BLEURTEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.05 | 0.4803 | 0.4648 | |
| BLEURTEvaluation Strategy=SRE2023.05 | 0.4739 | 0.4566 | |
| MoverScoreEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.05 | 0.4693 | 0.4214 | |
| BLEURTEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.05 | 0.4656 | 0.4456 | |
| MoverScoreEvaluation Strategy=SRE2023.05 | 0.4383 | 0.3882 | |
| MoverScoreEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.05 | 0.4297 | 0.3885 | |
| BLEURTEvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.05 | 0.4287 | 0.4193 | |
| METEOREvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.05 | 0.4116 | 0.378 | |
| BERTScoreEvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.05 | 0.4033 | 0.3859 | |
| METEOREvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.05 | 0.3877 | 0.3562 | |
| BERTScoreEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.05 | 0.3877 | 0.3723 | |
| MoverScoreEvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.05 | 0.3835 | 0.3643 | |
| ROUGE-LEvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.05 | 0.3799 | 0.3637 | |
| BERTScoreEvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.05 | 0.3634 | 0.3568 | |
| BLEU-4Evaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.05 | 0.363 | 0.334 | |
| BERTScoreEvaluation Strategy=SRE2023.05 | 0.3556 | 0.3462 | |
| BERTScoreEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.05 | 0.3552 | 0.3327 | |
| METEOREvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.05 | 0.348 | 0.3159 | |
| ROUGE-LEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.05 | 0.3447 | 0.333 | |
| METEOREvaluation Strategy=SRE2023.05 | 0.3447 | 0.3111 | |
| ROUGE-LEvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.05 | 0.3325 | 0.327 | |
| ROUGE-LEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.05 | 0.3241 | 0.305 | |
| BLEU-4Evaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.05 | 0.3162 | 0.3021 | |
| METEOREvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.05 | 0.2968 | 0.2822 | |
| RQUGEEvaluation Strategy=SRE2023.05 | 0.2932 | 0.2571 | |
| ROUGE-LEvaluation Strategy=SRE2023.05 | 0.2908 | 0.2787 | |
| BLEU-4Evaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.05 | 0.2782 | 0.2688 | |
| BLEU-4Evaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.05 | 0.2443 | 0.3224 | |
| BLEU-4Evaluation Strategy=SRE2023.05 | 0.2028 | 0.2772 |