List your company
ResearchFormerly WizWand

Paraphrase Evaluation on Quiz design dataset

0.5019Pearson CorrelationBLEURT
0.1908360.2715930.352350.433107May 26, 2023
Updated 8mo ago

Evaluation results

MethodLinks
BLEURTEvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.050.50190.4816
MoverScoreEvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.050.49530.4292
BLEURTEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.050.48030.4648
BLEURTEvaluation Strategy=SRE2023.050.47390.4566
MoverScoreEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.050.46930.4214
BLEURTEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.050.46560.4456
MoverScoreEvaluation Strategy=SRE2023.050.43830.3882
MoverScoreEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.050.42970.3885
BLEURTEvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.050.42870.4193
METEOREvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.050.41160.378
BERTScoreEvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.050.40330.3859
METEOREvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.050.38770.3562
BERTScoreEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.050.38770.3723
MoverScoreEvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.050.38350.3643
ROUGE-LEvaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.050.37990.3637
BERTScoreEvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.050.36340.3568
BLEU-4Evaluation Strategy=MRE, Reference Generator=ChatGPT, Shot-count=0-shot2023.050.3630.334
BERTScoreEvaluation Strategy=SRE2023.050.35560.3462
BERTScoreEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.050.35520.3327
METEOREvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.050.3480.3159
ROUGE-LEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.050.34470.333
METEOREvaluation Strategy=SRE2023.050.34470.3111
ROUGE-LEvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.050.33250.327
ROUGE-LEvaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.050.32410.305
BLEU-4Evaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=3-shot2023.050.31620.3021
METEOREvaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.050.29680.2822
RQUGEEvaluation Strategy=SRE2023.050.29320.2571
ROUGE-LEvaluation Strategy=SRE2023.050.29080.2787
BLEU-4Evaluation Strategy=MRE, Reference Generator=GPT-3, Shot-count=0-shot2023.050.27820.2688
BLEU-4Evaluation Strategy=MRE, Reference Generator=HRQ-VAE2023.050.24430.3224
BLEU-4Evaluation Strategy=SRE2023.050.20280.2772