List your company
ResearchFormerly WizWand

Text Summarization on NYT (test)

49.18R1 ScoreORACLE
10.096820.243430.3940.5366Aug 22, 2019Mar 11, 2020Sep 30, 2020Apr 21, 2021Nov 9, 2021May 31, 2022Dec 20, 2022
Updated 8mo ago

Evaluation results

MethodLinks
ORACLEevaluation_protocol=limited-length ROUGE Recall, type=upper bound2019.0849.1833.2446.02
BERTSUMEXTABSevaluation_protocol=limited-length ROUGE Recall, type=abstractive, backbone=BERT2019.0849.0231.0245.55
BERTSUMABSevaluation_protocol=limited-length ROUGE Recall, type=abstractive, backbone=BERT2019.0848.9230.8445.41
BERTSUMEXTevaluation_protocol=limited-length ROUGE Recall, type=extractive, backbone=BERT2019.0846.6626.3542.62
PTGEN + COVevaluation_protocol=limited-length ROUGE Recall, type=abstractive2019.0843.7126.4—
DRMevaluation_protocol=limited-length ROUGE Recall, type=abstractive2019.0842.9426.02—
PTGENevaluation_protocol=limited-length ROUGE Recall, type=abstractive2019.0842.4725.61—
SUMOevaluation_protocol=limited-length ROUGE Recall, type=extractive2019.0842.322.738.6
COMPRESSevaluation_protocol=limited-length ROUGE Recall, type=extractive2019.0842.224.9—
TransformerEXTevaluation_protocol=limited-length ROUGE Recall, type=extractive, pre-trained=false2019.0841.9522.6838.51
LEAD-3evaluation_protocol=limited-length ROUGE Recall, type=baseline2019.0839.5820.1135.78
TransformerABSevaluation_protocol=limited-length ROUGE Recall, type=abstractive2019.0835.7517.2331.41
DIONYSUSModel Size=Large, Evaluation Protocol=Zero-shot, Strategy=Better ROUGE2022.1224.1415.4
PEGASUSModel Size=Large, Evaluation Protocol=Zero-shot2022.1223.73.213.2
DIONYSUSModel Size=Large, Evaluation Protocol=Zero-shot, Strategy=All G2022.1221.73.714.8
DIONYSUSModel Size=Large, Evaluation Protocol=Zero-shot, Strategy=All P2022.1219.82.712.9
GSG+Evaluation Protocol=Zero-shot2022.1217.12.412.9
T5v1.1Model Size=Large, Evaluation Protocol=Zero-shot2022.1211.61.48.7