Sentence Simplification on Newsela (test)
70.13SARIReference
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reference2020.06 | 70.13 | 100 | 3.2 | — | — | — | — | — | — | — | — | — | 83.74 | 12.75 | — | — | |
| MUSSTraining Paradigm=Supervised2020.05 | 42.59 | — | 2.74 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SimpleBART2023.05 | 41.6 | — | — | — | — | — | — | — | — | 6.9 | 77.4 | — | — | — | 40.5 | 0.902 | |
| SimpleBART2023.05 | 41.6 | — | — | — | — | — | — | — | — | 6.9 | 77.4 | — | — | — | 40.5 | 0.902 | |
| MUSSTraining Paradigm=Supervised, Data augmentation=+ mined data2020.05 | 41.17 | — | 2.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BART-CPtraining_strategy=continued pre-training with random masking2023.05 | 40.3 | — | — | — | — | — | — | — | — | 6.9 | 72.6 | — | — | — | 41.7 | 0.908 | |
| BART2023.05 | 40.1 | — | — | — | — | — | — | — | — | 6.2 | 73.8 | — | — | — | 40.5 | 0.904 | |
| ControlTS2023.05 | 39.7 | — | — | — | — | — | — | — | — | 4.1 | 77.3 | — | — | — | 37.6 | 0.894 | |
| EditNTSTraining Paradigm=Supervised2020.05 | 39.3 | — | 3.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-3.5variant=Turbo-0301, Evaluation Protocol=Zero-shot2023.05 | 38.7 | — | — | — | — | — | — | — | — | 5.3 | 78.1 | — | — | — | 32.5 | 0.897 | |
| RM+EX+LS+ROTraining Paradigm=Unsupervised2020.05 | 38.33 | — | 2.98 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MUSSTraining Paradigm=Unsupervised2020.05 | 38.09 | — | 5.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Dress-LSTraining Paradigm=Supervised2020.05 | 38 | — | 4.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| S2S-Cluster-FAreranking=feature-based, clustering=true2019.04 | 37.22 | — | — | — | — | — | — | — | 50.36 | — | — | — | — | — | — | — | |
| BTRLTSTraining Paradigm=Unsupervised2020.05 | 37.22 | — | 3.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| S2S-All-FAreranking=feature-based, combined_components=full2019.04 | 37.11 | — | — | — | — | — | — | — | 50.4 | — | — | — | — | — | — | — | |
| EditNTS2023.05 | 37.1 | — | — | — | — | — | — | — | — | 1.6 | 74.8 | — | — | — | 34.9 | 0.897 | |
| S2S-FAreranking=feature-based2019.04 | 36.47 | — | — | — | — | — | — | — | 54.01 | — | — | — | — | — | — | — | |
| S2S2019.04 | 36.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| S2S-All-FASreranking=feature-based, combined_components=full2019.04 | 36.3 | — | — | — | — | — | — | — | 50.4 | — | — | — | — | — | — | — | |
| S2S-Lossloss_function=Complexity-Weighted2019.04 | 36.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DRESS2019.04 | 36 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T5Model size=base2023.05 | 36 | — | — | — | — | — | — | — | — | 4.4 | 61.9 | — | — | — | 41.8 | 0.905 | |
| S2S-Diverse-FAreranking=feature-based, beam_search=diverse2019.04 | 35.36 | — | — | — | — | — | — | — | 52.65 | — | — | — | — | — | — | — | |
| DMASS2019.04 | 34.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Hybrid2019.04 | 33.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EntPar2020.06 | 33.22 | 11.14 | 1.34 | — | — | — | — | — | — | 2.42 | 89.32 | 7.92 | 19.24 | 7.88 | — | — | |
| FLAN-T5variant=XL, Evaluation Protocol=Zero-shot2023.05 | 32.2 | — | — | — | — | — | — | — | — | 1.3 | 65.7 | — | — | — | 29.7 | 0.891 | |
| DMass2020.06 | 31.06 | 11.92 | 3.6 | — | — | — | — | — | — | 1.25 | 84.12 | 7.82 | 19.24 | 15.07 | — | — | |
| S2S-All-FA2020.06 | 30.73 | 19.55 | 2.6 | — | — | — | — | — | — | 2.64 | 81.6 | 7.97 | 24.51 | 10.81 | — | — | |
| RM+EX+LS+ROTuning objective=SARI2020.06 | 30.44 | 17.36 | 2.24 | — | — | — | — | — | — | 2.05 | 81.77 | 7.49 | 22.99 | 9.61 | — | — | |
| Edit-NTS2020.06 | 30.27 | 19.85 | 3.41 | — | — | — | — | — | — | 2.71 | 80.34 | 7.76 | 24.51 | 10.92 | — | — | |
| Hybrid2017.03 | 30 | 14.46 | 4.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HYBRID2018.04 | 30 | 14.46 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLAN-T5variant=base, Evaluation Protocol=Fine-tuned2023.05 | 29.9 | — | — | — | — | — | — | — | — | 2.7 | 46.7 | — | — | — | 40.3 | 0.902 | |
| NSELSTM-SOptimization=SARI2018.04 | 29.58 | 22.62 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LSTMLSTM-SOptimization=SARI2018.04 | 28.67 | 23.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Hybrid2020.06 | 28.61 | 14.46 | 4.03 | — | — | — | — | — | — | 0.95 | 78.86 | 6.01 | 20.34 | 12.41 | — | — | |
| Reduce-2502020.06 | 28.39 | 11.79 | -0.23 | — | — | — | — | — | — | — | — | — | 18.29 | 14.48 | — | — | |
| EncDecP2020.06 | 28.31 | 23.72 | — | — | — | — | — | — | — | — | — | — | 25.91 | — | — | — | |
| LSTMLSTM-BOptimization=Basic2018.04 | 27.66 | 24.38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NSELSTM-BOptimization=Basic2018.04 | 27.42 | 26.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DRESS2017.03 | 27.37 | 23.21 | 4.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DRESS2018.04 | 27.37 | 23.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Dress2020.06 | 27.37 | 23.2 | 4.11 | — | — | — | — | — | — | 3.08 | 71.61 | 7.43 | 25.2 | 14.2 | — | — | |
| RM+EX+LS+ROTuning objective=geometric mean (GM)2020.06 | 27.11 | 26.21 | 3.12 | — | — | — | — | — | — | 2.4 | 71.26 | 7.67 | 26.66 | 12.81 | — | — | |
| RM+EX+ROTuning objective=geometric mean (GM)2020.06 | 26.99 | 26.31 | 3.14 | — | — | — | — | — | — | 2.47 | 70.88 | 7.63 | 26.64 | 12.81 | — | — | |
| DRESS-LS2017.03 | 26.63 | 24.3 | 4.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DRESS-LSLexical Simplification=true2018.04 | 26.63 | 24.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Dress-Ls2020.06 | 26.63 | 24.25 | 4.21 | — | — | — | — | — | — | 3.21 | 69.28 | 7.4 | 25.41 | 14.37 | — | — | |
| RM+EX+LSTuning objective=geometric mean (GM)2020.06 | 26.26 | 27.17 | 2.93 | — | — | — | — | — | — | 2.28 | 68.94 | 7.57 | 26.71 | 12.88 | — | — | |
| RM+EXTuning objective=geometric mean (GM)2020.06 | 26.07 | 27.22 | 2.95 | — | — | — | — | — | — | 2.35 | 68.35 | 7.5 | 26.64 | 12.9 | — | — | |
| Truncate Baseline2020.05 | 25.49 | — | 6.68 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EncDecA2017.03 | 24.12 | 21.7 | 5.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EncDecA2020.06 | 24.12 | 21.68 | 5.11 | — | — | — | — | — | — | 2.73 | 62.66 | 6.98 | 22.87 | 16.96 | — | — | |
| LLaMAvariant=7B, Evaluation Protocol=Zero-shot2023.05 | 19.9 | — | — | — | — | — | — | — | — | 0.8 | 23.2 | — | — | — | 35.8 | 0.822 | |
| PBMT-R2017.03 | 15.77 | 18.19 | 7.59 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PBMT-R2018.04 | 15.77 | 18.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PBMT-R2020.06 | 15.77 | 18.1 | 7.59 | — | — | — | — | — | — | 3.07 | 38.34 | 5.9 | 16.89 | 23.06 | — | — | |
| Identity Baseline2020.05 | 12.24 | — | 8.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Complex2020.06 | 2.82 | 21.3 | 8.62 | — | — | — | — | — | — | — | — | — | 7.75 | 23.06 | — | — | |
| DRESStype=deep reinforcement learning model2017.03 | — | — | — | 3.65 | 2.94 | 3.1 | 3.23 | — | — | — | — | — | — | — | — | — | |
| DRESS2018.04 | — | — | — | 3.98 | 2.84 | 2.93 | — | 3.25 | — | — | — | — | — | — | — | — | |
| DRESS2019.06 | — | — | — | 4.16 | 3.08 | 3 | — | 3.41 | — | — | — | — | — | — | — | — | |
| DRESS-LSlexical_simplification=true2017.03 | — | — | — | 3.71 | 3.07 | 3.04 | 3.28 | — | — | — | — | — | — | — | — | — | |
| DRESS-LS2018.04 | — | — | — | 3.99 | 2.9 | 2.98 | — | 3.29 | — | — | — | — | — | — | — | — | |
| EditNTS2019.06 | — | — | — | 4.34 | 3.13 | 3.16 | — | 3.54 | — | — | — | — | — | — | — | — | |
| EncDecAtype=attention-based encoder-decoder2017.03 | — | — | — | 3.63 | 2.99 | 2.56 | 3.06 | — | — | — | — | — | — | — | — | — | |
| Hybridtype=sentence splitting and deletion + PBMT-R2017.03 | — | — | — | 2.7 | 2.51 | 2.99 | 2.73 | — | — | — | — | — | — | — | — | — | |
| Hybrid2019.06 | — | — | — | 3.62 | 2.88 | 2.97 | — | 3.16 | — | — | — | — | — | — | — | — | |
| HYBRID2018.04 | — | — | — | 2.77 | 2.56 | 2.41 | — | 2.58 | — | — | — | — | — | — | — | — | |
| LSTMLSTMvariant=B2018.04 | — | — | — | 3.95 | 2.93 | 3.14 | — | 3.34 | — | — | — | — | — | — | — | — | |
| LSTMLSTMvariant=S2018.04 | — | — | — | 4.24 | 3.03 | 3.45 | — | 3.57 | — | — | — | — | — | — | — | — | |
| NSELSTMvariant=B2018.04 | — | — | — | 4.26 | 3.13 | 3.39 | — | 3.59 | — | — | — | — | — | — | — | — | |
| NSELSTMvariant=S2018.04 | — | — | — | 3.83 | 2.78 | 3.01 | — | 3.21 | — | — | — | — | — | — | — | — | |
| PBMT-Rtype=phrase-based machine translation with reranking2017.03 | — | — | — | 3.56 | 3.58 | 2.09 | 3.08 | — | — | — | — | — | — | — | — | — | |
| PBMT-R2018.04 | — | — | — | 3.73 | 3.9 | 1.98 | — | 3.2 | — | — | — | — | — | — | — | — | |
| PBMT-R2019.06 | — | — | — | 3.76 | 3.44 | 2.28 | — | 3.16 | — | — | — | — | — | — | — | — | |
| Referencetype=Gold standard2017.03 | — | — | — | 3.9 | 2.81 | 3.42 | 3.38 | — | — | — | — | — | — | — | — | — | |
| Reference2019.06 | — | — | — | 4.4 | 2.74 | 3.79 | — | 3.64 | — | — | — | — | — | — | — | — | |
| REFERENCE2018.04 | — | — | — | 4.58 | 2.98 | 3.99 | — | 3.85 | — | — | — | — | — | — | — | — | |
| seq-label2019.06 | — | — | — | 3.45 | 3.22 | 2.09 | — | 2.92 | — | — | — | — | — | — | — | — |