Text Simplification on Newsela auto (test)
99.9SARIReference
Evaluation results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ReferenceType=Ground Truth2023.05 | 99.9 | — | — | — | 4.93 | — | — | — | — | — | — | — | -0.93 | -0.93 | -0.93 | 100 | 100 | 100 | 671.5 | 42.6 | |
| Simple (reference)type=reference2021.06 | 86.9 | 84.7 | 78.4 | 97.6 | 6.5 | 13.3 | 13.3 | 0.63 | 0.8 | 25.7 | 33.5 | 0 | — | — | — | — | — | — | — | — | |
| O -> ConBARTPlanning Strategy=Oracle plan, Model Architecture=ConBART2023.05 | 83.8 | — | — | — | 4.92 | — | — | — | — | — | — | — | -1.32 | -1.32 | -1.32 | 88 | 87.7 | 87.8 | 659.6 | 42.3 | |
| O -> LED_paraPlanning Strategy=Oracle plan, Model Architecture=LED_para2023.05 | 82.3 | — | — | — | 5.03 | — | — | — | — | — | — | — | -1.36 | -1.33 | -1.35 | 87 | 87.3 | 87.1 | 673.6 | 42.4 | |
| PG_OraclePlanning=Oracle, Year=20232023.05 | 80.7 | — | — | — | 4.91 | — | — | — | — | — | — | — | -1.39 | -1.4 | -1.4 | 85.5 | 85 | 85.3 | 655.6 | 42.1 | |
| O -> BART_paraPlanning Strategy=Oracle plan, Model Architecture=BART_para2023.05 | 79.7 | — | — | — | 5.07 | — | — | — | — | — | — | — | -1.6 | -1.36 | -1.48 | 83.6 | 85.3 | 84.3 | 706.2 | 42.3 | |
| Ô -> LED_paraPlanning Strategy=Predicted plan, Model Architecture=LED_para2023.05 | 78.5 | — | — | — | 5.09 | — | — | — | — | — | — | — | -1.5 | -1.42 | -1.46 | 83.7 | 84.9 | 84.1 | 683.1 | 42.8 | |
| Ô -> ConBARTPlanning Strategy=Predicted plan, Model Architecture=ConBART2023.05 | 78.3 | — | — | — | 4.96 | — | — | — | — | — | — | — | -1.52 | -1.45 | -1.48 | 82.8 | 84 | 83.2 | 671.6 | 43 | |
| LED_paraGranularity=paragraph2023.05 | 76.9 | — | — | — | 5.15 | — | — | — | — | — | — | — | -1.61 | -1.4 | -1.5 | 81.1 | 85.5 | 83 | 712.9 | 44.9 | |
| ConBARTArchitecture=ConBART2023.05 | 75.8 | — | — | — | 5.01 | — | — | — | — | — | — | — | -1.59 | -1.5 | -1.54 | 81.2 | 82.5 | 81.7 | 669.8 | 42.8 | |
| PG_DynPlanning=Predicted, Year=20232023.05 | 75 | — | — | — | 4.98 | — | — | — | — | — | — | — | -1.6 | -1.54 | -1.57 | 80.2 | 81 | 80.5 | 667.2 | 42.6 | |
| Ô -> BART_paraPlanning Strategy=Predicted plan, Model Architecture=BART_para2023.05 | 74.9 | — | — | — | 5.11 | — | — | — | — | — | — | — | -1.75 | -1.47 | -1.61 | 79.4 | 81.9 | 80.4 | 715.3 | 42.7 | |
| BART_paraGranularity=paragraph2023.05 | 73.7 | — | — | — | 5.28 | — | — | — | — | — | — | — | -1.85 | -1.49 | -1.67 | 77.2 | 82.8 | 79.6 | 752.8 | 45.6 | |
| BART_sentGranularity=sentence2023.05 | 73 | — | — | — | 5.03 | — | — | — | — | — | — | — | -1.63 | -1.56 | -1.6 | 78.9 | 80.1 | 79.3 | 666.4 | 42.6 | |
| LED_docGranularity=document2023.05 | 68.7 | — | — | — | 4.87 | — | — | — | — | — | — | — | -1.68 | -1.73 | -1.7 | 75.3 | 74.9 | 74.8 | 643.7 | 41.5 | |
| BART_docGranularity=document2023.05 | 47.1 | — | — | — | 10.01 | — | — | — | — | — | — | — | -2.68 | -2.76 | -2.72 | 61.9 | 43.9 | 50.6 | 600.8 | 20.7 | |
| EditNTS + Alignerbackbone=EditNTS, aligner=neural semi-CRF word alignment model2021.06 | 37.5 | 1.3 | 33.4 | 77.9 | 7.2 | 14.3 | 14.3 | 0.66 | 1.5 | 49.1 | 7.6 | 0.8 | — | — | — | — | — | — | — | — | |
| EditNTSbackbone=EditNTS2021.06 | 36.6 | 1.1 | 32.9 | 75.7 | 7.5 | 14.3 | 14.3 | 0.66 | 5.4 | 50.2 | 6.5 | 1.2 | — | — | — | — | — | — | — | — | |
| InputType=Source Baseline2023.05 | 20.5 | — | — | — | 8.44 | — | — | — | — | — | — | — | -2.47 | -1.99 | -2.23 | 63.2 | 62.7 | 62.8 | 866.9 | 38.6 | |
| Complex (input)type=input2021.06 | 11.8 | 0 | 35.5 | 0 | 12.3 | 24.8 | 24.8 | 1 | 2 | 100 | 0 | 100 | — | — | — | — | — | — | — | — |