Unlabeled Parsing on Penn Treebank WSJ (test)
84.3F1 (mean)Oracle Trees
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Oracle Trees2019.06 | 84.3 | — | — | — | — | — | — | — | |
| Compound PCFG2019.06 | 55.2 | — | 60.1 | — | — | — | — | — | |
| Neural PCFG2019.06 | 50.8 | — | 52.6 | — | — | — | — | — | |
| ONTuning=Tuned for unsupervised parsing only2019.06 | 48.1 | — | 50 | — | — | — | — | — | |
| ON-LSTM 2nd-layerTraining Data=WSJ Train, Training Object=LM, Vocab Size=10k2018.10 | 47.7 | 1.5 | 49.4 | 5.6 | 46.2 | 61.4 | 55.4 | 0 | |
| ONTuning=Original hyperparameters2019.06 | 47.7 | — | 49.4 | — | — | — | — | — | |
| PRPNTuning=Tuned for unsupervised parsing only2019.06 | 47.3 | — | 47.9 | — | — | — | — | — | |
| Right Branching2018.10 | 39.8 | 0 | 39.8 | 12.4 | — | — | — | — | |
| Right Branching2019.06 | 39.5 | — | — | — | — | — | — | — | |
| PRPN-UPTraining Data=AllNLI Train, Training Object=LM, Vocab Size=76k2018.10 | 38.3 | 0.5 | 39.8 | 5.8 | 28.7 | 65.5 | 32.7 | 0 | |
| PRPN-LMTraining Data=WSJ Train, Training Object=LM, Vocab Size=10k2018.10 | 37.4 | 0.3 | 38.1 | 5.9 | 26.2 | 63.9 | 24.4 | 0 | |
| PRPNTuning=Original hyperparameters2019.06 | 37.4 | — | 38.1 | — | — | — | — | — | |
| ON-LSTM 3rd-layerTraining Data=WSJ Train, Training Object=LM, Vocab Size=10k2018.10 | 36.6 | 3.3 | 40.4 | 5.3 | 44.8 | 57.5 | 47.2 | 0 | |
| PRPN-LMTraining Data=AllNLI Train, Training Object=LM, Vocab Size=76k2018.10 | 35 | 5.4 | 42.8 | 6.1 | 37.8 | 59.7 | 61.5 | 100 | |
| PRPN-UPTraining Data=WSJ Train, Training Object=LM, Vocab Size=15.8k2018.10 | 26 | 2.3 | 32.8 | 5.8 | 24.8 | 54.4 | 17.8 | 0 | |
| Balanced Trees2018.10 | 24.5 | 0 | 24.5 | 4.6 | 22.1 | 20.2 | 9.3 | 55.9 | |
| 300D ST-Gumbel w/o Leaf GRUTraining Data=AllNLI Train, Training Object=NLI2018.10 | 22.8 | 1.6 | 25 | — | 18.9 | 24.1 | 14.2 | 51.8 | |
| ON-LSTM 1st-layerTraining Data=WSJ Train, Training Object=LM, Vocab Size=10k2018.10 | 20 | 2.8 | 24 | 5.6 | 38.1 | 23.8 | 18.3 | 100 | |
| Random Trees2019.06 | 19.2 | — | 19.5 | — | — | — | — | — | |
| 300D ST-GumbelTraining Data=AllNLI Train, Training Object=NLI2018.10 | 19 | 1 | 20.1 | — | 15.6 | 18.8 | 9.9 | 59.4 | |
| Random Trees2018.10 | 18.4 | 0.1 | 18.6 | 5.3 | 15.1 | 22.3 | 16 | 40.4 | |
| 300D RL-SPINNTraining Data=AllNLI Train, Training Object=NLI2018.10 | 13.2 | 0 | 13.2 | — | 1.7 | 16.8 | 4.6 | 50.6 | |
| 300D RL-SPINN w/o Leaf GRUTraining Data=AllNLI Train, Training Object=NLI2018.10 | 13.1 | 0.1 | 13.2 | — | 1.6 | 16.9 | 4.6 | 50 | |
| Left Branching2018.10 | 9 | 0 | 9 | 12.4 | — | — | — | — | |
| Left Branching2019.06 | 8.7 | — | — | — | — | — | — | — | |
| DIORA2019.06 | — | — | 58.9 | — | — | — | — | — | |
| URNNG2019.06 | — | — | 45.4 | — | — | — | — | — |