Unsupervised Parsing on PTB (test)
83.3F1 ScoreOracle
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Oracleevaluation_mode=Groundtruth tree2023.10 | 83.3 | — | — | — | — | — | — | — | |
| Oracledistillation=+RNNG2023.10 | 76 | — | — | — | — | — | — | — | |
| Oracledistillation=+URNNG2023.10 | 76 | — | — | — | — | — | — | — | |
| URNNGTraining Supervision=Supervised, Base Induction Source=Oracle Trees, Model Objective=Fine-tuning2019.06 | 72.8 | — | — | — | — | — | — | — | |
| Our ensemble (best teacher across runs)distillation=+URNNG2023.10 | 72.8 | — | — | — | — | — | — | — | |
| RNNGTraining Supervision=Supervised, Base Induction Source=Oracle Trees2019.06 | 71.9 | — | — | — | — | — | — | — | |
| Our ensemble (best teacher across runs)evaluation_mode=Run selection2023.10 | 71.9 | — | — | — | — | — | — | — | |
| Our ensemble (corresponding run)distillation=+URNNG2023.10 | 71.7 | — | — | — | — | — | — | — | |
| Selective MBRdistillation=+URNNG2023.10 | 71.5 | — | — | — | — | — | — | — | |
| Our ensemble (best teacher across runs)distillation=+RNNG2023.10 | 71.1 | — | — | — | — | — | — | — | |
| Our ensemble (corresponding run)evaluation_mode=Run 12023.10 | 70.5 | — | — | — | — | — | — | — | |
| Our ensemble (corresponding run)evaluation_mode=Mean over 5 runs2023.10 | 70.4 | — | — | — | — | — | — | — | |
| Our ensemble (worst teacher across runs)distillation=+URNNG2023.10 | 70 | — | — | — | — | — | — | — | |
| Our ensemble (corresponding run)distillation=+RNNG2023.10 | 69.7 | — | — | — | — | — | — | — | |
| Our ensemble (worst teacher across runs)evaluation_mode=Run selection2023.10 | 69.4 | — | — | — | — | — | — | — | |
| Our ensemble (worst teacher across runs)distillation=+RNNG2023.10 | 69.1 | — | — | — | — | — | — | — | |
| Selective MBRdistillation=+RNNG2023.10 | 68.6 | — | — | — | — | — | — | — | |
| ConTestdistillation=+URNNG2023.10 | 68.5 | — | — | — | — | — | — | — | |
| Compound PCFGdistillation=+URNNG2023.10 | 67.4 | — | — | — | — | — | — | — | |
| Induced URNNGTraining Supervision=Unsupervised, Base Induction Source=Compound PCFG, Model Objective=URNNG Fine-tuning2019.06 | 66.9 | — | — | — | — | — | — | — | |
| Selective MBRevaluation_mode=Run 12023.10 | 66.7 | — | — | — | — | — | — | — | |
| Selective MBRevaluation_mode=Mean over 5 runs2023.10 | 66.3 | — | — | — | — | — | — | — | |
| ConTestevaluation_mode=Run 12023.10 | 65.9 | — | — | — | — | — | — | — | |
| Union distillationdistillation=+RNNG2023.10 | 65.6 | — | — | — | — | — | — | — | |
| Union distillationdistillation=+URNNG2023.10 | 65.4 | — | — | — | — | — | — | — | |
| ReCAT_noshare [3, 1, 3]Memory Complexity=O(n), m=4, Compose function=non-shared2023.09 | 65 | — | 81.71 | 70.55 | 82.94 | 78.28 | — | — | |
| ConTestdistillation=+RNNG2023.10 | 64.6 | — | — | — | — | — | — | — | |
| TN-PCFGMemory Complexity=O(n^3)2023.09 | 64.1 | — | — | — | — | — | — | — | |
| ReCAT_noshare [3, 1, 3]Memory Complexity=O(n), m=2, Compose function=non-shared2023.09 | 64.06 | — | 80.36 | 64.38 | 80.93 | 80.96 | — | — | |
| ConTestevaluation_mode=Mean over 5 runs2023.10 | 62.9 | — | — | — | — | — | — | — | |
| DIORAdistillation=+URNNG2023.10 | 62.3 | — | — | — | — | — | — | — | |
| S-DIORAdistillation=+URNNG2023.10 | 62.2 | — | — | — | — | — | — | — | |
| Compound PCFGdistillation=+RNNG2023.10 | 60.5 | — | — | — | — | — | — | — | |
| NBL-PCFGMemory Complexity=O(n^3)2023.09 | 60.4 | — | — | — | — | — | — | — | |
| Compound PCFGTraining Supervision=Unsupervised2019.06 | 60.1 | — | — | — | — | — | — | — | |
| Compound PCFGevaluation_mode=Run 12023.10 | 60.1 | — | — | — | — | — | — | — | |
| S-DIORAdistillation=+RNNG2023.10 | 59.4 | — | — | — | — | — | — | — | |
| DIORAevaluation_mode=Mean over 5 runs2023.10 | 58.9 | — | — | — | — | — | — | — | |
| Induced URNNGTraining Supervision=Unsupervised, Base Induction Source=Neural PCFG, Model Objective=URNNG Fine-tuning2019.06 | 58.7 | — | — | — | — | — | — | — | |
| DIORAdistillation=+RNNG2023.10 | 58.6 | — | — | — | — | — | — | — | |
| Induced RNNGTraining Supervision=Unsupervised, Base Induction Source=Compound PCFG2019.06 | 58.1 | — | — | — | — | — | — | — | |
| S-DIORAMemory Complexity=O(n^3)2023.09 | 57.6 | — | — | — | — | — | — | — | |
| Fast-R2D2Memory Complexity=O(n), m=42023.09 | 57.22 | — | 83.44 | 63.8 | 70.56 | 68.47 | — | — | |
| S-DIORAevaluation_mode=Mean over 5 runs2023.10 | 57 | — | — | — | — | — | — | — | |
| S-DIORAevaluation_mode=Run 12023.10 | 56.3 | — | — | — | — | — | — | — | |
| ReCAT_share [3, 1, 3]Memory Complexity=O(n), m=4, Compose function=shared2023.09 | 56.07 | — | 85.41 | 66.14 | 68.43 | 72.92 | — | — | |
| Compound PCFGevaluation_mode=Mean over 5 runs2023.10 | 55.5 | — | — | — | — | — | — | — | |
| DIORAevaluation_mode=Run 12023.10 | 55.4 | — | — | — | — | — | — | — | |
| C-PCFGMemory Complexity=O(n^3)2023.09 | 55.2 | — | — | — | — | — | — | — | |
| ReCAT_share [3, 1, 3]Memory Complexity=O(n), m=2, Compose function=shared2023.09 | 55.11 | — | 77.22 | 67.05 | 66.53 | 69.44 | — | — | |
| Induced URNNGTraining Supervision=Unsupervised, Base Induction Source=ON, Model Objective=URNNG Fine-tuning2019.06 | 55.1 | — | — | — | — | — | — | — | |
| StructFormerMemory Complexity=O(n^2)2023.09 | 54 | — | — | — | — | — | — | — | |
| Neural PCFGTraining Supervision=Unsupervised2019.06 | 52.6 | — | — | — | — | — | — | — | |
| Induced URNNGTraining Supervision=Unsupervised, Base Induction Source=PRPN, Model Objective=URNNG Fine-tuning2019.06 | 51.6 | — | — | — | — | — | — | — | |
| Induced RNNGTraining Supervision=Unsupervised, Base Induction Source=Neural PCFG2019.06 | 51.4 | — | — | — | — | — | — | — | |
| Neural PCFGdistillation=+URNNG2023.10 | 51.1 | — | — | — | — | — | — | — | |
| Neural PCFGevaluation_mode=Mean over 5 runs2023.10 | 51 | — | — | — | — | — | — | — | |
| ContexDistortdistillation=+URNNG2023.10 | 50.8 | — | — | — | — | — | — | — | |
| Induced RNNGTraining Supervision=Unsupervised, Base Induction Source=ON2019.06 | 50.6 | — | — | — | — | — | — | — | |
| ONTraining Supervision=Unsupervised2019.06 | 50 | — | — | — | — | — | — | — | |
| Neural PCFGdistillation=+RNNG2023.10 | 48.9 | — | — | — | — | — | — | — | |
| ContexDistortevaluation_mode=Run 12023.10 | 48.8 | — | — | — | — | — | — | — | |
| ContexDistortdistillation=+RNNG2023.10 | 48.5 | — | — | — | — | — | — | — | |
| Neural PCFGevaluation_mode=Run 12023.10 | 48.4 | — | — | — | — | — | — | — | |
| PRPNTraining Supervision=Unsupervised2019.06 | 47.9 | — | — | — | — | — | — | — | |
| Induced RNNGTraining Supervision=Unsupervised, Base Induction Source=PRPN2019.06 | 47.8 | — | — | — | — | — | — | — | |
| ContexDistortevaluation_mode=Mean over 5 runs2023.10 | 47.8 | — | — | — | — | — | — | — | |
| ON-LSTMMemory Complexity=O(n)2023.09 | 47.7 | — | — | — | — | — | — | — | |
| Ordered Neuronsdistillation=+RNNG2023.10 | 45.4 | — | — | — | — | — | — | — | |
| Ordered Neuronsdistillation=+URNNG2023.10 | 45.3 | — | — | — | — | — | — | — | |
| ReCAT_noshare w/o iterMemory Complexity=O(n), m=22023.09 | 45.2 | — | — | — | — | — | — | — | |
| Ordered Neuronsevaluation_mode=Run 12023.10 | 44.8 | — | — | — | — | — | — | — | |
| Ordered Neuronsevaluation_mode=Mean over 5 runs2023.10 | 44.3 | — | — | — | — | — | — | — | |
| Right branchingevaluation_mode=Baseline2023.10 | 39.5 | — | — | — | — | — | — | — | |
| Left branchingevaluation_mode=Baseline2023.10 | 8.7 | — | — | — | — | — | — | — | |
| C-PCFG2022.05 | — | 55.2 | — | — | — | — | — | — | |
| C-PCFGMethod category=explicit grammar2024.10 | — | — | — | — | — | — | 55.2 | 60.1 | |
| CoMethod category=probing methods2024.10 | — | — | — | — | — | — | 63.1 | 66.8 | |
| Constituency test2022.05 | — | 62.8 | — | — | — | — | — | — | |
| CTMethod category=probing methods2024.10 | — | — | — | — | — | — | 62.8 | 65.9 | |
| DIORA2024.10 | — | — | — | — | — | — | 55.7 | 56.2 | |
| DIORA+span constraint2022.05 | — | 61.2 | — | — | — | — | — | — | |
| Ensemble2024.10 | — | — | — | — | — | — | 70.4 | 71.9 | |
| Fast R2D22024.10 | — | — | — | — | — | — | 48.9 | — | |
| GPST2024.10 | — | — | — | — | — | — | 57.5 | — | |
| Left Branching2024.10 | — | — | — | — | — | — | 8.7 | 8.7 | |
| N-PCFG2022.05 | — | 50.8 | — | — | — | — | — | — | |
| NBL-PCFG2022.05 | — | 60.4 | — | — | — | — | — | — | |
| NBL-PCFGMethod category=explicit grammar2024.10 | — | — | — | — | — | — | 60.4 | — | |
| NL-PCFG2022.05 | — | 55.3 | — | — | — | — | — | — | |
| NL-PCFGMethod category=explicit grammar2024.10 | — | — | — | — | — | — | 55.3 | — | |
| ON-LSTM2024.10 | — | — | — | — | — | — | 47.7 | 49.4 | |
| Oracle2024.10 | — | — | — | — | — | — | 84.3 | 84.3 | |
| Parserker2Backbone=BERT-base, Number of bits=16 bits, Method category=implicit grammar2024.10 | — | — | — | — | — | — | 55.3 | 58.8 | |
| Parserker2Backbone=BERT-base, Number of bits=20 bits, Method category=implicit grammar2024.10 | — | — | — | — | — | — | 56.7 | 59.8 | |
| Parserker2Backbone=BERT-base, Number of bits=24 bits, Method category=implicit grammar2024.10 | — | — | — | — | — | — | 57.4 | 59.6 | |
| Parserker2Backbone=BERT-base, Number of bits=28 bits, Method category=implicit grammar2024.10 | — | — | — | — | — | — | 54.5 | 60.9 | |
| Parserker2Backbone=RoBERTa-base, Number of bits=8 bits, Method category=implicit grammar2024.10 | — | — | — | — | — | — | 56.5 | 63.1 | |
| Parserker2Backbone=RoBERTa-base, Number of bits=12 bits, Method category=implicit grammar2024.10 | — | — | — | — | — | — | 58 | 62.9 | |
| Parserker2Backbone=RoBERTa-base, Number of bits=16 bits, Method category=implicit grammar2024.10 | — | — | — | — | — | — | 62.4 | 64.1 |