# Unsupervised Learning of Syntactic Structure with Invertible Neural Projections

Source: [Unsupervised Learning of Syntactic Structure with Invertible Neural Projections](<https://www.sota2.com/research/paper/68fa64bd1e209131cea062e0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:02:00.754Z

Authors: Junxian He, Graham Neubig, Taylor Berg-Kirkpatrick

Published: 2018-08-28T04:33:25.000Z

## Abstract

Unsupervised learning of syntactic structure is typically performed using generative models with discrete latent variables and multinomial parameters. In most cases, these models have not leveraged continuous word representations. In this work, we propose a novel generative model that jointly learns discrete syntactic structure and continuous word representations in an unsupervised fashion by cascading an invertible neural network with a structured generative prior. We show that the invertibility condition allows for efficient exact inference and marginal likelihood computation in our model so long as the prior is well-behaved. In experiments we instantiate our approach with both Markov and tree-structured priors, evaluating on two tasks: part-of-speech (POS) induction, and unsupervised dependency parsing without gold POS annotation. On the Penn Treebank, our Markov-structured model surpasses state-of-the-art results on POS induction. Similarly, we find that our tree-structured model achieves state-of-the-art performance on unsupervised dependency parsing for the difficult training condition where neither gold POS annotation nor punctuation-based constraints are available.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Unsupervised POS tagging on WSJ entire corpus (full)](<https://www.sota2.com/research/sota/unsupervised-pos-tagging-on-wsj-entire-corpus-full.md>): Unsupervised POS tagging; WSJ entire corpus (full); M1 Score: 80.8.

- [Unsupervised Dependency Parsing on WSJ section 23 length \<= 10 (test)](<https://www.sota2.com/research/sota/unsupervised-dependency-parsing-on-wsj-section-23-length-10-test.md>): Unsupervised Dependency Parsing; WSJ section 23 length \<= 10 (test); DDA: 60.2.

- [Unsupervised Dependency Parsing on WSJ section 23 (all lengths) (test)](<https://www.sota2.com/research/sota/unsupervised-dependency-parsing-on-wsj-section-23-all-lengths-test.md>): Unsupervised Dependency Parsing; WSJ section 23 (all lengths) (test); Directed Dependency Accuracy (DDA): 47.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1808.09111>)

[PDF](<https://arxiv.org/pdf/1808.09111.pdf>)

[Implementation](<https://github.com/jxhe/struct-learning-with-flow>)
