Inducing Constituency Trees through Neural Machine Translation (1909.10056v1)

Abstract: Latent tree learning(LTL) methods learn to parse sentences using only indirect supervision from a downstream task. Recent advances in latent tree learning have made it possible to recover moderately high quality tree structures by training with LLMing or auto-encoding objectives. In this work, we explore the hypothesis that decoding in machine translation, as a conditional LLMing task, will produce better tree structures since it offers a similar training signal as LLMing, but with more semantic signal. We adapt two existing latent-tree LLMs--PRPN andON-LSTM--for use in translation. We find that they indeed recover trees that are better in F1 score than those seen in LLMing on WSJ test set, while maintaining strong translation quality. We observe that translation is a better objective than LLMing for inducing trees, marking the first success at latent tree learning using a machine translation objective. Additionally, our findings suggest that, although translation provides better signal for inducing trees than LLMing, translation models can perform well without exploiting the latent tree structure.