Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections

Junxian He, Zhisong Zhang, Taylor Berg-Kirkpatrick, Graham Neubig


Abstract
Cross-lingual transfer is an effective way to build syntactic analysis tools in low-resource languages. However, transfer is difficult when transferring to typologically distant languages, especially when neither annotated target data nor parallel corpora are available. In this paper, we focus on methods for cross-lingual transfer to distant languages and propose to learn a generative model with a structured prior that utilizes labeled source data and unlabeled target data jointly. The parameters of source model and target model are softly shared through a regularized log likelihood objective. An invertible projection is employed to learn a new interlingual latent embedding space that compensates for imperfect cross-lingual word embedding input. We evaluate our method on two syntactic tasks: part-of-speech (POS) tagging and dependency parsing. On the Universal Dependency Treebanks, we use English as the only source corpus and transfer to a wide range of target languages. On the 10 languages in this dataset that are distant from English, our method yields an average of 5.2% absolute improvement on POS tagging and 8.3% absolute improvement on dependency parsing over a direct transfer method using state-of-the-art discriminative models.
Anthology ID:
P19-1311
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3211–3223
Language:
URL:
https://aclanthology.org/P19-1311
DOI:
10.18653/v1/P19-1311
Bibkey:
Cite (ACL):
Junxian He, Zhisong Zhang, Taylor Berg-Kirkpatrick, and Graham Neubig. 2019. Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3211–3223, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections (He et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1311.pdf
Code
 jxhe/cross-lingual-struct-flow