Paper

Interpreting Sequence-to-Sequence Models for Russian Inflectional Morphology

---

Abstract

Morphological inflection, as an engineering task in NLP, has seen a rise in the use of neural sequence-to-sequence models (Kann et al. 2016, Cotterell et al. 2018, Aharoni et al. 2017). While these outperform traditional systems based on edit rule induction, it is hard to interpret what they are learning in linguistic terms. We propose a new method of analyzing morphological sequence-to-sequence models which groups errors into linguistically meaningful classes, making what the model learns more transparent. As a case study, we analyze a seq2seq model on Russian, finding that semantic and lexically conditioned allomorphy (e.g. inanimate nouns like zavod `factory\' and animates like otec `father\' have different, animacy-conditioned accusative forms) are responsible for its relatively low accuracy. Augmenting the model with word embeddings as a proxy for lexical semantics leads to significant improvements in predicted wordform accuracy.

Keywords: morphology, sequence-to-sequence, interpretability, error analysis

How to Cite:

King, D. L., Sims, A. D. & Elsner, M., (2020) “Interpreting Sequence-to-Sequence Models for Russian Inflectional Morphology”, Society for Computation in Linguistics 3(1), 402-411. doi: https://doi.org/10.7275/4pxd-zc54