Neural Networks Can Learn Patterns of Island-insensitivity in Norwegian
- Anastasia Kobzeva (Norwegian University of Science and Technology)
- Suhas Arehalli (Johns Hopkins University)
- Tal Linzen (New York University)
- Dave Kush (University of Toronto)
Abstract
Recent research suggests that Recurrent Neural Networks (RNNs) can capture abstract generalizations about filler-gap dependencies (FGDs) in English and so-called island constraints on their distribution (Wilcox et al., 2018; 2021). These results have been interpreted as evidence that it is possible, in principle, to induce complex syntactic knowledge from the input without domain-specific learning biases. However, the English results alone do not establish that island constraints were induced from distributional properties of the training data instead of simply reflecting architectural limitations independent of the input to the models. We address this concern by investigating whether such models can learn the distribution of acceptable FGDs in Norwegian, a language that is sensitive to fewer islands than English (Christensen, 1982). Results from five experiments show that Long Short-Term Memory (LSTM) RNNs can (i) learn that Norwegian FGD formation is unbounded, (ii) recover the island status of temporal adjunct and subject islands, and (iii) learn that Norwegian, unlike English, permits FGDs into two types of embedded questions. The fact that LSTM RNNs can learn cross-linguistic differences in island facts therefore strengthens the claim that RNN language models can induce the constraints from patterns in the input.
Keywords: Island Constraints, Filler-Gap Dependencies, Neural Language Models, Norwegian
How to Cite:
Kobzeva, A., Arehalli, S., Linzen, T. & Kush, D., (2023) “Neural Networks Can Learn Patterns of Island-insensitivity in Norwegian”, Society for Computation in Linguistics 6(1), 175-185. doi: https://doi.org/10.7275/qb8z-qc91
Downloads:
Download PDF