L0-regularization induces subregular biases in LSTMs
- Charles J Torres (University of California, Irvine)
- Richard Futrell (University of California, Irvine)
Abstract
Ongoing work attempts to identify the formal language patterns in natural language. In phonology, recent work has identified the subregular languages as a good candidate (Heinz, 2018). However, there remain few explanations for the source of this bias. This abstract proposes a means of investigating formal language learnability. We propose using a variant of minimum description length (MDL) as defined on LSTMs with varying priors on LSTM size. We will show its utility on a test case from Heinz and Idsardi (2013) and Rawski et al. (2017).
Keywords: formal language theory, computational complexity, subregularity, phonology
How to Cite:
Torres, C. J. & Futrell, R., (2023) “L0-regularization induces subregular biases in LSTMs”, Society for Computation in Linguistics 6(1), 394-396. doi: https://doi.org/10.7275/5ss3-d749
Downloads:
Download PDF