Abstract

L0-regularization induces subregular biases in LSTMs

Authors
  • Charles J Torres (University of California, Irvine)
  • Richard Futrell (University of California, Irvine)

Abstract

Ongoing work attempts to identify the formal language patterns in natural language. In phonology, recent work has identified the subregular languages as a good candidate (Heinz, 2018). However, there remain few explanations for the source of this bias. This abstract proposes a means of investigating formal language learnability. We propose using a variant of minimum description length (MDL) as defined on LSTMs with varying priors on LSTM size. We will show its utility on a test case from Heinz and Idsardi (2013) and Rawski et al. (2017).

Keywords: formal language theory, computational complexity, subregularity, phonology

How to Cite:

Torres, C. J. & Futrell, R., (2023) “L0-regularization induces subregular biases in LSTMs”, Society for Computation in Linguistics 6(1), 394-396. doi: https://doi.org/10.7275/5ss3-d749

Downloads:
Download PDF

95 Views

29 Downloads

Published on
01 Jun 2023