Paper

A LSTM language model learns Hindi-Urdu case-agreement interactions, and has a linear encoding of case

---

Abstract

Much evaluation work in the literature shows that neural language models seem capable of capturing syntactic dependencies in natural languages, but they usually look at relatively simple syntactic phenomena.  We show that a two-layer LSTM language model trained on 250M morphemes of Hindi data can capture the relatively complex interaction between case and agreement in Hindi-Urdu, at an accuracy of 81.17%.  Furthermore, we show that this model encodes case-marking linearly, implementing a geometrically intuitive and interpretable syntactic processing mechanism.  We also show that this model doesn't calculate agreement extremely eagerly, as case information seems to be persistent over time as a sentence unfolds.  This is surprising given LSTMs autoregressive and recurrent nature, which should exert an incremental processing pressure onto our model.

Keywords: Hindi, Hindi-Urdu, syntax, case, agreement, LSTM, intervention

How to Cite:

Ozaki, S., Bhatt, R. & Dillon, B., (2025) “A LSTM language model learns Hindi-Urdu case-agreement interactions, and has a linear encoding of case”, Society for Computation in Linguistics 8(1): 21. doi: https://doi.org/10.7275/scil.3176