Probing RNN Encoder-Decoder Generalization of Subregular Functions using Reduplication
- Max Nelson (University of Massachusetts Amherst)
- Hossep Dolatian (Stony Brook University)
- Jonathan Rawski (Stony Brook University)
- Brandon Prickett (University of Massachusetts Amherst)
Abstract
This paper examines the generalization abilities of encoder-decoder networks on a class of subregular functions characteristic of natural language reduplication. We find that, for the simulations we run, attention is a necessary and sufficient mechanism for learning generalizable reduplication. We examine attention alignment to connect RNN computation to a class of 2-way transducers.
Keywords: encoder-decoder, neural network, reduplication, seq2seq, finite-state tranducer, complexity, attention
How to Cite:
Nelson, M., Dolatian, H., Rawski, J. & Prickett, B., (2020) “Probing RNN Encoder-Decoder Generalization of Subregular Functions using Reduplication”, Society for Computation in Linguistics 3(1), 31-42. doi: https://doi.org/10.7275/xd0r-pg04
Downloads:
Download PDF