Paper

On the Dangers of Naïve Replication: The Case of Implicature

---

Abstract

Other people’s code, data, and definition of a language task often provide the groundwork for new research efforts. The work we present here began as a straightforward investigation of conversational implicature, a central aspect of natural dialogue, starting with updating a prior method to employ more recent LLMs. But differences in results with the work we were replicating led to a deep dive into why those differences were occurring, and this led us to consider more carefully what it means to begin working on a topic with prior work “as a starting point”. We describe our process, what we found, and lessons suggested about data quality, task definition, and the current pace of change in NLP.

Keywords: language model, implicature, replication

How to Cite:

Korde, A. & Resnik, P., (2025) “On the Dangers of Naïve Replication: The Case of Implicature”, Society for Computation in Linguistics 8(1): 28. doi: https://doi.org/10.7275/scil.3192