Full search page →

Paper guides

Optional reading guides to the papers behind NLP 02 · Linguistic Essentials, one page each: the problem, the idea, what it showed and what to check when you read it.

Navigate: this page and this track: Navigate: this page and this track

These optional guides connect the chapter to original papers. Read the challenge and figure first, then unfold the method, evidence and limitations.

  1. BPE — Neural Machine Translation of Rare Words with Subword Units

    A fixed word vocabulary turns a rare compound or unfamiliar name into an unknown token.

References

  1. Masato Hagiwara, Real-World Natural Language Processing: Practical Applications with Deep Learning, Manning, 2021. Publisher
  2. Susan McRoy, Principles of Natural Language Processing, University of Wisconsin–Milwaukee open textbook, 2021. Grammars and Syntactic Processing
  3. Catherine Anderson, Bronwyn Bjorkman, Derek Denis, Julianne Doner, Margaret Grant, Nathan Sanders and Ai Taniguchi, Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Open textbook
  4. Catherine Anderson et al., “3.1 Modality,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
  5. Catherine Anderson et al., “4.1 Phonemes and allophones,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
  6. Catherine Anderson et al., “3.12 Tone and intonation,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
  7. Catherine Anderson et al., “5.1 What is morphology?,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
  8. Catherine Anderson et al., “6.4 Identifying phrases: Constituency tests,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
  9. Catherine Anderson et al., “8.4 Conversational implicatures,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
  10. Bolei Ma, Yuting Li, Wei Zhou, Ziwei Gong, Yang Janet Liu, Katja Jasinskaja, Annemarie Friedrich, Julia Hirschberg, Frauke Kreuter and Barbara Plank, “Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges,” ACL, pp. 8679–8696, 2025. Paper
  11. Catherine Anderson et al., “10.2 Language varies,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
  12. Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che and Philip S. Yu, “A survey of multilingual large language models,” Patterns, 6(1), 101118, 2025. Paper
  13. Yanru Lu and Stefan Müller, “Deliminative verbal reduplication in Mandarin Chinese,” Journal of Linguistics, 62(3), pp. 595–632, 2026. Paper
  14. A. Seza Doğruöz, Sunayana Sitaram, Barbara E. Bullock and Almeida Jacqueline Toribio, “A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies,” ACL-IJCNLP, pp. 1654–1666, 2021. Paper
  15. Genta Winata, Alham Fikri Aji, Zheng Xin Yong and Thamar Solorio, “The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges,” Findings of ACL, pp. 2936–2978, 2023. Paper
  16. M. F. Porter, “An algorithm for suffix stripping,” Program, 14(3), pp. 130–137, 1980. DOI
  17. Rico Sennrich, Barry Haddow and Alexandra Birch, “Neural Machine Translation of Rare Words with Subword Units,” ACL, 2016. Paper
  18. Mike Schuster and Kaisuke Nakajima, “Japanese and Korean Voice Search,” ICASSP, pp. 5149–5152, 2012. DOI
  19. Yonghui Wu et al., “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation,” arXiv:1609.08144, 2016. Paper
  20. Taku Kudo, “Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates,” ACL, 2018. Paper
  21. Taku Kudo and John Richardson, “SentencePiece: A Simple and Language Independent Subword Tokenizer and Detokenizer for Neural Text Processing,” EMNLP System Demonstrations, 2018. Paper
  22. Sabrina J. Mielke et al., “Between Words and Characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP,” arXiv:2112.10508, 2021. Review
  23. Hugging Face, “The Tokenization Pipeline,” Tokenizers documentation, current online edition. Documentation
  24. Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre and Daniel Zeman, “Universal Dependencies,” Computational Linguistics, 47(2), 2021. Paper
  25. Universal Dependencies contributors, “Universal POS tags,” Universal Dependencies documentation, current online edition. Documentation
  26. Mitchell P. Marcus, Beatrice Santorini and Mary Ann Marcinkiewicz, “Building a Large Annotated Corpus of English: The Penn Treebank,” Computational Linguistics, 19(2), 1993. Paper
  27. Thomas Emerson, “The Second International Chinese Word Segmentation Bakeoff,” Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing, 2005. Paper
  28. Jacob Cohen, “A Coefficient of Agreement for Nominal Scales,” Educational and Psychological Measurement, 20(1), pp. 37–46, 1960. DOI
  29. Steven T. Piantadosi, “Zipf’s word frequency law in natural language: A critical review and future directions,” Psychonomic Bulletin & Review, 21(5), pp. 1112–1130, 2014. DOI
  30. Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” NAACL-HLT, 2019. Paper