Paper guides
Optional reading guides to the papers behind NLP 02 · Linguistic Essentials, one page each: the problem, the idea, what it showed and what to check when you read it.
Navigate: this page and this track: Navigate: this page and this track
These optional guides connect the chapter to original papers. Read the challenge and figure first, then unfold the method, evidence and limitations.
- BPE — Neural Machine Translation of Rare Words with Subword Units
A fixed word vocabulary turns a rare compound or unfamiliar name into an unknown token.
References
- Masato Hagiwara, Real-World Natural Language Processing: Practical Applications with Deep Learning, Manning, 2021. Publisher
- Susan McRoy, Principles of Natural Language Processing, University of Wisconsin–Milwaukee open textbook, 2021. Grammars and Syntactic Processing
- Catherine Anderson, Bronwyn Bjorkman, Derek Denis, Julianne Doner, Margaret Grant, Nathan Sanders and Ai Taniguchi, Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Open textbook
- Catherine Anderson et al., “3.1 Modality,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
- Catherine Anderson et al., “4.1 Phonemes and allophones,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
- Catherine Anderson et al., “3.12 Tone and intonation,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
- Catherine Anderson et al., “5.1 What is morphology?,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
- Catherine Anderson et al., “6.4 Identifying phrases: Constituency tests,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
- Catherine Anderson et al., “8.4 Conversational implicatures,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
- Bolei Ma, Yuting Li, Wei Zhou, Ziwei Gong, Yang Janet Liu, Katja Jasinskaja, Annemarie Friedrich, Julia Hirschberg, Frauke Kreuter and Barbara Plank, “Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges,” ACL, pp. 8679–8696, 2025. Paper
- Catherine Anderson et al., “10.2 Language varies,” Essentials of Linguistics, 2nd ed., eCampusOntario, 2022. Chapter
- Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che and Philip S. Yu, “A survey of multilingual large language models,” Patterns, 6(1), 101118, 2025. Paper
- Yanru Lu and Stefan Müller, “Deliminative verbal reduplication in Mandarin Chinese,” Journal of Linguistics, 62(3), pp. 595–632, 2026. Paper
- A. Seza Doğruöz, Sunayana Sitaram, Barbara E. Bullock and Almeida Jacqueline Toribio, “A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies,” ACL-IJCNLP, pp. 1654–1666, 2021. Paper
- Genta Winata, Alham Fikri Aji, Zheng Xin Yong and Thamar Solorio, “The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges,” Findings of ACL, pp. 2936–2978, 2023. Paper
- M. F. Porter, “An algorithm for suffix stripping,” Program, 14(3), pp. 130–137, 1980. DOI
- Rico Sennrich, Barry Haddow and Alexandra Birch, “Neural Machine Translation of Rare Words with Subword Units,” ACL, 2016. Paper
- Mike Schuster and Kaisuke Nakajima, “Japanese and Korean Voice Search,” ICASSP, pp. 5149–5152, 2012. DOI
- Yonghui Wu et al., “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation,” arXiv:1609.08144, 2016. Paper
- Taku Kudo, “Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates,” ACL, 2018. Paper
- Taku Kudo and John Richardson, “SentencePiece: A Simple and Language Independent Subword Tokenizer and Detokenizer for Neural Text Processing,” EMNLP System Demonstrations, 2018. Paper
- Sabrina J. Mielke et al., “Between Words and Characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP,” arXiv:2112.10508, 2021. Review
- Hugging Face, “The Tokenization Pipeline,” Tokenizers documentation, current online edition. Documentation
- Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre and Daniel Zeman, “Universal Dependencies,” Computational Linguistics, 47(2), 2021. Paper
- Universal Dependencies contributors, “Universal POS tags,” Universal Dependencies documentation, current online edition. Documentation
- Mitchell P. Marcus, Beatrice Santorini and Mary Ann Marcinkiewicz, “Building a Large Annotated Corpus of English: The Penn Treebank,” Computational Linguistics, 19(2), 1993. Paper
- Thomas Emerson, “The Second International Chinese Word Segmentation Bakeoff,” Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing, 2005. Paper
- Jacob Cohen, “A Coefficient of Agreement for Nominal Scales,” Educational and Psychological Measurement, 20(1), pp. 37–46, 1960. DOI
- Steven T. Piantadosi, “Zipf’s word frequency law in natural language: A critical review and future directions,” Psychonomic Bulletin & Review, 21(5), pp. 1112–1130, 2014. DOI
- Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” NAACL-HLT, 2019. Paper