Full search page →

Paper guides

Optional reading guides to the papers behind NLP 08 · Sequence-to-Sequence Models with Transformers (Machine Translation, Chatbots), one page each: the problem, the idea, what it showed and what to check when you read it.

Navigate: this page and this track: Navigate: this page and this track

These optional guides connect the chapter to original papers. Read the challenge and figure first, then unfold the method, evidence and limitations.

  1. Bahdanau attention — Jointly Learning to Align and Translate

    A fixed-vector encoder–decoder must compress the source before generating any translation.

References

  1. M. Hagiwara, Real-World Natural Language Processing, Manning, 2021, Chapters 6 and 8. Book · Sequence-to-sequence chapter
  2. S. McRoy, Principles of Natural Language Processing, open textbook, 2021, Chapter 9. Book · Generation chapter
  3. P. F. Brown, S. A. Della Pietra, V. J. Della Pietra and R. L. Mercer, “The Mathematics of Statistical Machine Translation: Parameter Estimation,” Computational Linguistics 19(2), 1993. Paper
  4. P. Koehn, F. J. Och and D. Marcu, “Statistical Phrase-Based Translation,” HLT-NAACL, 2003. Paper
  5. K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk and Y. Bengio, “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation,” EMNLP, 2014. Paper
  6. I. Sutskever, O. Vinyals and Q. V. Le, “Sequence to Sequence Learning with Neural Networks,” NeurIPS, 2014. Paper
  7. D. Bahdanau, K. Cho and Y. Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate,” ICLR, 2015. Paper
  8. A. Vaswani et al., “Attention Is All You Need,” NeurIPS, 2017. Paper
  9. R. Sennrich, B. Haddow and A. Birch, “Neural Machine Translation of Rare Words with Subword Units,” ACL, 2016. Paper
  10. M. Ranzato, S. Chopra, M. Auli and W. Zaremba, “Sequence Level Training with Recurrent Neural Networks,” ICLR, 2016. Paper
  11. S. Bengio, O. Vinyals, N. Jaitly and N. Shazeer, “Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks,” NeurIPS, 2015. Paper
  12. Y. Wu et al., “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation,” arXiv:1609.08144, 2016. Paper
  13. K. Papineni, S. Roukos, T. Ward and W.-J. Zhu, “Bleu: a Method for Automatic Evaluation of Machine Translation,” ACL, 2002. Paper
  14. M. Post, “A Call for Clarity in Reporting BLEU Scores,” WMT, 2018. Paper
  15. M. Popović, “chrF: character n-gram F-score for automatic MT evaluation,” WMT, 2015. Paper
  16. R. Rei, C. Stewart, A. C. Farinha and A. Lavie, “COMET: A Neural Framework for MT Evaluation,” EMNLP, 2020. Paper
  17. C.-Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,” Text Summarization Branches Out, 2004. Paper
  18. M. Freitag et al., “Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More Robust,” WMT, 2022. Report
  19. A. Lavie et al., “Findings of the WMT25 Shared Task on Automated Translation Evaluation Systems: Linguistic Diversity is Challenging and References Still Help,” WMT, 2025. Report
  20. T. Kocmi et al., “Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet,” WMT, 2024. Report
  21. NLLB Team et al., “Scaling neural machine translation to 200 languages,” Nature 630, 2024. DOI
  22. F. Stahlberg, “Neural Machine Translation: A Review,” Journal of Artificial Intelligence Research 69, 2020. DOI
  23. O. Vinyals and Q. V. Le, “A Neural Conversational Model,” ICML Deep Learning Workshop, 2015. Paper
  24. J. Li, M. Galley, C. Brockett, J. Gao and B. Dolan, “A Diversity-Promoting Objective Function for Neural Conversation Models,” NAACL-HLT, 2016. Paper
  25. J. Deriu, Á. Rodrigo, A. Otegi, G. Echegoyen, S. Rosset, E. Agirre and M. Cieliebak, “Survey on Evaluation Methods for Dialogue Systems,” Artificial Intelligence Review 54, 2021. Survey
  26. J. Ni, T. Young, V. Pandelea, F. Xue and E. Cambria, “Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey,” Artificial Intelligence Review 56, 2023. Survey
  27. Y. Qin et al., “Tool Learning with Foundation Models,” ACM Computing Surveys 57, 2024. Author manuscript · Publication
  28. P. Schmidtova et al., “Do My Eyes Deceive Me? A Survey of Human Evaluations of Hallucinations in NLG,” INLG, 2025. Survey
  29. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” CVPR, 2016. Paper
  30. M. Freitag, G. Foster, D. Grangier, V. Ratnakar, Q. Tan and W. Macherey, “Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation,” Transactions of the Association for Computational Linguistics 9, 2021. DOI