Paper guides
Optional reading guides to the papers behind NLP 08 · Sequence-to-Sequence Models with Transformers (Machine Translation, Chatbots), one page each: the problem, the idea, what it showed and what to check when you read it.
Navigate: this page and this track: Navigate: this page and this track
These optional guides connect the chapter to original papers. Read the challenge and figure first, then unfold the method, evidence and limitations.
- Bahdanau attention — Jointly Learning to Align and Translate
A fixed-vector encoder–decoder must compress the source before generating any translation.
References
- M. Hagiwara, Real-World Natural Language Processing, Manning, 2021, Chapters 6 and 8. Book · Sequence-to-sequence chapter
- S. McRoy, Principles of Natural Language Processing, open textbook, 2021, Chapter 9. Book · Generation chapter
- P. F. Brown, S. A. Della Pietra, V. J. Della Pietra and R. L. Mercer, “The Mathematics of Statistical Machine Translation: Parameter Estimation,” Computational Linguistics 19(2), 1993. Paper
- P. Koehn, F. J. Och and D. Marcu, “Statistical Phrase-Based Translation,” HLT-NAACL, 2003. Paper
- K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk and Y. Bengio, “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation,” EMNLP, 2014. Paper
- I. Sutskever, O. Vinyals and Q. V. Le, “Sequence to Sequence Learning with Neural Networks,” NeurIPS, 2014. Paper
- D. Bahdanau, K. Cho and Y. Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate,” ICLR, 2015. Paper
- A. Vaswani et al., “Attention Is All You Need,” NeurIPS, 2017. Paper
- R. Sennrich, B. Haddow and A. Birch, “Neural Machine Translation of Rare Words with Subword Units,” ACL, 2016. Paper
- M. Ranzato, S. Chopra, M. Auli and W. Zaremba, “Sequence Level Training with Recurrent Neural Networks,” ICLR, 2016. Paper
- S. Bengio, O. Vinyals, N. Jaitly and N. Shazeer, “Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks,” NeurIPS, 2015. Paper
- Y. Wu et al., “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation,” arXiv:1609.08144, 2016. Paper
- K. Papineni, S. Roukos, T. Ward and W.-J. Zhu, “Bleu: a Method for Automatic Evaluation of Machine Translation,” ACL, 2002. Paper
- M. Post, “A Call for Clarity in Reporting BLEU Scores,” WMT, 2018. Paper
- M. Popović, “chrF: character n-gram F-score for automatic MT evaluation,” WMT, 2015. Paper
- R. Rei, C. Stewart, A. C. Farinha and A. Lavie, “COMET: A Neural Framework for MT Evaluation,” EMNLP, 2020. Paper
- C.-Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,” Text Summarization Branches Out, 2004. Paper
- M. Freitag et al., “Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More Robust,” WMT, 2022. Report
- A. Lavie et al., “Findings of the WMT25 Shared Task on Automated Translation Evaluation Systems: Linguistic Diversity is Challenging and References Still Help,” WMT, 2025. Report
- T. Kocmi et al., “Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet,” WMT, 2024. Report
- NLLB Team et al., “Scaling neural machine translation to 200 languages,” Nature 630, 2024. DOI
- F. Stahlberg, “Neural Machine Translation: A Review,” Journal of Artificial Intelligence Research 69, 2020. DOI
- O. Vinyals and Q. V. Le, “A Neural Conversational Model,” ICML Deep Learning Workshop, 2015. Paper
- J. Li, M. Galley, C. Brockett, J. Gao and B. Dolan, “A Diversity-Promoting Objective Function for Neural Conversation Models,” NAACL-HLT, 2016. Paper
- J. Deriu, Á. Rodrigo, A. Otegi, G. Echegoyen, S. Rosset, E. Agirre and M. Cieliebak, “Survey on Evaluation Methods for Dialogue Systems,” Artificial Intelligence Review 54, 2021. Survey
- J. Ni, T. Young, V. Pandelea, F. Xue and E. Cambria, “Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey,” Artificial Intelligence Review 56, 2023. Survey
- Y. Qin et al., “Tool Learning with Foundation Models,” ACM Computing Surveys 57, 2024. Author manuscript · Publication
- P. Schmidtova et al., “Do My Eyes Deceive Me? A Survey of Human Evaluations of Hallucinations in NLG,” INLG, 2025. Survey
- C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” CVPR, 2016. Paper
- M. Freitag, G. Foster, D. Grangier, V. Ratnakar, Q. Tan and W. Macherey, “Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation,” Transactions of the Association for Computational Linguistics 9, 2021. DOI