Coreference — End-to-end Neural Coreference Resolution
Optional · paper guide 1 of 1 for NLP 12 · Syntactic and Semantic Tasks. Citation numbers refer to the reference list at the bottom of this page.
Navigate: this page and this track: Navigate: this page and this track
Challenge
Coreference resolution must decide which text spans refer to the same entity, including spans that were not supplied by a separate mention detector. Lee, He, Lewis and Zettlemoyer propose jointly learning span representations, mention scores and antecedent decisions [19]. Their thesis is that document-level coreference can be trained from clusters while considering candidate spans directly. First understand this chapter’s spans, mentions, antecedents and clustering; an entity label on one token is not yet a coreference link.
Method

A span representation combines bidirectional recurrent endpoint states, an attention-weighted representation of words inside the span, and span-width features. The learned attention acts as a soft head; it does not require gold syntactic head annotations. A mention scorer prunes candidates before antecedent pairs are evaluated.

For a retained span, score earlier candidates using both mention quality and pair compatibility:
Here is the document, the current span, a non-dummy earlier span, a mention score and a pairwise antecedent score. is the selected antecedent, while contains available earlier candidates plus dummy ; indexes that set. The dummy uses instead of the additive formula. Its role includes starting a new entity cluster.
Training maximizes the sum of probabilities assigned to all correct antecedents, since a gold cluster can contain several valid earlier mentions. It does not force one arbitrary gold link. If no gold antecedent survives, the dummy is used. At inference, chosen antecedent links connect mentions into clusters; errors in early candidate pruning cannot be repaired by the later scorer.
Results
Table 1 reports English CoNLL-2012 test performance as the average F1 of MUC, B³ and CEAFφ4: 67.2 for the single model and 68.8 for a five-model ensemble. These are cluster metrics, not token accuracy or mention-detection F1. The ensemble and single-model results have different compute costs. Compare under the same evaluation script and mention setting before assigning significance to a small score difference.
Ablations
Table 2 uses development data: the full single model scores 67.7, removing head attention gives 66.4, and removing distance/width features gives 63.9. These drops must use 67.7 as their baseline, not the 67.2 test score. The system also limits span width to 10 words, retains up to 0.4 times the document’s token count in spans, and caps antecedents at 250. Those pruning choices control cost while potentially removing correct mentions or links.
Conclusion
The important connection is between representation learning and a structured decision over candidates. Trace “the company” through span construction, pruning and antecedent scoring in the two figures. Then inspect the chapter’s coreference evaluation discussion: detecting a mention, selecting its antecedent and forming the right cluster are three different places where errors can occur.
Problems
Pruning imposes a recall ceiling, and the dummy combines several situations rather than expressing a complete uncertainty model. Document genre, speaker metadata and annotation conventions affect transfer. A high-scoring link can still be semantically wrong, especially when resolving it requires knowledge absent from the text. End-to-end training removes a separately supervised mention pipeline; it does not remove feature choices, search approximations or dataset assumptions.
References
- Susan McRoy, Principles of Natural Language Processing, Chapters 3–5 and 7 and the chart-parsing appendix, open textbook, 2021. Open textbook
- Daniel Jurafsky and James H. Martin, Speech and Language Processing, 3rd ed. draft (August 19, 2026), Chapters 19, 20, 22 and 24. Book site
- E. Black, S. Abney, D. Flickenger, C. Gdaniec, R. Grishman, P. Harrison, D. Hindle, R. Ingria, F. Jelinek, J. Klavans, M. Liberman, M. Marcus, S. Roukos, B. Santorini and T. Strzalkowski, “A Procedure for Quantitatively Comparing the Syntactic Coverage of English Grammars,” Speech and Natural Language Workshop, 1991. Paper
- Mitchell P. Marcus, Beatrice Santorini and Mary Ann Marcinkiewicz, “Building a Large Annotated Corpus of English: The Penn Treebank,” Computational Linguistics 19(2), 1993. Paper
- Mitchell Stern, Jacob Andreas and Dan Klein, “A Minimal Span-Based Neural Constituency Parser,” ACL, 2017. Paper
- Nikita Kitaev and Dan Klein, “Constituency Parsing with a Self-Attentive Encoder,” ACL, 2018. Paper
- Nikita Kitaev, Steven Cao and Dan Klein, “Multilingual Constituency Parsing with Self-Attention and Pre-Training,” ACL, 2019. Paper
- Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre and Daniel Zeman, “Universal Dependencies,” Computational Linguistics 47(2), 2021. Paper
- Joakim Nivre, “Algorithms for Deterministic Incremental Dependency Parsing,” Computational Linguistics 34(4), 2008. Paper
- Danqi Chen and Christopher Manning, “A Fast and Accurate Dependency Parser using Neural Networks,” EMNLP, 2014. Paper
- Ryan McDonald, Fernando Pereira, Kiril Ribarov and Jan Hajič, “Non-Projective Dependency Parsing using Spanning Tree Algorithms,” HLT-EMNLP, 2005. Paper
- Eliyahu Kiperwasser and Yoav Goldberg, “Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations,” Transactions of the ACL 4, 2016. Paper
- Timothy Dozat and Christopher D. Manning, “Deep Biaffine Attention for Neural Dependency Parsing,” ICLR, 2017. Paper
- Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre and Slav Petrov, “CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies,” CoNLL Shared Task, 2018. Paper
- Daniel Gildea and Daniel Jurafsky, “Automatic Labeling of Semantic Roles,” Computational Linguistics 28(3), 2002. Paper
- Martha Palmer, Daniel Gildea and Paul Kingsbury, “The Proposition Bank: An Annotated Corpus of Semantic Roles,” Computational Linguistics 31(1), 2005. Paper
- Xavier Carreras and Lluís Màrquez, “Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling,” CoNLL, 2005. Paper
- Luheng He, Kenton Lee, Mike Lewis and Luke Zettlemoyer, “Deep Semantic Role Labeling: What Works and What’s Next,” ACL, 2017. Paper
- Kenton Lee, Luheng He, Mike Lewis and Luke Zettlemoyer, “End-to-end Neural Coreference Resolution,” EMNLP, 2017. Paper
- Marc Vilain, John Burger, John Aberdeen, Dennis Connolly and Lynette Hirschman, “A Model-Theoretic Coreference Scoring Scheme,” MUC-6, 1995. Paper
- Xiaoqiang Luo, “On Coreference Resolution Performance Metrics,” HLT-EMNLP, 2005. Paper
- Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng and Michael Strube, “Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation,” ACL, 2014. Paper
- Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina and Yuchen Zhang, “CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes,” EMNLP-CoNLL Shared Task, 2012. Paper
- Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer and Nathan Schneider, “Abstract Meaning Representation for Sembanking,” Linguistic Annotation Workshop, 2013. Paper
- Shu Cai and Kevin Knight, “Smatch: an Evaluation Metric for Semantic Feature Structures,” ACL, 2013. Paper
- Mandar Joshi, Omer Levy, Luke Zettlemoyer and Daniel Weld, “BERT for Coreference Resolution: Baselines and Analysis,” EMNLP-IJCNLP, 2019. Paper
- Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer and Omer Levy, “SpanBERT: Improving Pre-training by Representing and Predicting Spans,” Transactions of the ACL 8, 2020. Paper
- John Hewitt and Christopher D. Manning, “A Structural Probe for Finding Syntax in Word Representations,” NAACL-HLT, 2019. Paper
- Ian Tenney, Dipanjan Das and Ellie Pavlick, “BERT Rediscovers the Classical NLP Pipeline,” ACL, 2019. Paper
- Wayne Xin Zhao et al., “A Survey of Large Language Models,” Frontiers of Computer Science, 2026; online manuscript updated March 2026. Survey
- Meishan Zhang, “A survey of syntactic-semantic parsing based on constituent and dependency structures,” Science China Technological Sciences 63, 2020. Survey
- Massimo Poesio, Juntao Yu, Silviu Paun, Abdulrahman Aloraini, Pengcheng Lu, Janosch Haber and Derya Cokal, “Computational Models of Anaphora,” Annual Review of Linguistics 9, 2023. Survey
- Yupeng Chang et al., “A Survey on Evaluation of Large Language Models,” ACM Transactions on Intelligent Systems and Technology, 2024. Survey
- Huiyao Chen, Meishan Zhang, Jing Li, Lilja Øvrelid, Jan Hajič, Hao Fei and Min Zhang, “A Systematic Survey of Semantic Role Labeling in the Era of Pretrained Language Models,” arXiv:2502.08660, 2025; accepted at Artificial Intelligence Review. Survey
- Yuanhe Tian, Fei Xia and Yan Song, “Large Language Models are No Longer Shallow Parsers,” ACL, 2024. Paper
- John T. Hale and Miloš Stanojević, “Do LLMs learn a true syntactic universal?” EMNLP, 2024. Paper
- Nghia T. Le and Alan Ritter, “Are Large Language Models Robust Coreference Resolvers?” arXiv:2305.14489, 2023. Paper
- Ian Porada, Xiyuan Zou and Jackie Chi Kit Cheung, “A Controlled Reevaluation of Coreference Resolution Models,” LREC-COLING, 2024. Paper
- Zhijing Jin, Yuen Chen, Fernando Gonzalez Adauto, Jiarui Liu, Jiayi Zhang, Julian Michael, Bernhard Schölkopf and Mona Diab, “Analyzing the Role of Semantic Representations in the Era of Large Language Models,” NAACL, 2024. Paper
- NLTK contributors,
nltk.grammar.PCFG, API documentation, accessed 2026-11-05. Normalization contract