完整搜尋頁面 →

論文導讀

選讀:NLP 04・詞彙語意與詞嵌入 所依據論文的導讀,每篇一頁:說明它要解決的問題、核心想法、結果,以及閱讀時該注意什麼。

導覽:本頁與本系列: 導覽:本頁與本系列

以下為選讀導讀,將本章連到原始論文。先讀挑戰與圖,再逐層展開方法、證據及限制。

  1. word2vec — Efficient Estimation of Word Representations in Vector Space

    替每個詞配置向量,仍需要訓練目標讓座標具有意義。

  2. GloVe — Global Vectors for Word Representation

    預測式嵌入從局部視窗學習,計數方法則明確整理語料統計。

參考文獻

  1. M. Hagiwara, Real-World Natural Language Processing: Practical Applications with Deep Learning, Manning, 2021; Chapters 3–4. publisher
  2. S. McRoy, Principles of Natural Language Processing, open textbook, 2021; Chapter 6, “Benchmark Tasks for Language Modelling.” chapter
  3. Y. Bengio, R. Ducharme, P. Vincent and C. Jauvin, “A neural probabilistic language model,” Journal of Machine Learning Research 3, 2003. paper
  4. G. A. Miller, “WordNet: a lexical database for English,” Communications of the ACM 38(11), 1995. paper
  5. Z. S. Harris, “Distributional structure,” Word 10(2–3), 1954. paper
  6. P. D. Turney and P. Pantel, “From frequency to meaning: vector space models of semantics,” Journal of Artificial Intelligence Research 37, 2010. survey
  7. T. Mikolov, K. Chen, G. Corrado and J. Dean, “Efficient estimation of word representations in vector space,” ICLR Workshop, 2013. paper
  8. T. Mikolov, I. Sutskever, K. Chen, G. Corrado and J. Dean, “Distributed representations of words and phrases and their compositionality,” Advances in Neural Information Processing Systems 26, 2013. paper
  9. O. Levy and Y. Goldberg, “Neural word embedding as implicit matrix factorization,” Advances in Neural Information Processing Systems 27, 2014. paper
  10. J. Pennington, R. Socher and C. D. Manning, “GloVe: global vectors for word representation,” EMNLP, 2014. paper
  11. P. Bojanowski, E. Grave, A. Joulin and T. Mikolov, “Enriching word vectors with subword information,” Transactions of the Association for Computational Linguistics 5, 2017. paper
  12. L. Finkelstein, E. Gabrilovich, Y. Matias, E. Rivlin, Z. Solan, G. Wolfman and E. Ruppin, “Placing search in context: the concept revisited,” ACM Transactions on Information Systems 20(1), 2002. paper
  13. F. Hill, R. Reichart and A. Korhonen, “SimLex-999: evaluating semantic models with (genuine) similarity estimation,” Computational Linguistics 41(4), 2015. paper
  14. T. Mikolov, W.-t. Yih and G. Zweig, “Linguistic regularities in continuous space word representations,” NAACL-HLT, 2013. paper
  15. T. Linzen, “Issues in evaluating semantic spaces using word analogies,” Workshop on Evaluating Vector-Space Representations for NLP (RepEval), 2016. paper
  16. T. Schnabel, I. Labutov, D. Mimno and T. Joachims, “Evaluation methods for unsupervised word embeddings,” EMNLP, 2015. paper
  17. T. Bolukbasi, K.-W. Chang, J. Zou, V. Saligrama and A. Kalai, “Man is to computer programmer as woman is to homemaker? Debiasing word embeddings,” Advances in Neural Information Processing Systems 29, 2016. paper
  18. A. Caliskan, J. J. Bryson and A. Narayanan, “Semantics derived automatically from language corpora contain human-like biases,” Science 356(6334), 2017. paper
  19. H. Gonen and Y. Goldberg, “Lipstick on a pig: debiasing methods cover up systematic gender biases in word embeddings but do not remove them,” NAACL-HLT, 2019. paper
  20. N. Reimers and I. Gurevych, “Sentence-BERT: sentence embeddings using Siamese BERT-networks,” EMNLP-IJCNLP, 2019. paper
  21. T. Gao, X. Yao and D. Chen, “SimCSE: simple contrastive learning of sentence embeddings,” EMNLP, 2021. paper
  22. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee and L. Zettlemoyer, “Deep contextualized word representations,” NAACL-HLT, 2018. paper
  23. J. Devlin, M.-W. Chang, K. Lee and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” NAACL-HLT, 2019. paper
  24. M. Apidianaki, “From word types to tokens and back: a survey of approaches to word meaning representation and interpretation,” Computational Linguistics 49(2), 2023. survey
  25. N. Muennighoff, N. Tazi, L. Magne and N. Reimers, “MTEB: massive text embedding benchmark,” EACL, 2023. paper