GloVe: Global Vectors for Word Representation

Authors: Jeffrey Pennington, Richard Socher, Christopher D. Manning (2014)

Domains

Architecture

TLDR (English)

GloVe learns word vectors by factorizing word co-occurrence matrices, combining the advantages of count-based methods (LSA) and prediction-based methods (Word2Vec). It achieved state-of-the-art on word analogy and similarity tasks and remains a widely used baseline word vector in academia.

TLDR（中文）

GloVe 通过分解词共现矩阵来学习词向量，结合了基于计数的方法（LSA）和基于预测的方法 （Word2Vec）的优点。在词类比和词相似度任务上达到了当时最先进的性能， 是学术界广泛使用的基线词向量。

Appears in These Articles

• Embeddings：把离散符号放进连续空间
• Embeddings: Putting Discrete Symbols into Continuous Space

Co-cited Papers

These papers appear in the same articles as this one

• Efficient Estimation of Word Representations in Vector Space Tomas Mikolov et al. (2013)
• Deep contextualized word representations Matthew E. Peters et al. (2018)

Related Papers

Other papers in the same domain

• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model (2024)
  ArchitectureInference
• DeepSeek-V3 Technical Report (2024)
  ArchitectureMixture of Experts
• Mixtral of Experts Albert Q. Jiang et al. (2024)
  Mixture of ExpertsArchitecture
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints Joshua Ainslie et al. (2023)
  InferenceArchitecture
• GLM-130B: An Open Bilingual Pre-trained Model (2022)
  Architecture
• Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (2021)
  ArchitectureLong Context