Distributed Representations of Words and Phrases and their Compositionality

作者： Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean (2013)

arXiv： 1310.4546

领域

架构

TLDR（中文）

word2vec 的 NeurIPS 版正篇，引入 Negative Sampling、Hierarchical Softmax 与 phrase-level 向量。之后 GloVe / fastText / 大模型 embedding 层的训练目标都受其影响。

TLDR (English)

The NeurIPS version of word2vec, introducing Negative Sampling, Hierarchical Softmax, and phrase-level vectors. Influenced all subsequent embedding training objectives including GloVe, fastText, and modern LLM embedding layers.

出现在这些文章里

• Embeddings：把离散符号放进连续空间
• Embeddings: Putting Discrete Symbols into Continuous Space

同被引用

这些论文与本文出现在同一篇文章中

• ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators Kevin Clark et al. (2020)
• Convolutional Neural Networks for Sentence Classification Yoon Kim (2014)
• Efficient Estimation of Word Representations in Vector Space Tomas Mikolov et al. (2013)
• GloVe: Global Vectors for Word Representation Jeffrey Pennington et al. (2014)

相关论文

同一领域的其他论文

• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model DeepSeek-AI (2024)
  架构推理
• DeepSeek-V3 Technical Report DeepSeek-AI (2024)
  架构混合专家
• Mixtral of Experts Albert Q. Jiang et al. (2024)
  混合专家架构
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints Joshua Ainslie et al. (2023)
  推理架构
• GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng et al. (2022)
  架构
• Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press et al. (2021)
  架构长上下文