Effective Approaches to Attention-based Neural Machine Translation

作者： Minh-Thang Luong, Hieu Pham, Christopher D. Manning (2015)

arXiv： 1508.04025

领域

架构

TLDR（中文）

系统化地比较 global vs local attention、不同打分函数（dot / general / concat），是后人讲 "attention score 是怎么算的" 时最常引用的工程化版本。

TLDR (English)

Systematically compares global vs local attention and different scoring functions (dot/general/concat). The most commonly cited engineering reference when explaining "how attention scores are computed".

出现在这些文章里

• Attention：让每个位置选择上下文
• Attention: Choosing the Relevant Context

同被引用

这些论文与本文出现在同一篇文章中

• Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau et al. (2014)
• Neural Machine Translation in Linear Time Nal Kalchbrenner et al. (2016)
• Sequence to Sequence Learning with Neural Networks Ilya Sutskever et al. (2014)
• Attention Is All You Need Ashish Vaswani et al. (2017)

相关论文

同一领域的其他论文

• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model DeepSeek-AI (2024)
  架构推理
• DeepSeek-V3 Technical Report DeepSeek-AI (2024)
  架构混合专家
• Mixtral of Experts Albert Q. Jiang et al. (2024)
  混合专家架构
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints Joshua Ainslie et al. (2023)
  推理架构
• GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng et al. (2022)
  架构
• Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press et al. (2021)
  架构长上下文