Neural Machine Translation in Linear Time

作者： Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aaron van den Oord, Alex Graves, Koray Kavukcuoglu (2016)

arXiv： 1610.10099

领域

架构

TLDR（中文）

用扩张卷积做 seq2seq，把序列建模从"必须 RNN 顺序计算"中解放出来；和同期 ConvS2S 一起是 Transformer 之前"并行序列建模"的最强尝试。

TLDR (English)

Uses dilated convolutions for seq2seq, liberating sequence modeling from "must use RNN sequential computation". Together with ConvS2S, represents the strongest attempt at parallel sequence modeling before Transformer.

出现在这些文章里

• Attention：让每个位置选择上下文
• Attention: Choosing the Relevant Context

同被引用

这些论文与本文出现在同一篇文章中

• Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau et al. (2014)
• Effective Approaches to Attention-based Neural Machine Translation Minh-Thang Luong et al. (2015)
• Sequence to Sequence Learning with Neural Networks Ilya Sutskever et al. (2014)
• Attention Is All You Need Ashish Vaswani et al. (2017)

相关论文

同一领域的其他论文

• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model DeepSeek-AI (2024)
  架构推理
• DeepSeek-V3 Technical Report DeepSeek-AI (2024)
  架构混合专家
• Mixtral of Experts Albert Q. Jiang et al. (2024)
  混合专家架构
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints Joshua Ainslie et al. (2023)
  推理架构
• GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng et al. (2022)
  架构
• Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press et al. (2021)
  架构长上下文