Efficient Guided Generation for Large Language Models

作者： Brandon T. Willard, Remi Louf (2023)

arXiv： 2307.09702

领域

推理应用

TLDR（中文）

提出高效的约束解码方法，让大语言模型在生成过程中实时遵守 JSON Schema、正则表达式或上下文无关文法。通过将语法约束转化为有限状态自动机，在几乎不增加延迟的情况下保证输出格式正确。

TLDR (English)

Proposes efficient constrained decoding that enforces JSON Schema, regular expressions, or context-free grammars during generation. Converts syntax constraints into finite-state automata, guaranteeing correct output format with minimal latency overhead.

相关论文

同一领域的其他论文

• Model Context Protocol (MCP) Anthropic (2024)
  应用
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints Joshua Ainslie et al. (2023)
  推理架构
• H2O: Heavy-Hitter Oracle for Accurate KV Cache Compression Zichang Liu et al. (2023)
  推理
• Efficient Streaming Language Models with Attention Sinks Guangxuan Xiao et al. (2023)
  推理长上下文
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models Shunyu Yao et al. (2023)
  推理能力应用
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao et al. (2022)
  推理