Inference
Inference
Efficient inference and optimization
4 Articles
23 Papers Referenced
~32 min Reading Time
Recommended Reading Order
1
KV Cache and Quantization: Making Large Models Faster
KV cache principles, quantization methods, and inference cost optimization.
2
Efficient Attention: Breaking the Quadratic Sequence Bottleneck
FlashAttention, sparse attention, and long-context inference optimization.
3
Long Context: Helping Models Read Farther
Context window extension, positional encoding extrapolation, and long-text evaluation.
4
Edge and On-Device Inference: LLMs Move to the Terminal
LLM inference techniques and optimizations for mobile devices, embedded systems, and edge computing.