In the rapidly evolving world of technology and digital communication, a new method known as speculative decoding is enhancing the way we interact with machines. This technique is making a notable ...

推测解码（Speculative Decoding, SD）已成为一种有效加速大语言模型（LLM）推理的技术，且不会牺牲输出质量。然而，所能实现的加速效果在很大程度上取决于草稿模型（drafting model）的有效性。基于模型的方法（如 EAGLE-2）虽然准确，但计算成本高昂；而基于检索的 ...

不牺牲任何生成质量，将多模态大模型推理最高加速3.2倍！ 华为诺亚方舟实验室最新研究已入选NeurIPS 2025。 截至目前，投机推理（Speculative Decoding）技术已成为大语言模型（LLM）推理加速的“标准动作”，但在多模态大模型（VLM）上的应用却举步维艰，现有方法 ...

This figure shows an overview of SPECTRA and compares its functionality with other training-free state-of-the-art approaches across a range of applications. SPECTRA comprises two main modules, namely ...

Researchers from Intel Labs and the Weizmann Institute of Science have introduced a major advance in speculative decoding. The new technique, presented at the International Conference on Machine ...

Speculative decoding accelerates large language model generation by allowing multiple tokens to be drafted swiftly by a lightweight model before being verified by a larger, more powerful one. This ...