Research arXiv cs.CL

DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding

speculative decodinginference accelerationstochastic sampling

Speculative decoding accelerates LLM inference by having a lightweight drafter propose tokens that a target model verifies. Existing block and diffusion-style drafters often assume the positions in a draft block are conditionally independent, which is problematic for stochastic (non-greedy) sampling where the verification must match the target distribution. DBLAST introduces dependent block drafting, explicitly modeling the relationships between draft tokens to increase acceptance rates and sampling fidelity. The method is particularly relevant for on-device or latency-sensitive applications where autoregressive decoding is a bottleneck.

Read original →

← Back to home