PYSEC-2026-145: vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters
vLLM is an inference and serving engine for large language models (LLMs). From 0.18.0 to before 0.20.0, the extracthiddenstates speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetitionpenalty, frequencypenalty, or presencepenalty). A single request with a penalty parameter (e.g., "repetitionpenalty": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.
Affected Software
Remediation
Recommended actions to resolve this vulnerability, in priority order.
- Upgrade
Upgrade
vLLMto a version that resolves this vulnerability.Fixed in 0.20.0