CVE-2026-44223: vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters
Summary
The extracthiddenstates speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetitionpenalty, frequencypenalty, or presencepenalty).
A single request with a penalty parameter (e.g., "repetitionpenalty": 1.1) is sufficient to crash the server. The crash is deterministic and immediate — no concurrency, race condition, or special workload is required.
Details
In vLLM v0.17.0, the extracthiddenstates proposer's propose() method returned sampledtokenids.unsqueeze(-1), producing a tensor of shape (batchsize, 1).
In PR #37013 (first released in v0.18.0), the KV connector interface was refactored out of propose(). The return type changed from tuple[Tensor, KVConnectorOutput | None] to Tensor, and the .unsqueeze(-1) call was removed along with the KV connector output:
python Before (v0.17.0): return sampledtokenids.unsqueeze(-1), kvconnectoroutput # shape (batchsize, 1)
After (v0.18.0+): return sampledtokenids # shape (batchsize, 2) after first decode step
The refactor missed that sampledtokenids changed semantics between the first and subsequent decode steps. After the first decode step, the rejection sampler allocates its output as (batchsize, maxspeclen + 1). With numspeculativetokens=1, this produces shape (batchsize, 2) instead of the expected (batchsize, 1), causing a broadcast shape mismatch during penalty application.
Impact
Any vLLM deployment between v0.18.0 and v0.19.1 (inclusive) configured with extracthiddenstates speculative decoding is affected. A single API request containing any penalty parameter immediately and permanently crashes the EngineCore process, resulting in complete loss of service availability.
Patches
Fixed in PR #38610, first included in vLLM v0.20.0. The fix slices the return value to sampledtokenids[:, :1], ensuring the correct (batchsize, 1) shape regardless of the rejection sampler's output dimensions.
Workarounds
- Upgrade to vLLM v0.20.0 or later. - If upgrading is not possible, avoid using extracthiddenstates as the speculative decoding method on affected versions. - Alternatively, reject or strip penalty parameters (repetitionpenalty, frequencypenalty, presencepenalty) from incoming requests at an API gateway before they reach vLLM.
Other sources
vLLM is an inference and serving engine for large language models (LLMs). From 0.18.0 to before 0.20.0, the extracthiddenstates speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetitionpenalty, frequencypenalty, or presencepenalty). A single request with a penalty parameter (e.g., "repetitionpenalty": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.
— MITRE
Affected Software
Remediation
Recommended actions to resolve this vulnerability, in priority order.
- Upgrade
Upgrade
pip/vllmto a version that resolves this vulnerability.Fixed in 0.20.0 - Upgrade
Upgrade
pip/vllmto a version that resolves this vulnerability.Fixed in 0.20.0Patch PR #38610 - Configuration
Configure the API gateway to reject any incoming request that includes sampling penalty parameters (repetition_penalty, frequency_penalty, presence_penalty), or to strip those parameters from requests before they are forwarded to vLLM.
API gateway repetition_penalty, frequency_penalty, presence_penalty handling = reject or strip - Configuration
If running vLLM versions 0.18.0 through 0.19.1 inclusive, disable the extract_hidden_states speculative decoding proposer (select an alternative speculative decoding method) because extract_hidden_states returns an incorrectly-shaped tensor after the first decode step and crashes the EngineCore when penalty parameters are present.
vLLM speculative decoding (extract_hidden_states) speculative_decoding_method / extract_hidden_states = disabled (do not use extract_hidden_states)
Event History
Frequently Asked Questions
What is the severity of CVE-2026-44223?
CVE-2026-44223 is considered a high severity vulnerability due to its potential to crash the EngineCore process.
How do I fix CVE-2026-44223?
To fix CVE-2026-44223, upgrade vLLM to version 0.20.0 or later.
What products are affected by CVE-2026-44223?
CVE-2026-44223 affects versions of vLLM from 0.18.0 to below 0.20.0.
What is the impact of CVE-2026-44223?
The impact of CVE-2026-44223 includes a RuntimeError causing a crash during speculative decoding when batch requests use sampling penalty parameters.
Who is responsible for addressing CVE-2026-44223?
The maintainers of the vLLM project are responsible for addressing CVE-2026-44223 and releasing patches.