PYSEC-2026-4347: vLLM: Out-of-vocabulary prompt token id passes input validation and triggers a CUDA device-side assert (engine crash / DoS)
For a model whose vocabulary size is an exact multiple of the embedding padding size (64) — e.g. any Llama-2-family model, vocab 32000 — a single request carrying a pre-tokenized prompt with token id equal to the vocab size (32000) passes vLLM's input-id validation but is one past the last embedding row. At tensor-parallel size 1 the embedding lookup does not mask indices, so the id reaches F.embedding out of range and raises a CUDA device-side assert, which poisons the engine's CUDA context and takes the engine down. Any client that can send pre-tokenized input (/v1/completions with prompt as token ids, /tokenize-fed flows, or the kvtransferparams["prompttokenids"] reuse path) can trigger it.
Root cause vllm/v1/engine/inputprocessor.py accepts a token id when id <= max(tokenizer.maxtokenid, modelconfig.getvocabsize() - 1). vllm/tokenizers/hf.py computes maxtokenid = max(max(vocab.values()), tokenizer.vocabsize). For Llama-2 tokenizers max(vocab.values()) = 31999 but tokenizer.vocabsize = 32000, so maxtokenid = 32000 — one greater than the largest valid id. The engine bound therefore becomes max(32000, 31999) = 32000, and id 32000 is accepted. The model's embedding has padvocabsize(32000, 64) = 32000 rows (no padding), valid indices 0..31999. At VocabParallelEmbedding.forward with tpsize == 1 there is no masking, so index 32000 hits F.embedding out of range.
Affected - tp=1 (single GPU per model — common per-copy layout). tp>1 masks via getmaskedinputandmask, so the id is clamped, not an OOB — likely wrong output rather than a crash. - Models whose vocab is a multiple of 64 with tokenizer.vocabsize one past the max real id (Llama-2 family, vocab 32000, is the clear case). Qwen2.5 is NOT affected (padded headroom).
Suggested fix maxtokenid in vllm/tokenizers/hf.py now uses tokenizer.vocabsize - 1, so an id equal to the vocabulary size fails input validation instead of reaching the embedding. Fixed in MR59491, released in v0.32.0.
Reporter Shijie Lyu (GitHub shijie-lyu lshjhf@gmail.com). Reported privately per SECURITY.
This advisory was fixed in MR59491.
Other sources
vLLM is an inference and serving engine for large language models. Prior to 0.32.0, vllm/tokenizers/hf.py computes maxtokenid with an off-by-one upper bound, allowing a pre-tokenized prompt ID equal to the vocabulary size to pass validation. For a tensor-parallel size of one and an exact multiple-of-64 vocabulary, VocabParallelEmbedding.forward does not mask that value before F.embedding uses it, causing a CUDA device-side assertion and terminating the engine. This issue is fixed in version 0.32.0.
— OSV
Affected Software
Remediation
Recommended actions to resolve this vulnerability, in priority order.
- Upgrade
Upgrade
pip/vllmto a version that resolves this vulnerability.Fixed in 0.32.0 - Upgrade
Upgrade
vLLMto a version that resolves this vulnerability.Fixed in 0.32.0Patch MR59491
Event History
Frequently Asked Questions
Who can trigger the denial of service?
Any client with permission to submit requests can trigger it if they can provide pre-tokenized prompt IDs. Relevant paths include /v1/completions with prompt supplied as token IDs, flows using /tokenize, and reuse through kv_transfer_params["prompt_token_ids"].
Which deployments are known to be susceptible?
The described condition applies to models whose vocabulary size is an exact multiple of the embedding padding size of 64. The example given is Llama-2-family models with a vocabulary size of 32000, and the engine crash is described for tensor-parallel size 1.
What input causes the crash?
A pre-tokenized prompt containing a token ID equal to the vocabulary size can pass validation even though it is outside the embedding range. For a vocabulary size of 32000, the triggering ID is 32000, while the highest valid ID is 31999.