GHSA-q97c-p368-75c4: Out-of-bounds Read
For a model whose vocabulary size is an exact multiple of the embedding padding size (64) — e.g. any Llama-2-family model, vocab 32000 — a single request carrying a pre-tokenized prompt with token id equal to the vocab size (32000) passes vLLM's input-id validation but is one past the last embedding row. At tensor-parallel size 1 the embedding lookup does not mask indices, so the id reaches F.embedding out of range and raises a CUDA device-side assert, which poisons the engine's CUDA context and takes the engine down. Any client that can send pre-tokenized input (/v1/completions with prompt as token ids, /tokenize-fed flows, or the kvtransferparams["prompttokenids"] reuse path) can trigger it.
Root cause vllm/v1/engine/inputprocessor.py accepts a token id when id <= max(tokenizer.maxtokenid, modelconfig.getvocabsize() - 1). vllm/tokenizers/hf.py computes maxtokenid = max(max(vocab.values()), tokenizer.vocabsize). For Llama-2 tokenizers max(vocab.values()) = 31999 but tokenizer.vocabsize = 32000, so maxtokenid = 32000 — one greater than the largest valid id. The engine bound therefore becomes max(32000, 31999) = 32000, and id 32000 is accepted. The model's embedding has padvocabsize(32000, 64) = 32000 rows (no padding), valid indices 0..31999. At VocabParallelEmbedding.forward with tpsize == 1 there is no masking, so index 32000 hits F.embedding out of range.
Affected - tp=1 (single GPU per model — common per-copy layout). tp>1 masks via getmaskedinputandmask, so the id is clamped, not an OOB — likely wrong output rather than a crash. - Models whose vocab is a multiple of 64 with tokenizer.vocabsize one past the max real id (Llama-2 family, vocab 32000, is the clear case). Qwen2.5 is NOT affected (padded headroom).
Suggested fix maxtokenid in vllm/tokenizers/hf.py now uses tokenizer.vocabsize - 1, so an id equal to the vocabulary size fails input validation instead of reaching the embedding. Fixed in MR59491, released in v0.32.0.
Reporter Shijie Lyu (GitHub shijie-lyu lshjhf@gmail.com). Reported privately per SECURITY.
This advisory was fixed in MR59491.
Affected Software
Remediation
Recommended actions to resolve this vulnerability, in priority order.
- Upgrade
Upgrade
pip/vllmto a version that resolves this vulnerability.Fixed in 0.32.0 - Upgrade
Upgrade
vLLMto a version that resolves this vulnerability.Fixed in v0.32.0Patch MR59491
Event History
Frequently Asked Questions
Which deployments are exposed to a service outage?
Deployments using a model whose vocabulary size is an exact multiple of 64 are exposed when running with tensor-parallel size 1. The example given is a Llama-2-family model with a vocabulary size of 32000.
What access does an attacker need?
The attacker needs the ability to submit pre-tokenized input containing the token ID equal to the model vocabulary size. This can be sent through /v1/completions with prompt supplied as token IDs, /tokenize-fed flows, or the kv_transfer_params["prompt_token_ids"] reuse path.
What is the impact of a successful request?
At tensor-parallel size 1, the invalid ID reaches the embedding lookup and causes a CUDA device-side assertion. That assertion poisons the engine CUDA context and takes the engine down.
How can operators identify relevant exposure or an attempted trigger?
Check whether the deployment accepts client-provided pre-tokenized prompts, uses tensor-parallel size 1, and serves a model with vocabulary size divisible by 64. A triggering request uses a token ID equal to the vocabulary size, such as 32000 for a 32000-token vocabulary, and results in a CUDA device-side assertion.