Summary The /v1/chat/completions and /tokenize endpoints allow a chattemplatekwargs request parameter that is used in the code before it is properly validated against the chat template. With the right chattemplatekwargs parameters, it is possible to block processing of the API server for long periods of time, delaying all other requests

Details In servingengine.py, the chattemplatekwargs are unpacked into kwargs passed to chatutils.py applyhfchattemplate with no validation on the keys or values in that chattemplatekwargs dict. This means they can be used to override optional parameters in the applyhfchattemplate method, such as tokenize, changing its default from False to True.

https://github.com/vllm-project/vllm/blob/2a6dc67eb520ddb9c4138d8b35ed6fe6226997fb/vllm/entrypoints/openai/servingengine.py#L809-L814

https://github.com/vllm-project/vllm/blob/2a6dc67eb520ddb9c4138d8b35ed6fe6226997fb/vllm/entrypoints/chatutils.py#L1602-L1610

Both servingchat.py and servingtokenization.py call into this preprocesschat method of servingengine.py and they both pass in chattemplatekwargs.

So, a chattemplatekwargs like {"tokenize": True} makes tokenization happen as part of applying the chat template, even though that is not expected. Tokenization is a blocking operation, and with sufficiently large input can block the API server's event loop, which blocks handling of all other requests until this tokenization is complete.

This optional tokenize parameter to applyhfchattemplate does not appear to be used, so one option would be to just hard-code that to always be False instead of allowing it to be optionally overridden by callers. A better option may be to not pass chattemplatekwargs as unpacked kwargs but instead as a dict, and only unpack them after the logic in applyhfchattemplate that resolves the kwargs against the chat template.

Impact

Any authenticated user can cause a denial of service to a vLLM server with Chat Completion or Tokenize requests.

Fix

https://github.com/vllm-project/vllm/pull/27205

Summary

Users can crash the vLLM engine serving multimodal models by passing multimodal embedding inputs with correct ndim but incorrect shape (e.g. hidden dimension is wrong), regardless of whether the model is intended to support such inputs (as defined in the Supported Models page).

The issue has existed ever since we added support for image embedding inputs, i.e. #6613 (released in v0.5.5)

Details

Using image embeddings as an example:

- For models that support image embedding inputs, the engine crashes when scattering the embeddings to inputsembeds (mismatched shape) - For models that don't support image embedding inputs, the engine crashes when validating the inputs inside getinputembeddings (validation fails).

This happens because we only validate ndim of the tensor, but not the full shape, in input processor (via MultiModalDataParser).

Impact

- Denial of service by crashing the engine

Mitigation

- Use API key to limit access to trusted users. - Set --limit-mm-per-prompt to 0 for all non-text modalities to ban multimodal inputs, which includes multimodal embedding inputs. However, the model would then only accept text, defeating the purpose of using a multi-modal model.

Resolution

- https://github.com/vllm-project/vllm/pull/27204

Issue Description Librosa defaults to using numpy.mean for mono downmixing (tomono), while the international standard ITU-R BS.775-4 specifies a weighted downmixing algorithm. This discrepancy results in: - Inconsistency between audio heard by humans (e.g., through headphones/regular speakers) and audio processed by AI models (Which infra via Librosa, such as vllm, transformer).

https://github.com/librosa/librosa/blob/af8c839fb15317fa2712ea66e7a22da6a9267b32/librosa/core/audio.py#L478 Attack Scenario and Impact

LFE (Low-Frequency Effects) Channel Exploit Attackers can craft special multichannel audio files containing: 1. Normal content in front channels (L/R) 2. Either interference signals or hidden content in the LFE channel

Notice: It is worth noting that not only the LFE channel is excluded, but in fact, channels beyond the 6th (such as rear surround channels, overhead channels, height speakers, etc.) are also not supported.

Attack Methodology:

Attackers can create specially engineered multichannel audio with LFE interference, where front channels (L/R) contain normal content while the LFE channel carries interference signals or hidden content. When played on consumer devices that ignore LFE channels, only the normal content is heard. However, when processed by AI systems using Librosa (which mixes all channels), the LFE interference affects speech recognition feature extraction or masks critical detection features. This enables malicious content to bypass AI detection while still reaching end users, potentially compromising voice authentication systems, evading content moderation, or disrupting speech recognition accuracy.

Potential Exploitation Scenarios: - Voice authentication systems may be tricked into accepting anomalous audio - Content moderation systems may fail to detect prohibited content hidden in LFE channels - Speech recognition systems may produce incorrect transcriptions

Note: torch.audio implements this correctly. Failure to do so may lead to inconsistencies between training and test audio, resulting in performance degradation.

Resources

- ITU-R BS.775-4 Standard - Librosa Source Code - Librosa securty report

Fixes

- https://github.com/vllm-project/vllm/pull/37058, which removes the librosa dependency from vLLM.

Summary

Integer truncation of tensor dimensions in vLLM's GGUF dequantize kernels (csrc/quantization/gguf/ggufkernel.cu) causes partial tensor processing. The output tensor is allocated at full size via torch::empty (uninitialized memory), but the dequantize CUDA kernel processes only a truncated number of elements. The unfilled portion of the output tensor retains whatever was previously in GPU memory. In multi-tenant inference deployments, this residual GPU memory may contain tensor data from other users' inference requests, constituting information disclosure.

Root Cause

The tocudaggmlt function pointer type at ggml-common.h:1067 declares its element count parameter as int (32-bit):

cpp using tocudaggmlt = void ()(const void restrict x, dstt restrict y, int k, // 32-bit cudaStreamt stream);

All dequantize kernel functions (dequantizeblockcuda, dequantizerowq2Kcuda, etc. in dequantize.cuh) inherit this int k parameter and use it as the kernel launch grid size:

cpp static void dequantizeblockcuda(..., const int k, cudaStreamt stream) { const int numblocks = (k + 2CUDADEQUANTIZEBLOCKSIZE - 1) / (2CUDADEQUANTIZEBLOCKSIZE); dequantizeblock<<<numblocks, CUDADEQUANTIZEBLOCKSIZE, 0, stream>>>(vx, y, k); }

In ggmldequantize() at ggufkernel.cu:85, the caller passes m n (an int64t product) to this int k parameter:

cpp at::Tensor DW = torch::empty({m, n}, options); // line 80: full-size, UNINITIALIZED // ... tocuda((void)W.dataptr(), (scalart)DW.dataptr(), m n, stream); // line 85: mn truncated to int

When m n > INTMAX, the truncated k is smaller than the actual tensor size. The kernel processes k elements. The remaining (m n) - k elements in DW are never written and contain stale GPU memory.

This is a single root cause -- the int type on the k parameter in tocudaggmlt -- with a single fix: change int k to int64t k. All dequantize functions inherit this type through the same typedef.

Affected Functions

All in csrc/quantization/gguf/ggufkernel.cu:

| Function | Line | Allocation | Info Disclosure? | |----------|------|-----------|-----------------| | ggmldequantize | 74 | torch::empty({m, n}) at line 80 | Yes -- mn truncated to int k at line 85 | | ggmlmulmatveca8 | 91 | torch::empty({vecs, row}) at line 99 | Yes -- int col = X.sizes()[1] at line 94 | | ggmlmulmata8 | 207 | torch::empty({batch, row}) at line 215 | Yes -- int col = X.sizes()[1] at line 210 | | ggmlmoea8 | 279 | torch::empty({tokenstopk, row}) at line 289 | Yes -- int col = X.sizes()[1] at line 285 |

All four functions allocate output tensors with torch::empty (uninitialized) and then run CUDA kernels that use truncated dimension values as loop bounds. The unfilled portion of each output tensor retains stale GPU memory.

ggmlmoea8vec (line 382) uses torch::zeros instead of torch::empty, so it is not affected by the info disclosure variant.

Impact: Information Disclosure in Multi-Tenant Serving

vLLM is designed for multi-tenant inference serving. GPU memory is reused across requests from different users. When the dequantize kernel partially fills an output tensor:

1. The output tensor DW is allocated with torch::empty -- the buffer contains whatever was previously in that GPU memory region 2. The dequantize kernel fills only a truncated portion of the buffer 3. The unfilled portion retains residual data from prior GPU operations, which may include tensor data from other users' inference requests 4. The contaminated tensor proceeds through the model computation 5. No error or warning is generated -- the partial fill is silent

This is a confidentiality violation. In shared inference deployments (the primary vLLM use case), one user's inference data can leak into another user's model computation through residual GPU memory.

Attacker Control

The attacker crafts a GGUF model file with weight tensor dimensions whose product exceeds INTMAX (e.g., a matrix with shape [65536, 65536] gives m n = 4,294,967,296). The model is hosted on HuggingFace or any model hub. The victim loads the model with vLLM for inference serving. The truncation happens automatically during model weight dequantization.

Fix

A fix for this vulnerability was added here: https://github.com/vllm-project/vllm/pull/44971