GHSA-8397-8qf6-f2gm: Medium severity pip/vllm vulnerability
Summary
The Phi-4 multimodal audio encoder (phi4mmaudio.py / phi4mmutils.py) runs its conformer attention in eager mode by default: the official HF config of microsoft/Phi-4-multimodal-instruct does not set useptscaleddotproductattention, so the default False applies (phi4mmaudio.py:166, :876; phi4mmutils.py:1669). The eager path fully materializes the attention probability matrix of shape [nchunks, 16, 500, 500] per layer — and then stores it on the module: self.attn = attn (phi4mmutils.py:1802, a torch.jit.Attribute declared at :1693), keeping a live reference for every one of the 24 encoder layers.
The only gates on audio length are the server-side limits VLLMMAXAUDIOCLIPFILESIZEMB=25 and VLLMMAXAUDIODECODEDURATIONS=600 (envs.py:83-84) — i.e. a 10-minute clip is a legal input by default. A 600 s clip becomes ~60,000 mel frames, which the encoder unfolds into 500-frame chunks (phi4mmaudio.py:996, maxseqlen = 500), giving nchunks ≈ 120. Each layer therefore materializes a 120×16×500×500 matrix (~1–2 GiB depending on dtype), and the module attribute keeps all completed layers' matrices alive, so GPU memory climbs monotonically within a single request until an allocation fails.
Verified end-to-end on the official vllm/vllm-openai:v0.31.0 image: one 19.2 MB silent 600 s WAV → worker torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.29 GiB ... 501.75 MiB free ... 22.09 GiB allocated by PyTorch inside getaudiofeatures (phi4mmaudio.py:1279 → :1332 → phi4mmutils.py:1386) → EngineCore encountered a fatal error → API EngineDeadError → whole server down; /health refuses connections. Total time from request to dead service: ~6 seconds.
A secondary effect: because self.attn persists on the module after the request ends, the matrices of the last audio request stay resident indefinitely (observed +50 MiB pinned after a single 5 s control request); they are only overwritten by the next audio request, so the resident footprint equals the largest audio processed since startup.
Details
1. Root cause. MultiHeadedAttention.forwardattention in eager mode computes attn = maskedsoftmax(scores, mask) and unconditionally executes self.attn = attn (phi4mmutils.py:1795-1802). The attribute is a leftover inference/debug hook (used by torch.jit.Attribute(None, Tensor | None) at :1693); in serving it serves no purpose but pins the full [batch·chunks, heads, T, T] tensor per layer past both the layer's forward and the request's lifetime. The SDP attention branch (useptscaleddotproductattention=True) never materializes this matrix — but the HF config does not enable it and the code default is False.
2. Why one request suffices. The conformer chunks long audio to maxseqlen = 500 frames (phi4mmaudio.py:996). Duration is bounded only by VLLMMAXAUDIODECODEDURATIONS=600, so nchunks reaches ~120 for a maximal clip. With 16 heads and fp32 scores, one layer's attention tensor is ~1.8 GiB (the observed failing allocation was 2.29 GiB, consistent with the scores+bias materialization). The encoder has numblocks=24, attentionheads=16, attentiondim=1024, timereduction=8 (HF config). Weights plus a handful of retained per-layer matrices exhaust the ~2.3 GiB headroom that --gpu-memory-utilization 0.9 leaves on a 24 GB card — mid-forward, on the first request.
4. Deployment reality. --trust-remote-code is the documented, standard way to serve this model (the HF model card instructs it), so the finding applies to default deployments. The crash is deterministic, not load-dependent: scheduler stats at death show a single running request and near-zero KV cache usage.
Impact
Any unauthenticated client that can reach the chat/completions API of a default deployment serving Phi-4-multimodal-instruct (or any model reusing this audio tower) can kill the entire engine with one request carrying a perfectly legal 10-minute audio clip (silent noise suffices — no crafted content needed). Full denial of service, requiring a process restart. On GPUs with more headroom (e.g. 80 GB at default utilization) a single request may survive, but the eager path still materializes GiB-scale per-layer matrices and retains the last request's tensors indefinitely, so a few sequential max-length requests — or normal traffic following one — achieve the same result.
Relation to prior art: the only public Phi-4-mm advisory we found, GHSA-vc6m-hm49-g9qg, concerns a CPU-side quadratic-cost path — different subsystem, different mechanism. GitHub search for "phi4mm audio OOM/memory" (31 results, 2026-10-08) shows no report of this attention-retention issue; main (2026-10-08) is unchanged.
Suggested remediation (any one breaks the chain; 1+2 recommended):
1. Remove self.attn = attn from the serving path (or gate it behind an explicit debug/outputattentions flag) so attention matrices are freed when the layer returns. 2. Default useptscaleddotproductattention to True (flash/SDPA kernels never materialize the [B, H, T, T] matrix); keep eager only as an opt-in. 3. Additionally bound the encoder input (frames or duration) for this model so the eager path cannot reach GiB-scale single-layer allocations even if enabled.
This advisory was fixed in MR60614.
Affected Software
Remediation
Recommended actions to resolve this vulnerability, in priority order.
- Upgrade
Upgrade
pip/vllmto a version that resolves this vulnerability.Fixed in 0.32.0 - Upgrade
Upgrade to a fixed release to a version that resolves this vulnerability.
Patch MR60614 - Configuration
Enable flash/SDPA attention by default; keep eager attention only as an explicit opt-in.
Phi-4 multimodal audio encoder use_pt_scaled_dot_product_attention = True - Configuration
Remove the assignment that retains attention matrices, or gate it behind an explicit debug/output_attentions flag so matrices are freed after the layer returns.
MultiHeadedAttention serving path self.attn = disabled unless debug/output_attentions is explicitly enabled - Configuration
Bound the encoder input by frames or duration so the eager path cannot process inputs large enough to create GiB-scale single-layer attention allocations.
Phi-4 multimodal audio encoder encoder input frame or duration limit = bounded
Event History
Frequently Asked Questions
Which deployments are exposed to this denial-of-service condition?
Deployments serving Phi-4 multimodal audio requests through vLLM are exposed when they accept long audio clips. The official microsoft/Phi-4-multimodal-instruct configuration does not enable scaled dot-product attention, so the memory-intensive eager attention path is used by default.
What does an attacker need to do to trigger the issue?
An attacker needs permission to submit an audio request and can trigger the condition with a long audio clip. A 600-second clip is permitted by the documented default server-side decode-duration limit, while the default file-size limit is 25 MB.
What is the operational impact of a successful attack?
The audio encoder materializes large attention-probability matrices for each encoder layer and retains completed layers' matrices through a module attribute. This can exhaust GPU memory and cause a denial of service.
What can be changed if patching is not immediately possible?
Reduce the server-side maximum audio clip file size and maximum audio decode duration from their defaults, particularly preventing long clips from reaching the encoder. The relevant limits are VLLM_MAX_AUDIO_CLIP_FILESIZE_MB and VLLM_MAX_AUDIO_DECODE_DURATION_S.