Summary Short summary of the problem. Make the impact and severity as clear as possible. For example: An unsafe deserialization vulnerability allows any unauthenticated user to execute arbitrary code on the server.

Sending a pure prompt embeds payload in a /v1/completions request with a model using M-RoPE causes the EngineCore to fail an assertion and fatally crash, shutting down the entire server application.

Any remote user who is authorized to make a /v1/completions endpoint can trivially make such a request and induce a crash.

Details Give all details on the vulnerability. Pointing to the incriminated source code is very helpful for the maintainer.

In commit 56669c1, a simple assert intended to be a type-narrowing assert was added to the initmropepositions method in GPUModelRunner (the offending line on main at the time of writing: https://github.com/vllm-project/vllm/blob/2d481f8a946ee0521872af0f098674a8ee01ce4a/vllm/v1/worker/gpumodelrunner.py#L1588-L1607).

python assert reqstate.prompttokenids is not None, ( "M-RoPE requires prompttokenids to be available." )

This type narrowing assert is to prevent mypy errors later in the function because None is not a valid type for mropemodel.getmropeinputpositions. Unfortunately, this assertion is not always true. /v1/completions requests that specify prompt=None and promptembeds=<not none> will indeed create a CachedRequestState where prompttokenids is None. This triggers the assertion, which in turn crashes the EngineCore and the Server application.

(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpumodelrunner.py", line 3997, in executemodel (EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] deferredstatecorrectionsfn = self.updatestates(scheduleroutput) (EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpumodelrunner.py", line 1239, in updatestates (EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] self.initmropepositions(reqstate) (EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpumodelrunner.py", line 1582, in initmropepositions (EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] assert reqstate.prompttokenids is not None, ( (EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] AssertionError: M-RoPE requires prompttokenids to be available. (APIServer pid=1) ERROR 06-11 00:48:03 [asyncllm.py:704] AsyncLLM outputhandler failed. (APIServer pid=1) ERROR 06-11 00:48:03 [asyncllm.py:704] Traceback (most recent call last): (APIServer pid=1) ERROR 06-11 00:48:03 [asyncllm.py:704] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/asyncllm.py", line 660, in outputhandler (APIServer pid=1) ERROR 06-11 00:48:03 [asyncllm.py:704] outputs = await enginecore.getoutputasync() (APIServer pid=1) ERROR 06-11 00:48:03 [asyncllm.py:704] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=1) ERROR 06-11 00:48:03 [asyncllm.py:704] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/coreclient.py", line 1030, in getoutputasync (APIServer pid=1) ERROR 06-11 00:48:03 [asyncllm.py:704] raise self.formatexception(outputs) from None (APIServer pid=1) ERROR 06-11 00:48:03 [asyncllm.py:704] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.

All requests using the /v1/chat/completions endpoints will have text/prompttokenids parts (corresponding to the chat template), and promptembeds parts are handled as mmfeatures. This method (rightly) filters out those promptembeds content parts as they are treated as text positions.

A sufficient solution to type narrowing here without raising a fatal assertion is to instead replace the assertion with a using dummy token ids:

python def initmropepositions(self, reqstate: CachedRequestState): model = self.getmodel() assert supportsmrope(model), "M-RoPE support is not implemented." mropemodel = cast(SupportsMRoPE, model)

# Filter out promptembeds modality (text-only position info) mropefeatures = [ f for f in reqstate.mmfeatures if f.modality != "promptembeds" ] # Handle both tokenids and embeddings-only inputs if reqstate.prompttokenids is not None: inputtokens = reqstate.prompttokenids elif reqstate.promptembeds is not None: # For text-only embeddings, dummy token IDs are safe since # getmropeinputpositions only uses len(inputtokens) when mmfeatures is empty seqlen = reqstate.promptembeds.shape[0] inputtokens = list(range(seqlen)) # Verify no mmfeatures remain (should be true after promptembeds filter) assert len(mropefeatures) == 0, ( "M-RoPE with promptembeds-only input should have no multimodal features" ) else: raise ValueError( "M-RoPE requires either prompttokenids or promptembeds." )

reqstate.mropepositions, reqstate.mropepositiondelta = ( mropemodel.getmropeinputpositions( inputtokens, mropefeatures, ) )

Technically, in isolation, this method still crashes in the case where reqstate.prompttokenids is None and reqstate.mmfeatures, so the solution above still leaves that potential vector open. As far as can be determined, however, such a reqstate is impossible in the first place in online mode, because it would require a /v1/completions request with promptembeds AND multimodal features, but the /v1/completions request schema does not expose multimodal inputs in any discernible way. Today, those are the only two endpoints with promptembeds support.

When in offline mode, it is technically possible to directly create an EngineCoreRequest that has promptembeds and not prompttokenids and mmfeatures, and pass that to LLM.generate. That would trigger this same assertion, and no validation would prevent that combination. It is strongly suspected, though, that this combination would be undefined in any model that support M-RoPE, because it would not be possible to determine which token positions correspond to mmfeatures. The proposed solution above would end up not setting reqstate.mropepositions and reqstate.mropepositiondelta in this scenario, which could result in undefined behavior.

promptembeds is far more familiar here than M-RoPE, and it is understood that each model that supports it is responsible for defining its own getmropeinputpositions which have varying implementations. There is insufficient knowledge to be prescriptive in how the two features should interact in the offline case, other than possibly raising a validation error earlier on preventing that combination (which would emulate the current assertion behavior). Regardless, in offline mode, the chances of a remote user being able to exploit this are slim-to-nil compared to the online case which is incredibly straightforward.

Impact What kind of vulnerability is it? Who is impacted?

- Denial of Service caused by an incorrect assertion inside of the GPUModelRunner which causes a fatal EngineCore exception - Any configuration with --enable-prompt-embeds and M-RoPE-supported model is vulnerable - The attack is extremely easy from the remote attacker's perspective (copying the official promptembeds online mode docs examples almost-verbatim, accounting for model-name and connection details, of course, will induce a guaranteed shutdown)