See how kubeflow compares to other vendors in security performance
Summary
The Kubeflow Pipelines frontend reflected untrusted pod names and namespaces into HTML error responses from the pod-information and pod-events routes. An attacker could craft a link that causes JavaScript to execute in the frontend's origin when a user follows it and reaches an affected error response.
Affected routes include /pipeline/k8s/pod and /pipeline/k8s/pod/events (the deployment's configured base path may differ).
Impact and prerequisites
Successful exploitation requires a victim to open an attacker-supplied link and reach the vulnerable response. JavaScript execution may expose information accessible to the victim's browser or perform actions in the application's origin.
The attacker does not need an account merely to construct and deliver the link. The victim must satisfy the deployment's authentication and route-access requirements. Cookie theft depends on cookie attributes and deployment protections; it is not guaranteed. The submitted evidence does not establish an availability impact or server-side code execution.
Technical details
The pod handlers pass podname and podnamespace query values to getPod and listPodEvents in frontend/server/k8s-helper.ts. Before remediation, failure messages incorporated those values, along with backend error details, and were returned as HTML-capable string responses.
Kubernetes rejecting an invalid resource name did not prevent the issue: the rejected name could still be included in the frontend's error response.
PR #12295 changed the error construction to validate pod names and namespaces with isAllowedResourceName. Invalid values produce a constant error message instead of reflecting the supplied value, and backend error details are no longer included in these responses. The change includes invalid-resource-name tests.
Affected versions and remediation
Affected product: the Kubeflow Pipelines frontend server, identified as ghcr.io/kubeflow/kfp-frontend. Historical releases may use earlier image registry names.
Upgrade to KFP 2.15.0 or later.
The fix merged on October 10, 2025, in commit fedc25e3f7251933d4a37b964bc3e8176beea28e. It is first present in the 2.15.0 release, published November 25, 2025.
The affected range is recorded as < 2.15.0. The original introduction version and separate maintenance-branch backports have not been established. These are KFP component versions, not Kubeflow platform or Python SDK versions.
Distinction from existing artifact XSS
CVE-2023-6571, tracked in the repository as GHSA-j9xw-rrjg-2p9r, documents reflection of the artifact source parameter in an unknown-storage-source error.
This report concerns different handlers and error construction: pod information and pod events. The artifact-source fix in PR #9844, released in 2.0.2, did not repair these pod responses. They required the later change in #12295.
This independently fixable pod-route defect is tracked as CVE-2026-88986. It does not broaden CVE-2023-6571 or absorb artifact-content rendering and other XSS reports.
Attribution
@pwny0 submitted this report on September 23, 2023, and retains the reporter credit.
@JerT33 authored the remediation in PR #12295. Fix attribution is separate from vulnerability-report attribution.
Summary
The Kubeflow Pipelines frontend reflected untrusted pod names and namespaces into HTML error responses from the pod-information and pod-events routes. An attacker could craft a link that causes JavaScript to execute in the frontend's origin when a user follows it and reaches an affected error response.
Affected routes include /pipeline/k8s/pod and /pipeline/k8s/pod/events (the deployment's configured base path may differ).
Impact and prerequisites
Successful exploitation requires a victim to open an attacker-supplied link and reach the vulnerable response. JavaScript execution may expose information accessible to the victim's browser or perform actions in the application's origin.
The attacker does not need an account merely to construct and deliver the link. The victim must satisfy the deployment's authentication and route-access requirements. Cookie theft depends on cookie attributes and deployment protections; it is not guaranteed. The submitted evidence does not establish an availability impact or server-side code execution.
Technical details
The pod handlers pass podname and podnamespace query values to getPod and listPodEvents in frontend/server/k8s-helper.ts. Before remediation, failure messages incorporated those values, along with backend error details, and were returned as HTML-capable string responses.
Kubernetes rejecting an invalid resource name did not prevent the issue: the rejected name could still be included in the frontend's error response.
PR #12295 changed the error construction to validate pod names and namespaces with isAllowedResourceName. Invalid values produce a constant error message instead of reflecting the supplied value, and backend error details are no longer included in these responses. The change includes invalid-resource-name tests.
Affected versions and remediation
Affected product: the Kubeflow Pipelines frontend server, identified as ghcr.io/kubeflow/kfp-frontend. Historical releases may use earlier image registry names.
Upgrade to KFP 2.15.0 or later.
The fix merged on October 10, 2025, in commit fedc25e3f7251933d4a37b964bc3e8176beea28e. It is first present in the 2.15.0 release, published November 25, 2025.
The affected range is recorded as < 2.15.0. The original introduction version and separate maintenance-branch backports have not been established. These are KFP component versions, not Kubeflow platform or Python SDK versions.
Distinction from existing artifact XSS
CVE-2023-6571, tracked in the repository as GHSA-j9xw-rrjg-2p9r, documents reflection of the artifact source parameter in an unknown-storage-source error.
This report concerns different handlers and error construction: pod information and pod events. The artifact-source fix in PR #9844, released in 2.0.2, did not repair these pod responses. They required the later change in #12295.
This independently fixable pod-route defect is tracked as CVE-2026-88986. It does not broaden CVE-2023-6571 or absorb artifact-content rendering and other XSS reports.
Attribution
@pwny0 submitted this report on September 23, 2023, and retains the reporter credit.
@JerT33 authored the remediation in PR #12295. Fix attribution is separate from vulnerability-report attribution.
Summary
In multi-user mode, the Kubeflow Pipelines API server's HTTP multipart upload endpoints (POST /apis/v2beta1/pipelines/upload and POST /apis/v2beta1/pipelines/uploadversion) perform no authorization when the target namespace is empty. The authorization helper canUploadVersionedPipeline returns nil, skipping SubjectAccessReview, whenever the resolved namespace is the empty string (backend/src/apiserver/server/pipelineuploadserver.go:401-403). An empty namespace is how KFP addresses shared, cluster-wide pipelines and is the default whenever the caller omits the namespace query parameter, because REQUIRENAMESPACEFORPIPELINES defaults to false and is unset in every shipped manifest. As a result, an arbitrary low-privilege tenant (or, when the API server is reachable, an unauthenticated caller) can create cluster-shared pipelines and inject new versions into existing shared pipelines, replacing the container image, command, and args that other tenants execute. The gRPC sibling for the same write, canAccessPipeline, instead remaps the empty namespace to the system namespace and authorizes, with a comment stating shared-pipeline writes "must still be authorized". That confirms this is an omission, not intended behavior.
Description
uploadPipeline and uploadPipelineVersion both build a ResourceAttributes{Verb: create} and delegate the access decision to canUploadVersionedPipeline:
go // backend/src/apiserver/server/pipelineuploadserver.go:387-421 func (s PipelineUploadServer) canUploadVersionedPipeline(r http.Request, pipelineID string, resourceAttributes authorizationv1.ResourceAttributes) error { if !common.IsMultiUserMode() { return nil } if len(pipelineID) > 0 { namespace, err := s.resourceManager.FetchNamespaceFromPipelineId(pipelineID) if err != nil { return util.Wrap(err, "Failed to authorize with the Pipeline ID") } if len(resourceAttributes.Namespace) == 0 { resourceAttributes.Namespace = namespace } } if resourceAttributes.Namespace == "" { return nil // empty namespace = no SubjectAccessReview } // ... only here does IsAuthorized() run ... }
When the namespace is empty (a shared/cluster-wide pipeline, or simply an upload with no namespace parameter), the function returns at line 401-403 and IsAuthorized is never reached.
Reachability is the default. ValidateNamespaceRequired only rejects an empty namespace when IsNamespaceRequiredForPipelines() is true:
go // backend/src/apiserver/validation/namespace.go:25-30 func ValidateNamespaceRequired(namespace string) error { if common.IsMultiUserMode() && common.IsNamespaceRequiredForPipelines() && namespace == "" { return util.NewInvalidInputError("Namespace is required for pipeline operations") } return nil } // backend/src/apiserver/common/config.go:73-74 -> default FALSE func IsNamespaceRequiredForPipelines() bool { return GetBoolConfigWithDefault(RequireNamespaceForPipelines, false) }
REQUIRENAMESPACEFORPIPELINES (config.go:41) defaults to false and is not set in any manifest under manifests/. So in a default multi-user install, empty namespaces pass validation and flow into the unguarded branch.
The gRPC sibling proves the intended behavior is to authorize empty-namespace writes:
go // backend/src/apiserver/server/pipelineserver.go:1290-1306 (canAccessPipeline) // Skip authorization for read-only operations on shared pipelines in multi-user mode. // Write operations (create, update, delete) on shared pipelines must still be authorized // against the KFP system namespace, since shared pipelines have no namespace of their own. if s.resourceManager.IsEmptyNamespace(resourceAttributes.Namespace) { if resourceAttributes.Verb == common.RbacResourceVerbGet || resourceAttributes.Verb == common.RbacResourceVerbList { return nil } resourceAttributes.Namespace = common.GetPodNamespace() // ... IsAuthorized(...) ... return nil }
Reads of shared pipelines intentionally skip authorization; writes must be authorized against the system namespace. The upload endpoints only ever perform writes, yet canUploadVersionedPipeline applies the read behavior to them.
Proof of Concept
Reproduced live against the real API server binary (ghcr.io/kubeflow/kfp-api-server@sha256:621fdae9bfcc93405eca299c24f715bc7f17dbb76f1b53e8b2b35852e4d7d423, MULTIUSER=true, REQUIRENAMESPACEFORPIPELINES unset). The API server SA holds subjectaccessreviews/tokenreviews: [create] (authz fully functional); identity attacker@evil.com has no RBAC anywhere; victim-ns exists with no grant to the attacker.
1. Shared (empty-namespace) pipeline create as the unprivileged attacker (HTTP 200): curl -F uploadfile=@pipeline.yaml -H "kubeflow-userid: attacker@evil.com" \ "http://APISERVER:8888/apis/v2beta1/pipelines/upload?name=shared-pwn" -> {"pipelineid":"80cd99df-...","name":"shared-pwn", ...} 2. Inject a new version into that shared pipeline, the high-impact primitive (HTTP 200): curl -F uploadfile=@pipeline2.yaml -H "kubeflow-userid: attacker@evil.com" \ "http://APISERVER:8888/apis/v2beta1/pipelines/uploadversion?name=evilversion&pipelineid=80cd99df-..." -> {"pipelineversionid":"32299f65-...","name":"evilversion","pipelinespec":{ attacker IR }} 3. Same create with no identity header at all: HTTP 200 (anon-shared-pwn created). 4. Negative control, same attacker, &namespace=victim-ns (real namespace, no RBAC): HTTP 400, rejected at pipelineuploadserver.go:135 (the canUploadVersionedPipeline error path). Authorization is live; only the empty-namespace branch bypasses it. 5. Asymmetry, gRPC POST /apis/v2beta1/pipelines with empty namespace as the same attacker: HTTP 403 PermissionDenied (ResourceAttributes{Namespace:kubeflow,Verb:create,...}). The gRPC path remapped empty to the system namespace and authorized.
API server logs corroborate: every empty-namespace upload logs Upload pipeline called with no following Authorizing request, while every namespaced upload and every gRPC create logs Authorizing request then PermissionDenied. Database state persists the attacker's shared-pwn and anon-shared-pwn pipelines and the injected evilversion; the denied ns-pwn created nothing.
Full scaffolding (run.sh, 01-mysql.yaml through 05-apiserver.yaml, pipeline.yaml, pipeline2.yaml) and captured evidence are available on request.
Impact
In multi-user deployments that permit shared pipelines, a low-privilege tenant can create shared pipelines or add attacker-controlled versions to existing shared pipelines without the required write permission. Direct access to the API server without an authenticating proxy can also expose the bypass to unauthenticated callers; the CVSS vector below assesses the authenticated tenant scenario.
An attacker can upload a malicious version of an existing shared pipeline consumed by a recurring run configured to follow the latest version. On its next scheduled execution, the controller resolves the newly uploaded version and executes it in the consuming run's namespace using its configured service account, without further user interaction. Recurring runs pinned to a specific version or containing an embedded specification are not subject to this automatic substitution.
This automatic path requires an enabled, unpinned recurring run consuming the targeted shared pipeline and an uploaded specification compatible with the run's inputs. Latest-version selection queries the newest ready pipeline version; it does not depend on the UpdatePipelineVersionByDefault setting.
The automatic execution path was verified by source inspection, including the 2.17.2 tag: recurring-run creation preserves an unpinned pipeline reference, the scheduled-workflow controller submits it to CreateRun, and the API resolves the latest version before constructing the workflow. This is distinct from the live upload-bypass reproduction above; an end-to-end scheduled victim-execution reproduction has not been performed as part of this maintainer assessment.
Execution occurs with the victim workload's permissions and access to its mounted data and credentials. It can compromise confidential data and execution integrity available to that workload. It does not inherently grant access to every secret in the victim namespace or cluster-admin privileges.
CVSS rationale
The retained CVSS v3.1 vector is 9.6 Critical — CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:N.
- AV:N / AC:L / PR:L: an authenticated low-privilege tenant can submit a normal multipart version upload targeting a shared pipeline, without shared-pipeline write permission. - UI:N: an existing unpinned recurring run can consume the injected version automatically; no further victim action is required. - S:C: the scored consequence extends from unauthorized modification of the shared pipeline catalog to code execution under a separate victim workload's Kubernetes namespace and service-account authority, not merely modification of another catalog entry. - C:H / I:H: attacker-controlled workload code can compromise the confidentiality and integrity of resources available under that victim execution authority. These impacts do not imply cluster-wide access. - A:N: this vector does not claim an independently established availability impact.
Patches / Remediation
The fix merged into master on September 9, 2026 in kubeflow/pipelines#14352, commit 62516d8c525f146b754cb515cf441dd0290ad841.
The upload authorization helper now uses ResourceManager.IsEmptyNamespace to recognize both the empty string and the legacy model.NoNamespace ("-") representation of shared pipelines. It checks shared writes against the configured KFP system namespace through IsAuthorized, requiring create on the pipelines resource in the pipelines.kubeflow.org API group. This matches the gRPC shared-write policy without changing the stored pipeline namespace. Namespaced authorization, single-user behavior, and intentional latest-version scheduling remain unchanged.
The original fix by Vito Carolillo (@vcrll) was carried forward with its authorship and DCO sign-off preserved. Separate maintainer commits added regression coverage, sentinel handling, and test-harness compatibility corrections.
Validation
- 32 endpoint regression cases cover both v1beta1 and v2beta1 pipeline and pipeline-version uploads: allowed and denied shared writes (empty and sentinel namespaces), namespaced writes, and single-user behavior. Tests verify exact SubjectAccessReview attributes and preservation of stored namespaces. - A negative control restoring the original early return caused the new shared-upload tests to fail, including denied requests returning HTTP 200 without an authorization check. - The full server unit-test package and changed-package lint passed during remediation. - E2E test clients were corrected to send their generated authentication token and target the tenant namespace for HTTP uploads. Kubernetes-native clients retain their constructor-configured namespace rather than receiving an unsupported upload parameter. Helper regression tests and E2E/API package compilation passed.
Release status and upgrade guidance
The first fixed release is 2.18.0. Releases through 2.17.3 remain affected; the affected-version upper bound is <= 2.17.3 and the patched version is 2.18.0.
The 2.18 RBAC and client migration guide is included through PR #14655. Related trusted-scheduling changes do not replace upload authorization or remove previously uploaded versions. Roll out the matching server/manifests and review existing shared versions and schedules.
In multi-user deployments, callers that previously relied on unauthenticated or unauthorized shared uploads will now be rejected:
- UI: choose Private (the default in the upload dialog) for tenant-owned pipelines. Shared uploads require an authorized shared-pipeline writer. - Python SDK: explicitly pass the tenant namespace, for example client.uploadpipeline("pipeline.yaml", namespace="my-namespace"). Setting Client(namespace=...) alone does not populate this upload argument. - New versions: inherit the existing pipeline's namespace. Existing shared pipelines must be maintained by an identity with the required system-namespace pipeline permission, or re-uploaded as new namespaced pipelines with consuming runs and schedules repointed. - Kubernetes-native uploads: retain their existing client-configured namespace contract.
Do not grant blanket system-namespace access or restore the authorization bypass to preserve old upload scripts. Grant only the required pipeline-write permission to designated shared-pipeline maintainers.
As an interim hardening measure before deploying the fix, operators can set REQUIRENAMESPACEFORPIPELINES=true to reject empty-namespace pipeline operations, with the corresponding compatibility impact on shared uploads.
Restoring upload authorization does not remove previously uploaded malicious versions or migrate existing pipelines and runs. Operators should review shared-pipeline versions and consuming recurring runs for unauthorized changes; pinning to a reviewed version prevents subsequent latest-version substitution.
The visualization authorization issue is tracked separately in GHSA-w9j6-6675-75m4 and is outside this fix's scope.
Resources
- backend/src/apiserver/server/pipelineuploadserver.go:387-421 (canUploadVersionedPipeline; bypass at :401-403), :118-138, :290-312 - backend/src/apiserver/server/pipelineserver.go:1290-1306 (canAccessPipeline; load-bearing comment :1290-1292) - backend/src/apiserver/validation/namespace.go:25-30 - backend/src/apiserver/common/config.go:41, :73-74 - backend/src/apiserver/resource/resourcemanager.go (CreateJob, fetchPipelineVersionFromPipelineSpec, CreateRun) - backend/src/crd/controller/scheduledworkflow/controller.go (submitNewWorkflowIfNotAlreadySubmitted) - backend/src/apiserver/storage/pipelinestore.go (GetLatestPipelineVersion) - Distinct from CVE-2023-6570 and CVE-2023-6571 (both in kubeflow/kubeflow, different components and classes).
<!-- maintainer-family-cleanup:2026-09-04 --> Maintainer disposition
This is the canonical advisory for the shared pipeline upload authorization family. Related submissions closed as duplicates or superseded by this canonical:
- GHSA-87q5-j5jh-fhp8
Each linked submission retains its own reporter attribution. A cross-link does not transfer authorship or grant co-reporter credit on this advisory. The fix merged through #14352 on September 9, 2026. The fix is available in 2.18.0. This advisory remains a maintained draft pending publication; CVE-2026-77596 and the 9.6 CVSS score are unchanged.
Private patch disposition
The earlier private PR #1 is closed as superseded by public PR #14352 and the released follow-ups described above. The private branch and review history are retained; this closure does not dismiss the report or change reporter credit. The advisory remains a draft pending the publication review.
Summary
In multi-user mode, the Kubeflow Pipelines API server's HTTP multipart upload endpoints (POST /apis/v2beta1/pipelines/upload and POST /apis/v2beta1/pipelines/uploadversion) perform no authorization when the target namespace is empty. The authorization helper canUploadVersionedPipeline returns nil, skipping SubjectAccessReview, whenever the resolved namespace is the empty string (backend/src/apiserver/server/pipelineuploadserver.go:401-403). An empty namespace is how KFP addresses shared, cluster-wide pipelines and is the default whenever the caller omits the namespace query parameter, because REQUIRENAMESPACEFORPIPELINES defaults to false and is unset in every shipped manifest. As a result, an arbitrary low-privilege tenant (or, when the API server is reachable, an unauthenticated caller) can create cluster-shared pipelines and inject new versions into existing shared pipelines, replacing the container image, command, and args that other tenants execute. The gRPC sibling for the same write, canAccessPipeline, instead remaps the empty namespace to the system namespace and authorizes, with a comment stating shared-pipeline writes "must still be authorized". That confirms this is an omission, not intended behavior.
Description
uploadPipeline and uploadPipelineVersion both build a ResourceAttributes{Verb: create} and delegate the access decision to canUploadVersionedPipeline:
go // backend/src/apiserver/server/pipelineuploadserver.go:387-421 func (s PipelineUploadServer) canUploadVersionedPipeline(r http.Request, pipelineID string, resourceAttributes authorizationv1.ResourceAttributes) error { if !common.IsMultiUserMode() { return nil } if len(pipelineID) > 0 { namespace, err := s.resourceManager.FetchNamespaceFromPipelineId(pipelineID) if err != nil { return util.Wrap(err, "Failed to authorize with the Pipeline ID") } if len(resourceAttributes.Namespace) == 0 { resourceAttributes.Namespace = namespace } } if resourceAttributes.Namespace == "" { return nil // empty namespace = no SubjectAccessReview } // ... only here does IsAuthorized() run ... }
When the namespace is empty (a shared/cluster-wide pipeline, or simply an upload with no namespace parameter), the function returns at line 401-403 and IsAuthorized is never reached.
Reachability is the default. ValidateNamespaceRequired only rejects an empty namespace when IsNamespaceRequiredForPipelines() is true:
go // backend/src/apiserver/validation/namespace.go:25-30 func ValidateNamespaceRequired(namespace string) error { if common.IsMultiUserMode() && common.IsNamespaceRequiredForPipelines() && namespace == "" { return util.NewInvalidInputError("Namespace is required for pipeline operations") } return nil } // backend/src/apiserver/common/config.go:73-74 -> default FALSE func IsNamespaceRequiredForPipelines() bool { return GetBoolConfigWithDefault(RequireNamespaceForPipelines, false) }
REQUIRENAMESPACEFORPIPELINES (config.go:41) defaults to false and is not set in any manifest under manifests/. So in a default multi-user install, empty namespaces pass validation and flow into the unguarded branch.
The gRPC sibling proves the intended behavior is to authorize empty-namespace writes:
go // backend/src/apiserver/server/pipelineserver.go:1290-1306 (canAccessPipeline) // Skip authorization for read-only operations on shared pipelines in multi-user mode. // Write operations (create, update, delete) on shared pipelines must still be authorized // against the KFP system namespace, since shared pipelines have no namespace of their own. if s.resourceManager.IsEmptyNamespace(resourceAttributes.Namespace) { if resourceAttributes.Verb == common.RbacResourceVerbGet || resourceAttributes.Verb == common.RbacResourceVerbList { return nil } resourceAttributes.Namespace = common.GetPodNamespace() // ... IsAuthorized(...) ... return nil }
Reads of shared pipelines intentionally skip authorization; writes must be authorized against the system namespace. The upload endpoints only ever perform writes, yet canUploadVersionedPipeline applies the read behavior to them.
Proof of Concept
Reproduced live against the real API server binary (ghcr.io/kubeflow/kfp-api-server@sha256:621fdae9bfcc93405eca299c24f715bc7f17dbb76f1b53e8b2b35852e4d7d423, MULTIUSER=true, REQUIRENAMESPACEFORPIPELINES unset). The API server SA holds subjectaccessreviews/tokenreviews: [create] (authz fully functional); identity attacker@evil.com has no RBAC anywhere; victim-ns exists with no grant to the attacker.
1. Shared (empty-namespace) pipeline create as the unprivileged attacker (HTTP 200): curl -F uploadfile=@pipeline.yaml -H "kubeflow-userid: attacker@evil.com" \ "http://APISERVER:8888/apis/v2beta1/pipelines/upload?name=shared-pwn" -> {"pipelineid":"80cd99df-...","name":"shared-pwn", ...} 2. Inject a new version into that shared pipeline, the high-impact primitive (HTTP 200): curl -F uploadfile=@pipeline2.yaml -H "kubeflow-userid: attacker@evil.com" \ "http://APISERVER:8888/apis/v2beta1/pipelines/uploadversion?name=evilversion&pipelineid=80cd99df-..." -> {"pipelineversionid":"32299f65-...","name":"evilversion","pipelinespec":{ attacker IR }} 3. Same create with no identity header at all: HTTP 200 (anon-shared-pwn created). 4. Negative control, same attacker, &namespace=victim-ns (real namespace, no RBAC): HTTP 400, rejected at pipelineuploadserver.go:135 (the canUploadVersionedPipeline error path). Authorization is live; only the empty-namespace branch bypasses it. 5. Asymmetry, gRPC POST /apis/v2beta1/pipelines with empty namespace as the same attacker: HTTP 403 PermissionDenied (ResourceAttributes{Namespace:kubeflow,Verb:create,...}). The gRPC path remapped empty to the system namespace and authorized.
API server logs corroborate: every empty-namespace upload logs Upload pipeline called with no following Authorizing request, while every namespaced upload and every gRPC create logs Authorizing request then PermissionDenied. Database state persists the attacker's shared-pwn and anon-shared-pwn pipelines and the injected evilversion; the denied ns-pwn created nothing.
Full scaffolding (run.sh, 01-mysql.yaml through 05-apiserver.yaml, pipeline.yaml, pipeline2.yaml) and captured evidence are available on request.
Impact
In multi-user deployments that permit shared pipelines, a low-privilege tenant can create shared pipelines or add attacker-controlled versions to existing shared pipelines without the required write permission. Direct access to the API server without an authenticating proxy can also expose the bypass to unauthenticated callers; the CVSS vector below assesses the authenticated tenant scenario.
An attacker can upload a malicious version of an existing shared pipeline consumed by a recurring run configured to follow the latest version. On its next scheduled execution, the controller resolves the newly uploaded version and executes it in the consuming run's namespace using its configured service account, without further user interaction. Recurring runs pinned to a specific version or containing an embedded specification are not subject to this automatic substitution.
This automatic path requires an enabled, unpinned recurring run consuming the targeted shared pipeline and an uploaded specification compatible with the run's inputs. Latest-version selection queries the newest ready pipeline version; it does not depend on the UpdatePipelineVersionByDefault setting.
The automatic execution path was verified by source inspection, including the 2.17.2 tag: recurring-run creation preserves an unpinned pipeline reference, the scheduled-workflow controller submits it to CreateRun, and the API resolves the latest version before constructing the workflow. This is distinct from the live upload-bypass reproduction above; an end-to-end scheduled victim-execution reproduction has not been performed as part of this maintainer assessment.
Execution occurs with the victim workload's permissions and access to its mounted data and credentials. It can compromise confidential data and execution integrity available to that workload. It does not inherently grant access to every secret in the victim namespace or cluster-admin privileges.
CVSS rationale
The retained CVSS v3.1 vector is 9.6 Critical — CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:N.
- AV:N / AC:L / PR:L: an authenticated low-privilege tenant can submit a normal multipart version upload targeting a shared pipeline, without shared-pipeline write permission. - UI:N: an existing unpinned recurring run can consume the injected version automatically; no further victim action is required. - S:C: the scored consequence extends from unauthorized modification of the shared pipeline catalog to code execution under a separate victim workload's Kubernetes namespace and service-account authority, not merely modification of another catalog entry. - C:H / I:H: attacker-controlled workload code can compromise the confidentiality and integrity of resources available under that victim execution authority. These impacts do not imply cluster-wide access. - A:N: this vector does not claim an independently established availability impact.
Patches / Remediation
The fix merged into master on September 9, 2026 in kubeflow/pipelines#14352, commit 62516d8c525f146b754cb515cf441dd0290ad841.
The upload authorization helper now uses ResourceManager.IsEmptyNamespace to recognize both the empty string and the legacy model.NoNamespace ("-") representation of shared pipelines. It checks shared writes against the configured KFP system namespace through IsAuthorized, requiring create on the pipelines resource in the pipelines.kubeflow.org API group. This matches the gRPC shared-write policy without changing the stored pipeline namespace. Namespaced authorization, single-user behavior, and intentional latest-version scheduling remain unchanged.
The original fix by Vito Carolillo (@vcrll) was carried forward with its authorship and DCO sign-off preserved. Separate maintainer commits added regression coverage, sentinel handling, and test-harness compatibility corrections.
Validation
- 32 endpoint regression cases cover both v1beta1 and v2beta1 pipeline and pipeline-version uploads: allowed and denied shared writes (empty and sentinel namespaces), namespaced writes, and single-user behavior. Tests verify exact SubjectAccessReview attributes and preservation of stored namespaces. - A negative control restoring the original early return caused the new shared-upload tests to fail, including denied requests returning HTTP 200 without an authorization check. - The full server unit-test package and changed-package lint passed during remediation. - E2E test clients were corrected to send their generated authentication token and target the tenant namespace for HTTP uploads. Kubernetes-native clients retain their constructor-configured namespace rather than receiving an unsupported upload parameter. Helper regression tests and E2E/API package compilation passed.
Release status and upgrade guidance
The first fixed release is 2.18.0. Releases through 2.17.3 remain affected; the affected-version upper bound is <= 2.17.3 and the patched version is 2.18.0.
The 2.18 RBAC and client migration guide is included through PR #14655. Related trusted-scheduling changes do not replace upload authorization or remove previously uploaded versions. Roll out the matching server/manifests and review existing shared versions and schedules.
In multi-user deployments, callers that previously relied on unauthenticated or unauthorized shared uploads will now be rejected:
- UI: choose Private (the default in the upload dialog) for tenant-owned pipelines. Shared uploads require an authorized shared-pipeline writer. - Python SDK: explicitly pass the tenant namespace, for example client.uploadpipeline("pipeline.yaml", namespace="my-namespace"). Setting Client(namespace=...) alone does not populate this upload argument. - New versions: inherit the existing pipeline's namespace. Existing shared pipelines must be maintained by an identity with the required system-namespace pipeline permission, or re-uploaded as new namespaced pipelines with consuming runs and schedules repointed. - Kubernetes-native uploads: retain their existing client-configured namespace contract.
Do not grant blanket system-namespace access or restore the authorization bypass to preserve old upload scripts. Grant only the required pipeline-write permission to designated shared-pipeline maintainers.
As an interim hardening measure before deploying the fix, operators can set REQUIRENAMESPACEFORPIPELINES=true to reject empty-namespace pipeline operations, with the corresponding compatibility impact on shared uploads.
Restoring upload authorization does not remove previously uploaded malicious versions or migrate existing pipelines and runs. Operators should review shared-pipeline versions and consuming recurring runs for unauthorized changes; pinning to a reviewed version prevents subsequent latest-version substitution.
The visualization authorization issue is tracked separately in GHSA-w9j6-6675-75m4 and is outside this fix's scope.
Resources
- backend/src/apiserver/server/pipelineuploadserver.go:387-421 (canUploadVersionedPipeline; bypass at :401-403), :118-138, :290-312 - backend/src/apiserver/server/pipelineserver.go:1290-1306 (canAccessPipeline; load-bearing comment :1290-1292) - backend/src/apiserver/validation/namespace.go:25-30 - backend/src/apiserver/common/config.go:41, :73-74 - backend/src/apiserver/resource/resourcemanager.go (CreateJob, fetchPipelineVersionFromPipelineSpec, CreateRun) - backend/src/crd/controller/scheduledworkflow/controller.go (submitNewWorkflowIfNotAlreadySubmitted) - backend/src/apiserver/storage/pipelinestore.go (GetLatestPipelineVersion) - Distinct from CVE-2023-6570 and CVE-2023-6571 (both in kubeflow/kubeflow, different components and classes).
<!-- maintainer-family-cleanup:2026-09-04 --> Maintainer disposition
This is the canonical advisory for the shared pipeline upload authorization family. Related submissions closed as duplicates or superseded by this canonical:
- GHSA-87q5-j5jh-fhp8
Each linked submission retains its own reporter attribution. A cross-link does not transfer authorship or grant co-reporter credit on this advisory. The fix merged through #14352 on September 9, 2026. The fix is available in 2.18.0. This advisory remains a maintained draft pending publication; CVE-2026-77596 and the 9.6 CVSS score are unchanged.
Private patch disposition
The earlier private PR #1 is closed as superseded by public PR #14352 and the released follow-ups described above. The private branch and review history are retained; this closure does not dismiss the report or change reporter credit. The advisory remains a draft pending the publication review.
Summary
The legacy Kubeflow Pipelines V1/raw-Argo execution cache does not separate cached results by Kubernetes namespace. In multi-user deployments where tenants share the cache service, a tenant can receive another tenant's cached outputs or seed outputs that are subsequently substituted into another tenant's step.
This affects the Argo-template cache mutating webhook and its cache-server storage path. Standard KFP V2 component pods bypass this webhook cache; the separate V2 cache implementation is outside this advisory's scope.
Affected deployments and impact
The issue requires multiple tenant namespaces sharing the legacy cache service, cache-enabled V1/raw-Argo steps, and cache entries that satisfy the configured staleness policy. The standard multi-user configuration watches pods across namespaces.
A tenant able to submit a step with the same cache-relevant template fields as another tenant's step can:
- Read cached step outputs originally produced in another namespace. - Populate the shared cache before another tenant's matching step runs, causing that step to report the cached outputs instead of executing its intended containers.
Matching refers to the selected template fields used by the cache key, not merely an identical container image. This is an application-level cache-key collision caused by missing tenant scope, not a cryptographic SHA-256 collision. The finding establishes cross-namespace confidentiality and integrity impact; it does not establish arbitrary code execution.
CVSS rationale
Medium — CVSS 3.1: 5.4
CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:L/A:N
- AV:N / PR:L / UI:N: a tenant with ordinary workload-submission privileges can exercise the shared cache through network-accessible APIs. Reading an existing cached result or poisoning a subsequent automated execution requires no additional victim interaction. - AC:L: known or shared templates can produce the same cache key deterministically. Cache-enabled execution and matching template fields describe the affected configuration; they do not establish an unavoidable narrow race or secret prerequisite. - S:U: the established impact is disclosure or substitution of output data managed by the same cache service. Crossing tenant namespaces alone does not establish a change in security authority; execution under a separate victim authority has not been established here. - C:L / I:L / A:N: the confirmed impact is selective cached-output disclosure and substitution. Arbitrary code execution, complete workload compromise, and a service-availability impact are not established.
Root cause
The vulnerable implementation hashes selected Argo template fields without a namespace. Cache lookups and duplicate checks use the key alone, and stored rows have no namespace ownership field. On a hit, the admission handler restores cached outputs and substitutes a dummy container for the step's containers.
The completion watcher also trusts the mutable pipelines.kubeflow.org/executioncachekey annotation when publishing outputs. Adding a namespace to the hash alone is therefore insufficient: a tenant able to patch its own pod could supply another namespace's deterministic key. Namespace ownership must be enforced independently on cache writes and reads.
Relevant implementation files are backend/src/cache/server/mutation.go, backend/src/cache/server/watcher.go, backend/src/cache/storage/executioncachestore.go, and backend/src/cache/model/executioncache.go.
Fix status
The source fix merged on September 10, 2026 in PR #14359, commit 620d0cc89ae0d042a484a003f8ffe6e9a55d347e.
The merged fix:
- Derives namespaced keys from the API server's admission request namespace and selected template fields. - Recomputes completed pods' keys from their observed namespace and ARGOTEMPLATE, rejecting mismatched or malformed identities rather than trusting the annotation. - Stores the observed namespace separately and scopes normal reads and duplicate detection by namespace and key. - Adds a namespace column and composite index, leaving historical rows unowned and excluded from normal lookups. - Rejects structurally malformed template objects without panicking.
The first fixed release is 2.18.0. Releases through 2.17.3 remain affected; the affected-version upper bound is <= 2.17.3 and the patched version is 2.18.0.
Upgrade requirements and legacy fallback
The standard cache-server Deployment supports a normal rolling upgrade from 2.17.2. Apply the upgraded manifests without manually stopping old replicas or pausing submissions. Namespace isolation is fully enforced only after all old cache-server processes, including terminating pods, have exited and every new replica uses enforce. Custom installations sharing the cache database must upgrade every writer. Mixed-version operation does not yet provide the full isolation guarantee.
Expect reduced cache reuse and increased step execution/resource use as tasks next run; there is no startup cache rebuild. Do not infer or backfill namespace ownership for historical rows. Pre-upgrade pods carrying old-format keys do not seed the new namespaced cache. Completing the upgrade does not undo previously disclosed data or reused poisoned outputs. Do not roll back to the vulnerable implementation to recover cache hits.
PR #14427 merged into release-2.18 on October 5, 2026 and standardizes the compatibility setting as KFPSECURITYLEGACYCACHEMODE=enforce|audit on the cache-server Deployment. This supersedes the original fix's ALLOWLEGACYCACHEFALLBACK interface. Unset, empty, or enforce keeps full namespace isolation; invalid values prevent startup.
Explicit audit permits a genuine namespaced cache miss to reuse a historical template-only entry whose stored namespace is empty. That entry may contain another tenant's outputs, including poisoned outputs: audit accepts residual cross-tenant confidentiality and integrity risk, and is not full remediation. Scoped entries take precedence, known foreign namespaces remain excluded, and a database error does not enable legacy fallback. Legacy hits are not promoted or backfilled into namespaced rows.
The server emits a startup warning and scoped securityaudit events for legacy reuse without logging cached outputs or credentials. Keep replicas configured consistently, persist the mode in deployment configuration, and return to enforce after migration. Native V2 caching is outside this control and this advisory.
See the released cache-server guide for rollout completion checks and audit-to-enforce migration. This fix does not establish an integrity boundary between users who can modify workloads within the same namespace.
Mitigation until the fixed release is deployed
Disable legacy cache reuse for affected workloads, or ensure mutually untrusted namespaces do not share the legacy cache service and storage. Restricting the webhook's namespace selector or watcher scope alone is insufficient if mutually untrusted tenants still share cached entries.
Validation
Source review identified the shared key, unscoped storage, and mutable-annotation write path. The public fix includes regressions exercising admission and completion handling, forged foreign-namespace keys, normal same-namespace reuse, malformed templates, legacy-row exclusion, and optional fallback behavior.
During development, the cache test suite and lint passed; populated and repeatable schema migration and scoped storage behavior were verified on SQLite, MySQL 8.4, and PostgreSQL 16. Negative controls removing the watcher identity check or storage namespace filter failed their corresponding regressions. These tests use fake Kubernetes clients; they are not a claim of a live multi-tenant Kubernetes exploit reproduction.
Private patch disposition
The earlier private PR #1 is closed as superseded by public PR #14359 and the released follow-ups described above. The private branch and review history are retained; this closure does not dismiss the report or change reporter credit. The advisory remains a draft pending the publication review.
Summary
The legacy Kubeflow Pipelines V1/raw-Argo execution cache does not separate cached results by Kubernetes namespace. In multi-user deployments where tenants share the cache service, a tenant can receive another tenant's cached outputs or seed outputs that are subsequently substituted into another tenant's step.
This affects the Argo-template cache mutating webhook and its cache-server storage path. Standard KFP V2 component pods bypass this webhook cache; the separate V2 cache implementation is outside this advisory's scope.
Affected deployments and impact
The issue requires multiple tenant namespaces sharing the legacy cache service, cache-enabled V1/raw-Argo steps, and cache entries that satisfy the configured staleness policy. The standard multi-user configuration watches pods across namespaces.
A tenant able to submit a step with the same cache-relevant template fields as another tenant's step can:
- Read cached step outputs originally produced in another namespace. - Populate the shared cache before another tenant's matching step runs, causing that step to report the cached outputs instead of executing its intended containers.
Matching refers to the selected template fields used by the cache key, not merely an identical container image. This is an application-level cache-key collision caused by missing tenant scope, not a cryptographic SHA-256 collision. The finding establishes cross-namespace confidentiality and integrity impact; it does not establish arbitrary code execution.
CVSS rationale
Medium — CVSS 3.1: 5.4
CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:L/A:N
- AV:N / PR:L / UI:N: a tenant with ordinary workload-submission privileges can exercise the shared cache through network-accessible APIs. Reading an existing cached result or poisoning a subsequent automated execution requires no additional victim interaction. - AC:L: known or shared templates can produce the same cache key deterministically. Cache-enabled execution and matching template fields describe the affected configuration; they do not establish an unavoidable narrow race or secret prerequisite. - S:U: the established impact is disclosure or substitution of output data managed by the same cache service. Crossing tenant namespaces alone does not establish a change in security authority; execution under a separate victim authority has not been established here. - C:L / I:L / A:N: the confirmed impact is selective cached-output disclosure and substitution. Arbitrary code execution, complete workload compromise, and a service-availability impact are not established.
Root cause
The vulnerable implementation hashes selected Argo template fields without a namespace. Cache lookups and duplicate checks use the key alone, and stored rows have no namespace ownership field. On a hit, the admission handler restores cached outputs and substitutes a dummy container for the step's containers.
The completion watcher also trusts the mutable pipelines.kubeflow.org/executioncachekey annotation when publishing outputs. Adding a namespace to the hash alone is therefore insufficient: a tenant able to patch its own pod could supply another namespace's deterministic key. Namespace ownership must be enforced independently on cache writes and reads.
Relevant implementation files are backend/src/cache/server/mutation.go, backend/src/cache/server/watcher.go, backend/src/cache/storage/executioncachestore.go, and backend/src/cache/model/executioncache.go.
Fix status
The source fix merged on September 10, 2026 in PR #14359, commit 620d0cc89ae0d042a484a003f8ffe6e9a55d347e.
The merged fix:
- Derives namespaced keys from the API server's admission request namespace and selected template fields. - Recomputes completed pods' keys from their observed namespace and ARGOTEMPLATE, rejecting mismatched or malformed identities rather than trusting the annotation. - Stores the observed namespace separately and scopes normal reads and duplicate detection by namespace and key. - Adds a namespace column and composite index, leaving historical rows unowned and excluded from normal lookups. - Rejects structurally malformed template objects without panicking.
The first fixed release is 2.18.0. Releases through 2.17.3 remain affected; the affected-version upper bound is <= 2.17.3 and the patched version is 2.18.0.
Upgrade requirements and legacy fallback
The standard cache-server Deployment supports a normal rolling upgrade from 2.17.2. Apply the upgraded manifests without manually stopping old replicas or pausing submissions. Namespace isolation is fully enforced only after all old cache-server processes, including terminating pods, have exited and every new replica uses enforce. Custom installations sharing the cache database must upgrade every writer. Mixed-version operation does not yet provide the full isolation guarantee.
Expect reduced cache reuse and increased step execution/resource use as tasks next run; there is no startup cache rebuild. Do not infer or backfill namespace ownership for historical rows. Pre-upgrade pods carrying old-format keys do not seed the new namespaced cache. Completing the upgrade does not undo previously disclosed data or reused poisoned outputs. Do not roll back to the vulnerable implementation to recover cache hits.
PR #14427 merged into release-2.18 on October 5, 2026 and standardizes the compatibility setting as KFPSECURITYLEGACYCACHEMODE=enforce|audit on the cache-server Deployment. This supersedes the original fix's ALLOWLEGACYCACHEFALLBACK interface. Unset, empty, or enforce keeps full namespace isolation; invalid values prevent startup.
Explicit audit permits a genuine namespaced cache miss to reuse a historical template-only entry whose stored namespace is empty. That entry may contain another tenant's outputs, including poisoned outputs: audit accepts residual cross-tenant confidentiality and integrity risk, and is not full remediation. Scoped entries take precedence, known foreign namespaces remain excluded, and a database error does not enable legacy fallback. Legacy hits are not promoted or backfilled into namespaced rows.
The server emits a startup warning and scoped securityaudit events for legacy reuse without logging cached outputs or credentials. Keep replicas configured consistently, persist the mode in deployment configuration, and return to enforce after migration. Native V2 caching is outside this control and this advisory.
See the released cache-server guide for rollout completion checks and audit-to-enforce migration. This fix does not establish an integrity boundary between users who can modify workloads within the same namespace.
Mitigation until the fixed release is deployed
Disable legacy cache reuse for affected workloads, or ensure mutually untrusted namespaces do not share the legacy cache service and storage. Restricting the webhook's namespace selector or watcher scope alone is insufficient if mutually untrusted tenants still share cached entries.
Validation
Source review identified the shared key, unscoped storage, and mutable-annotation write path. The public fix includes regressions exercising admission and completion handling, forged foreign-namespace keys, normal same-namespace reuse, malformed templates, legacy-row exclusion, and optional fallback behavior.
During development, the cache test suite and lint passed; populated and repeatable schema migration and scoped storage behavior were verified on SQLite, MySQL 8.4, and PostgreSQL 16. Negative controls removing the watcher identity check or storage namespace filter failed their corresponding regressions. These tests use fake Kubernetes clients; they are not a claim of a live multi-tenant Kubernetes exploit reproduction.
Private patch disposition
The earlier private PR #1 is closed as superseded by public PR #14359 and the released follow-ups described above. The private branch and review history are retained; this closure does not dismiss the report or change reporter credit. The advisory remains a draft pending the publication review.
Summary
In multi-user mode, the Kubeflow Pipelines API server skipped authorization checks for shared/global pipelines with an empty namespace. This exception applied to write and delete operations as well as reads, allowing an authenticated user to modify or delete shared pipelines without the required Kubernetes RBAC permissions.
Impact
Unauthorized modification compromises the integrity of shared pipeline definitions and versions. Unauthorized deletion can make shared pipelines unavailable.
Execution of a modified pipeline can cause attacker-controlled workload behavior, but this is a conditional downstream consequence requiring the modified pipeline to be executed. Modification alone does not demonstrate immediate worker code execution.
Reading shared pipelines is intentionally permitted and remains permitted after the fix. Consequently, access to shared pipeline configuration does not establish an additional confidentiality impact from this write-authorization defect. The confidentiality metric is None; the integrity and availability metrics reflect unauthorized modification and deletion of shared pipelines.
The affected authorization boundary is shared pipelines in multi-user deployments. Single-user deployments intentionally bypass these multi-user authorization checks.
Patches
Fixed by kubeflow/pipelines#13288, with the fix included in 2.16.1.
Read-only operations on shared pipelines remain allowed. For write and delete operations, the API server checks Kubernetes RBAC permissions against the KFP system namespace, because shared pipelines have no namespace of their own.
Scope of this fix
The 2.16.1 fix addresses the shared-pipeline authorization paths described above. It does not fix every shared-pipeline write path: the separate shared-pipeline upload authorization defect is tracked in GHSA-rjr3-43j3-4q8g / CVE-2026-77596. Consult that advisory for its affected versions and remediation; 2.16.1 must not be interpreted as fixing that separate upload defect.
Workarounds
Until upgrading, restrict access to the Pipelines API to trusted users through deployment-level access controls. Namespace RBAC alone does not correct the affected shared-pipeline authorization path.
Summary
In multi-user mode, the Kubeflow Pipelines API server skipped authorization checks for shared/global pipelines with an empty namespace. This exception applied to write and delete operations as well as reads, allowing an authenticated user to modify or delete shared pipelines without the required Kubernetes RBAC permissions.
Impact
Unauthorized modification compromises the integrity of shared pipeline definitions and versions. Unauthorized deletion can make shared pipelines unavailable.
Execution of a modified pipeline can cause attacker-controlled workload behavior, but this is a conditional downstream consequence requiring the modified pipeline to be executed. Modification alone does not demonstrate immediate worker code execution.
Reading shared pipelines is intentionally permitted and remains permitted after the fix. Consequently, access to shared pipeline configuration does not establish an additional confidentiality impact from this write-authorization defect. The confidentiality metric is None; the integrity and availability metrics reflect unauthorized modification and deletion of shared pipelines.
The affected authorization boundary is shared pipelines in multi-user deployments. Single-user deployments intentionally bypass these multi-user authorization checks.
Patches
Fixed by kubeflow/pipelines#13288, with the fix included in 2.16.1.
Read-only operations on shared pipelines remain allowed. For write and delete operations, the API server checks Kubernetes RBAC permissions against the KFP system namespace, because shared pipelines have no namespace of their own.
Scope of this fix
The 2.16.1 fix addresses the shared-pipeline authorization paths described above. It does not fix every shared-pipeline write path: the separate shared-pipeline upload authorization defect is tracked in GHSA-rjr3-43j3-4q8g / CVE-2026-77596. Consult that advisory for its affected versions and remediation; 2.16.1 must not be interpreted as fixing that separate upload defect.
Workarounds
Until upgrading, restrict access to the Pipelines API to trusted users through deployment-level access controls. Namespace RBAC alone does not correct the affected shared-pipeline authorization path.
Summary
The Kubeflow Pipelines API server fetched caller-supplied pipeline package URLs without restricting the destination. A caller authorized to create a pipeline or pipeline version could cause requests to originate from the API server's network position, including to internal services that the caller could not otherwise reach.
Impact
The reported evidence demonstrates blind server-side request forgery and a limited status-code oracle. The fetched response is processed as a pipeline package; the report does not demonstrate arbitrary response-body disclosure or credential theft. A request to a cloud metadata endpoint returned HTTP 403, which does not establish access to metadata contents or tokens.
The confidentiality metric is Low to reflect the limited information obtainable through the status-code oracle. High confidentiality impact and integrity or availability impact are not established by the submitted evidence. Impact depends on the API server's network reachability and the deployment's trust boundaries.
Patches
Fixed by kubeflow/pipelines#12750, first released in 2.17.0. The fix validates URL schemes, ports, allowed domains, and resolved IP addresses; validates redirects; and enforces IP restrictions at connection time. The HTTP client also applies timeouts and disables environment proxy use so connections pass through the validated dialer.
Operators should keep pipeline URL validation enabled and limit additional allowed domains to trusted pipeline sources.
Workarounds
Until upgrading, restrict API-server egress to required destinations through deployment-specific network controls. Account for required Kubernetes and storage connectivity when applying these restrictions.
Related reports
This is the canonical advisory for the API-server pipeline URL SSRF family. GHSA-wg4h-pp3x-hxhv is tracked as a duplicate submission. Each submission retains its own reporter attribution; this cross-link does not transfer authorship.
Earlier related submission
@pwny0's GHSA-fxx8-rx8w-p9v7, submitted September 23, 2023, included an earlier report of external interaction through the API-server pipeline-package URL path. That portion is consolidated here; its namespace-based frontend SSRF is tracked separately under GHSA-895m-3hxm-h4mg / CVE-2023-6570.
This acknowledges the earlier pipeline-URL submission without attributing fxx8's frontend response-disclosure or cookie-forwarding evidence to the API-server path. @pwny0 retains reporter credit on fxx8; @gasbugs retains reporter credit for this submission. Neither report's authorship is transferred.
Summary
The Kubeflow Pipelines API server fetched caller-supplied pipeline package URLs without restricting the destination. A caller authorized to create a pipeline or pipeline version could cause requests to originate from the API server's network position, including to internal services that the caller could not otherwise reach.
Impact
The reported evidence demonstrates blind server-side request forgery and a limited status-code oracle. The fetched response is processed as a pipeline package; the report does not demonstrate arbitrary response-body disclosure or credential theft. A request to a cloud metadata endpoint returned HTTP 403, which does not establish access to metadata contents or tokens.
The confidentiality metric is Low to reflect the limited information obtainable through the status-code oracle. High confidentiality impact and integrity or availability impact are not established by the submitted evidence. Impact depends on the API server's network reachability and the deployment's trust boundaries.
Patches
Fixed by kubeflow/pipelines#12750, first released in 2.17.0. The fix validates URL schemes, ports, allowed domains, and resolved IP addresses; validates redirects; and enforces IP restrictions at connection time. The HTTP client also applies timeouts and disables environment proxy use so connections pass through the validated dialer.
Operators should keep pipeline URL validation enabled and limit additional allowed domains to trusted pipeline sources.
Workarounds
Until upgrading, restrict API-server egress to required destinations through deployment-specific network controls. Account for required Kubernetes and storage connectivity when applying these restrictions.
Related reports
This is the canonical advisory for the API-server pipeline URL SSRF family. GHSA-wg4h-pp3x-hxhv is tracked as a duplicate submission. Each submission retains its own reporter attribution; this cross-link does not transfer authorship.
Earlier related submission
@pwny0's GHSA-fxx8-rx8w-p9v7, submitted September 23, 2023, included an earlier report of external interaction through the API-server pipeline-package URL path. That portion is consolidated here; its namespace-based frontend SSRF is tracked separately under GHSA-895m-3hxm-h4mg / CVE-2023-6570.
This acknowledges the earlier pipeline-URL submission without attributing fxx8's frontend response-disclosure or cookie-forwarding evidence to the API-server path. @pwny0 retains reporter credit on fxx8; @gasbugs retains reporter credit for this submission. Neither report's authorship is transferred.
Kubeflow Pipelines enables users to build and deploy portable, scalable machine learning workflows. Prior to 2.18.0, the frontend GET /artifacts/get route accepts source=http or source=https with caller-controlled bucket and key values, and isAllowedResourceName permits an all-digit decimal host that the URL parser converts to an IPv4 address. The default ALLOWEDARTIFACTDOMAINREGEX and authorization-disabled standalone configuration allow an unauthenticated caller to fetch internal HTTP resources and receive their response body. In the authenticated multi-user configuration, the same destination-validation defect is reachable by a low-privilege tenant with a valid identity and authorized namespace. Cloud credential disclosure depends on reachable metadata services and attached roles, and the issue does not establish IMDSv2 bypass or credential access when metadata is blocked. This issue is fixed in version 2.18.0.
Kubeflow Pipelines enables users to build and deploy portable, scalable machine learning workflows. Prior to 2.18.0, the frontend GET /artifacts/get route accepts source=http or source=https with caller-controlled bucket and key values, and isAllowedResourceName permits an all-digit decimal host that the URL parser converts to an IPv4 address. The default ALLOWEDARTIFACTDOMAINREGEX and authorization-disabled standalone configuration allow an unauthenticated caller to fetch internal HTTP resources and receive their response body. In the authenticated multi-user configuration, the same destination-validation defect is reachable by a low-privilege tenant with a valid identity and authorized namespace. Cloud credential disclosure depends on reachable metadata services and attached roles, and the issue does not establish IMDSv2 bypass or credential access when metadata is blocked. This issue is fixed in version 2.18.0.
Summary
Kubeflow Pipelines API server does not sufficiently validate SQL identifier fields decoded from the pagetoken pagination parameter. Client-controlled fields can reach SQL query construction, allowing SQL injection through list APIs.
This advisory consolidates the original pipeline-listing report and the later run-listing report in GHSA-mhpc-2x8q-2vp9. Their contributions and chronology are distinguished below.
Impact and prerequisites
An attacker able to invoke an affected list API may read data accessible to the API server's database account, including data belonging to other namespaces. The original report demonstrated database read access through an authenticated Kubeflow deployment.
Authentication and authorization requirements depend on deployment configuration. In multi-user deployments, the attacker must pass the applicable endpoint authorization checks. A single-user deployment that exposes the API without an authentication layer may be reachable without credentials.
The submitted evidence establishes a confidentiality issue. Database modification was not demonstrated; this advisory does not claim arbitrary database writes.
Technical details
Pagination tokens are base64-encoded JSON, not trusted server-side state. Before the fix, token deserialization accepted structural fields such as key and sort identifiers without sufficient validation before SQL construction.
The reports describe complementary paths through this defect:
- Pipeline listing: the original report demonstrated injection through KeyFieldName. - Run listing: the later report documented SortByFieldName reaching query construction in both the v1beta1 and v2beta1 APIs, including metric joins, selected column aliases, ordering, and column propagation through resource-reference and task joins.
These paths share the unvalidated pagination-token boundary. Review of the later report and its discussion identified no reported path left unresolved by the merged fix.
Remediation
Upgrade the Kubeflow Pipelines API server to 2.17.0 or later.
PR #13127, included in 2.17.0, validates token identifiers and filter keys before query construction. It also separates user-facing metric names from SQL identifiers, uses a fixed metric-column alias, and parameterizes the metric-name join condition.
The PR additionally addresses first-page metric sorting without a pagination token. That additional protection is part of the remediation; the duplicate determination here concerns the page-token claims actually submitted in the two reports, not merely their inclusion in the same PR.
The original report identified its deployment as Kubeflow 1.7, which should not be interpreted as a verified Kubeflow Pipelines component-version boundary. The later report identified KFP 2.16.0. These are reported affected environments, not an established introduction version.
Attribution and report history
- @phisch1991 and @k0rti: credited reporters of the original pipeline-listing SQL injection submitted on December 13, 2023. - BugBunny.ai (@bugbunny-research): submitted the later report on April 1, 2026, documenting the run-listing and metric-query paths in GHSA-mhpc-2x8q-2vp9. This acknowledges that specific contribution without implying authorship of the 2023 report. - Cyber Nagle: authored the merged remediation in PR #13127.
GHSA-mhpc-2x8q-2vp9 is consolidated here for vulnerability tracking. Its submission and reporter credit remain preserved on that advisory; closure as a duplicate does not characterize the report as invalid.
Summary
Kubeflow Pipelines API server does not sufficiently validate SQL identifier fields decoded from the pagetoken pagination parameter. Client-controlled fields can reach SQL query construction, allowing SQL injection through list APIs.
This advisory consolidates the original pipeline-listing report and the later run-listing report in GHSA-mhpc-2x8q-2vp9. Their contributions and chronology are distinguished below.
Impact and prerequisites
An attacker able to invoke an affected list API may read data accessible to the API server's database account, including data belonging to other namespaces. The original report demonstrated database read access through an authenticated Kubeflow deployment.
Authentication and authorization requirements depend on deployment configuration. In multi-user deployments, the attacker must pass the applicable endpoint authorization checks. A single-user deployment that exposes the API without an authentication layer may be reachable without credentials.
The submitted evidence establishes a confidentiality issue. Database modification was not demonstrated; this advisory does not claim arbitrary database writes.
Technical details
Pagination tokens are base64-encoded JSON, not trusted server-side state. Before the fix, token deserialization accepted structural fields such as key and sort identifiers without sufficient validation before SQL construction.
The reports describe complementary paths through this defect:
- Pipeline listing: the original report demonstrated injection through KeyFieldName. - Run listing: the later report documented SortByFieldName reaching query construction in both the v1beta1 and v2beta1 APIs, including metric joins, selected column aliases, ordering, and column propagation through resource-reference and task joins.
These paths share the unvalidated pagination-token boundary. Review of the later report and its discussion identified no reported path left unresolved by the merged fix.
Remediation
Upgrade the Kubeflow Pipelines API server to 2.17.0 or later.
PR #13127, included in 2.17.0, validates token identifiers and filter keys before query construction. It also separates user-facing metric names from SQL identifiers, uses a fixed metric-column alias, and parameterizes the metric-name join condition.
The PR additionally addresses first-page metric sorting without a pagination token. That additional protection is part of the remediation; the duplicate determination here concerns the page-token claims actually submitted in the two reports, not merely their inclusion in the same PR.
The original report identified its deployment as Kubeflow 1.7, which should not be interpreted as a verified Kubeflow Pipelines component-version boundary. The later report identified KFP 2.16.0. These are reported affected environments, not an established introduction version.
Attribution and report history
- @phisch1991 and @k0rti: credited reporters of the original pipeline-listing SQL injection submitted on December 13, 2023. - BugBunny.ai (@bugbunny-research): submitted the later report on April 1, 2026, documenting the run-listing and metric-query paths in GHSA-mhpc-2x8q-2vp9. This acknowledges that specific contribution without implying authorship of the 2023 report. - Cyber Nagle: authored the merged remediation in PR #13127.
GHSA-mhpc-2x8q-2vp9 is consolidated here for vulnerability tracking. Its submission and reporter credit remain preserved on that advisory; closure as a duplicate does not characterize the report as invalid.
Kubeflow Pipelines enables users to build and deploy portable, scalable machine learning workflows. From 2.17.0 until 2.18.0, the TensorBoard proxy token introduced by the incomplete remediation for CVE-2026-54745 uses MINIOSECRETKEY as its HMAC secret, whose bundled default is publicly known. In an authorization-disabled deployment using that default, an unauthenticated attacker can forge a token containing a chosen viewerName and namespace; the proxy constructs the destination from the viewerName plus the -service suffix, the namespace, and the configured cluster domain, then reads responses on port 80 under the /tensorboard/<viewer>/ path. Resource-name validation and path-traversal rejection remain enforced, authorization-enabled deployments still check access to the selected namespace, and unrestricted metadata access, writes, or service disruption are not established. This issue is fixed in version 2.18.0.
Kubeflow Pipelines enables users to build and deploy portable, scalable machine learning workflows. From 2.17.0 until 2.18.0, the TensorBoard proxy token introduced by the incomplete remediation for CVE-2026-54745 uses MINIOSECRETKEY as its HMAC secret, whose bundled default is publicly known. In an authorization-disabled deployment using that default, an unauthenticated attacker can forge a token containing a chosen viewerName and namespace; the proxy constructs the destination from the viewerName plus the -service suffix, the namespace, and the configured cluster domain, then reads responses on port 80 under the /tensorboard/<viewer>/ path. Resource-name validation and path-traversal rejection remain enforced, authorization-enabled deployments still check access to the selected namespace, and unrestricted metadata access, writes, or service disruption are not established. This issue is fixed in version 2.18.0.
Kubeflow Pipelines enables users to build and deploy portable, scalable machine learning workflows. Prior to 2.18.0, the frontend artifact download path for source=s3 or source=minio accepts providerInfo.Params.endpoint, and the archived pod-log path accepts S3-compatible endpoint metadata from workflow status, without requiring those endpoints to match operator-controlled storage configuration. In the shipped authorization-disabled standalone configuration, an unauthenticated caller can make the shared ml-pipeline-ui pod connect to arbitrary reachable hosts and return response data; requests may also reveal an access-key identifier when signed with configured object-store credentials. Multi-user secret-backed metadata remains restricted to the server namespace. This issue is fixed in version 2.18.0.
Kubeflow Pipelines enables users to build and deploy portable, scalable machine learning workflows. Prior to 2.18.0, the frontend artifact download path for source=s3 or source=minio accepts providerInfo.Params.endpoint, and the archived pod-log path accepts S3-compatible endpoint metadata from workflow status, without requiring those endpoints to match operator-controlled storage configuration. In the shipped authorization-disabled standalone configuration, an unauthenticated caller can make the shared ml-pipeline-ui pod connect to arbitrary reachable hosts and return response data; requests may also reveal an access-key identifier when signed with configured object-store credentials. Multi-user secret-backed metadata remains restricted to the server namespace. This issue is fixed in version 2.18.0.
A flaw was found in the Trainer v2 TrainJob API. A user with permissions to create TrainJobs can exploit this by injecting valueFrom.secretKeyRef into environment variable definitions. This allows them to read and exfiltrate sensitive information (secrets) from the namespace, leading to a privilege escalation. This occurs when the TrainJob Role-Based Access Control (RBAC) is broader than the Secret read RBAC.
A flaw was found in the training-operator. An incorrect object casting in the PyTorchJob ValidateUpdate webhook allows users to bypass an immutability check. This vulnerability could enable users to detach a queued job from its quota management system or hijack a running job's controller ownership. The primary impact is a Denial of Service (DoS) condition.
Kubeflow Pipelines enables users to build and deploy portable, scalable machine learning workflows. From 0.5.1 until 2.17.0, the frontend artifact routes did not check whether an authenticated user was authorized to access the requested namespace. A user who can reach the frontend and knows an artifact location can select another tenant's namespace, or omit the namespace and reach an unguarded direct handler, to read artifact contents such as datasets, models, or pipeline outputs. The issue applies to multi-user deployments that rely on Kubeflow Pipelines authorization for tenant isolation. Authentication at ingress identifies the caller but does not authorize access to the selected artifact namespace. The issue does not establish arbitrary Kubernetes Secret disclosure, storage modification, or server-side code execution. This issue is fixed in version 2.17.0.
Kubeflow Pipelines enables users to build and deploy portable, scalable machine learning workflows. From 0.5.1 until 2.17.0, the frontend artifact routes did not check whether an authenticated user was authorized to access the requested namespace. A user who can reach the frontend and knows an artifact location can select another tenant's namespace, or omit the namespace and reach an unguarded direct handler, to read artifact contents such as datasets, models, or pipeline outputs. The issue applies to multi-user deployments that rely on Kubeflow Pipelines authorization for tenant isolation. Authentication at ingress identifies the caller but does not authorize access to the selected artifact namespace. The issue does not establish arbitrary Kubernetes Secret disclosure, storage modification, or server-side code execution. This issue is fixed in version 2.17.0.
Kubeflow Model Registry through 0.3.17 contains a server-side request forgery vulnerability that allows namespace users to steal the controller ServiceAccount token via the modelregistry.kubeflow.org/url annotation. Attackers can create an InferenceService pointing the annotation at their server, capture the bearer token, and modify other tenants' InferenceServices cluster-wide.
kubeflow/kubeflow is vulnerable to a Regular Expression Denial of Service (ReDoS) attack due to inefficient regular expression complexity in its email validation mechanism. An attacker can remotely exploit this vulnerability without authentication by providing specially crafted input that causes the application to consume an excessive amount of CPU resources. This vulnerability affects the latest version of kubeflow/kubeflow, specifically within the centraldashboard-angular backend component. The impact of exploiting this vulnerability includes resource exhaustion, and service disruption.
The AWS ALB Route Directive Adapter For Istio repo https://github.com/awslabs/aws-alb-route-directive-adapter-for-istio/tree/master provides an OIDC authentication mechanism that was integrated into the open source Kubeflow project. The adapter uses JWT for authentication, but lacks proper signer and issuer validation. In deployments of ALB that ignore security best practices, where ALB targets are directly exposed to internet traffic, an actor can provide a JWT signed by an untrusted entity in order to spoof OIDC-federated sessions and successfully bypass authentication.
The repository/package has been deprecated, is end of life, and is no longer supported. As a security best practice, ensure that your ELB targets (e.g. EC2 Instances, Fargate Tasks etc.) do not have public IP addresses. Ensure any forked or derivative code validate that the signer attribute in the JWT match the ARN of the Application Load Balancer that the service is configured to use.
There exists a stored XSS Vulnerability in Kubeflow Pipeline View web UI. The Kubeflow Web UI allows to create new pipelines. When creating a new pipeline, it is possible to add a description. The description field allows html tags, which are not filtered properly. Leading to a stored XSS. We recommend upgrading past commit 930c35f1c543998e60e8d648ce93185c9b5dbe8d
Cross-site Scripting (XSS) - Reflected in kubeflow/kubeflow
Server-Side Request Forgery (SSRF) in kubeflow/kubeflow