See how kcp compares to other vendors in security performance
Summary
The kcp front-proxy fails to strip client-supplied identity headers before forwarding requests to shards. Any authenticated tenant can inject their own X-Remote-Group and X-Remote-Extra- headers, which the shard trusts as a verified identity assertion — allowing a low-privilege user to escalate to cluster administrator (system:masters) and read, write, or delete resources in any workspace on the shard. This is a complete multi-tenant isolation and authorization bypass.
Impact
In a sharded kcp deployment, external clients reach shards through the front-proxy, which authenticates the client and then forwards the resulting identity to the shard using Kubernetes request-header authentication (X-Remote-User / X-Remote-Group / X-Remote-Extra-). The shard trusts these headers because they arrive over the front-proxy's mutually-authenticated connection.
Because the front-proxy appended its identity headers instead of replacing them — and never removed any copies the client sent — an authenticated attacker could smuggle forged identity headers through to the shard. With this, an attacker holding any ordinary credential (client certificate, OIDC token, or service-account token) and no special privileges could:
- assert X-Remote-Group: system:masters and act as cluster super-user, bypassing the entire kcp authorizer chain in every workspace on the shard; - forge authorization.kcp.io/warrant to assume an arbitrary user/group identity via kcp's delegated-identity mechanism; - forge authentication.kcp.io/scopes to escape the cluster-scoping that confines service-account and impersonated identities to their origin workspace; - satisfy per-workspace required-group gating by injecting the required group.
The result is arbitrary read/write/delete access to any tenant's resources, secrets, RBAC, APIExports/APIBindings, and LogicalClusters — a cross-workspace access break and authorizer bypass across the proxy's trust boundary.
Patches Fixed in v0.31.4, 0.32.2. The front-proxy and the shard's in-process local-proxy now unconditionally remove any inbound X-Remote- identity headers before stamping the authenticated identity, so no client-supplied value can be forwarded to a shard.
Operators should upgrade to a patched release. No configuration changes are required after upgrading.
Workarounds
There is no complete workaround other than upgrading. Deployments that terminate client connections at an external proxy capable of stripping X-Remote-User, X-Remote-Group, and all X-Remote-Extra- headers from inbound requests before they reach the kcp front-proxy can mitigate exposure in the interim.
Credit to 5ud0er / Tarmo Technologies.
Impact
The APIExport Virtual Workspace can be used to manage objects in workspaces that bind that APIExport for resources defined in the APIExport or specified and accepted via permission claims. This allows an API provider (via their APIExport) scoped down access to workspaces of API consumers to provide their services properly.
The identified vulnerability allows creating or deleting an object via the APIExport VirtualWorkspace in any arbitrary target workspace for pre-existing resources. By design, this should only be allowed when the workspace owner decides to give access to an API provider by creating an APIBinding.
With this vulnerability, it is possible for an attacker to create and delete objects even if none of these requirements are satisfied, i.e. even if there is no APIBinding in that workspace at all or the workspace owner has created an APIBinding, but rejected a permission claim.
Patches
A fix for this issue has been identified and has been published with kcp 0.26.3 and 0.27.0.
Workarounds
For users unable to upgrade to one of the patched versions, the following guidance can be given:
- Minimise the set of people with apiexport/content sub-resource access to APIExport resources. Be aware that this has to apply to all workspaces to be effective. - Filter incoming requests in a reverse proxy with a similar logic as the authorizer added in the referenced pull request.
References
See pull request (https://github.com/kcp-dev/kcp/pull/3338).
Summary
The cache server is directly exposed by the root shard and has no authentication or authorization in place. This allows anyone who can access the root shard to read and write to the cache server.
Details
The cache server is routed in the pre-mux chain in the shard code. The preHandlerChainMux is handled before any authn/authz in the cache server: https://github.com/kcp-dev/kcp/blob/aaf93d59cbcd0cefb70d94bd8959ce390547c4a2/pkg/server/config.go#L514-L518
This results in the cache server being proxied before any authn/authz in the handler chain takes place.
Attack Vectors
1. Unauthenticated Read Access (Primary) An attacker can read all replicated resources from the cache without any credentials. This exposes:
| Category | Resources | Severity | Reason | |---|---|---|---| | RBAC | clusterroles, clusterrolebindings (filtered by annotation) | High | Only subset with internal.kcp.io/replicate annotation: access rules, APIExport bind/content rules, WorkspaceType use rules. Reveals permission structure for API access and tenancy. Roles/RoleBindings NOT replicated. | | Infrastructure | logicalclusters, shards | High | Reveals full cluster topology and shard configuration | | API surface | apiexports, apiexportendpointslices, apiresourceschemas | High | Reveals all exported APIs and their network endpoints | | Admission control | mutatingwebhookconfigurations, validatingwebhookconfigurations, validatingadmissionpolicies | High | Reveals admission policies, aids bypass | | Tenancy | workspacetypes | Medium | Reveals workspace structure | | Cache metadata | cachedobjects, cachedresources, cachedresourceendpointslices | Medium | Exposes cache state and resource endpoints |
2. Write Access with Race Condition (Secondary) The cache server allows full CRUD operations. While injected objects are cleaned up by the replication controller, a race condition exists that could allow temporary privilege escalation.
The race window:
1. Attacker POSTs a malicious ClusterRole + ClusterRoleBinding to the cache server 2. Cache etcd watch fires and notifies two consumers in parallel: 2.1. The authorization informer (CacheKubeSharedInformerFactory) updates its in-memory store — the GlobalAuthorizer and WorkspaceContentAuthorizer now see the injected RBAC rules 2.2. The replication controller's informer enqueues a reconcile to its workqueue 3. Replication controller worker dequeues, calls getLocalCopy() → not found, deletes the object
Between steps 2 and 3, any API request hitting the GlobalAuthorizer (globalauthorizer.go:89-101) would evaluate RBAC against a store that includes the attacker's injected rules. The authorization informer and the replication controller share the same CacheKubeSharedInformerFactory (config.go:361), so the object is visible to authorization as soon as the informer cache updates — before the replication controller can process and delete it.
Practical exploitability is low — the window is sub-second, requiring the attacker to fire the privileged API request with precise timing. However, it could be automated in a tight loop. The workqueue rate limiter could also widen the window under load.
Self-healing mechanism: The replication controller acts as a self-healing mechanism. Objects injected into the cache are deleted almost instantly because:
Creating an object in cache triggers the cache informer Replication controller reconciles, calls getLocalCopy() → not found Controller calls deleteObject() on the cache copy (replicationreconcile.go:157-168)
Replicatable
Start a kcp root shard and query the cache server, e.g. with:
sh curl --insecure 'https://root.vespucci.genericcontrolplane.io:6443/services/cache/shards/root/clusters/root/apis/apis.kcp.io/v1alpha1'
Workarounds
Network-level access control: Restrict access to /services/cache/ paths at the load balancer, reverse proxy, or firewall level. External cache server: Deploy the cache server separately with its own kubeconfig (--cache-server-kubeconfig) and restrict network access to it.
Impact
Who is affected: Any kcp deployment where the root shard is network-reachable by untrusted clients. This applies when:
- Helm chart deployments: Affected if the shard's Service or Ingress exposes port 6443 externally. - Operator deployments: Affected if the Shard resource has spec.externalURL set (or spec.baseURL — externalURL defaults to baseURL if unset). When a shard has an external URL, clients route to it directly, exposing the /services/cache/ path. - Any deployment method: If the root shard's --shard-external-url is set and reachable from untrusted networks, the cache server is exposed.
Not affected: Deployments where the root shard is behind a front-proxy and is not directly reachable. The front-proxy does not forward /services/cache/ requests.
Write persistence: The replication controller watches the cache informer and acts as a self-healing mechanism. Objects injected into the cache are deleted almost instantly (sub-second) because:
- Creating an object in cache triggers the cache informer - Replication controller reconciles, calls getLocalCopy() → not found - Controller calls deleteObject() on the cache copy (replicationreconcile.go:157-168)