GHSA-r8x3-637f-c986: Memory leak in the Mbed TLS crypto shim on failed cipher/AEAD operations leads to remotely triggered crypto denial of service
The Mbed TLS crypto shim driver in drivers/crypto/cryptombedtlsshim.c drives PSA multipart cipher and AEAD operations. In mbedtlscbc() and mbedtlsaead(), every error path after a successful psacipherencryptsetup() / psaaeaddecryptsetup() returned directly without calling psacipherabort() / psaaeadabort(). Because the abort is the only thing that reaches mbedtlscipherfree(), each failed operation permanently leaked the heap-allocated mbedtlsaescontext (roughly 288 bytes) that setup had allocated from Mbed TLS's fixed static heap (mbedtlsheap[CONFIGMBEDTLSHEAPSIZE], default 512 bytes). With CONFIGPSACRYPTOPROVIDERTFM the leaked object is instead a finite secure-partition multipart-operation slot.
The failure is chosen by an unauthenticated attacker. With CONFIGNETL2IEEE802154SECURITY backed by this driver, ieee802154decryptauth() in subsys/net/l2/ieee802154/ieee802154security.c invokes cipherccmop() on every received secured frame, with no authentication ahead of the MIC check. A device in radio range can transmit a stream of syntactically valid 802.15.4 frames carrying a bogus MIC; each one makes psaaeadverify() fail and leaks a block. The CBC path is reachable the same way through the OSDP secure channel, where CONFIGOSDPSCENABLED selects this shim and osdpencrypt() / osdpdecrypt() leave outbufmax unset, so every operation fails inside psacipherupdate().
Exhausting the Mbed TLS heap — which the default configuration reaches after a couple of failed operations, and any realistic configuration reaches in well under a second of frames — makes all subsequent mbedtlscalloc() calls fail. Every later cryptographic operation in the image then fails permanently, including link-layer security and any TLS/DTLS sharing the same heap, until the device is rebooted. There is no memory corruption or information disclosure: frames with an invalid MIC are still rejected, so the impact is limited to availability.
The fix routes all post-setup failures in mbedtlscbc() and mbedtlsaead() through a common error: label that calls the matching PSA abort before returning. Setup failures need no change, since the PSA core aborts the operation itself when setup fails.