CVE-2026-42546: OP-TEE has missing OPTEE_MSG_ATTR_TYPE_MASK in cleanup_shm_refs() leaks mobj references

OP-TEE is a Trusted Execution Environment (TEE) designed as companion to a non-secure Linux kernel running on Arm; Cortex-A cores using the TrustZone technology. Starting in version 3.3.0 and prior to version 4.11.0, a resource leak exists in OP-TEE’s shared memory cleanup logic because the function cleanupshmrefs() in core/tee/entrystd.c fails to apply a required bitmask (OPTEEMSGATTRTYPEMASK) to parameter attributes. When processing non-contiguous memory parameters from a normal-world caller, the system fails to match the attribute type in its internal switch statement and skips the necessary mobjput() call. This results in a persistent reference leak of mobjregshm objects, which remain on internal lists with dangling refcounts. This affects non-FF-A configurations that support non-contiguous, non-secure shared memory. Over time, these accumulated leaks progressively consume the secure-world heap, degrading the system's ability to service trusted application operations and eventually requiring a reboot to recover. Version 4.11.0 contains a patch. No known workarounds are available.