CVE-2026-64009: xfrm: Check for underflow in xfrm_state_mtu

In the Linux kernel, the following vulnerability has been resolved:

xfrm: Check for underflow in xfrmstatemtu

Leo Lin reported OOB write issue in esp component:

xfrmstatemtu() returns u32 but performs its arithmetic in unsigned modulo-2^32 space using an attacker-influenced "headerlen + authsize + netadj" subtracted from a small "mtu" argument. A nobody user can install an IPv4 ESP tunnel SA with a large authentication key (XFRMAALGAUTHTRUNC, e.g. hmac(sha512), 64-byte key, 64-byte trunc), configure a small interface MTU (68 bytes), and set XFRMATFCPAD to a large value. When a single UDP datagram is then sent through the tunnel, xfrmstatemtu() underflows to a near-2^32 value, and espoutput() consumes it as a signed int via:

padto = min(x->tfcpad, xfrmstatemtu(x, mtucached)) esp.tfclen = padto - skb->len (assigned to int)

esp.tfclen ends up negative (e.g. -207). It is sign-extended to sizet when passed to memset() inside espoutputfilltrailer(), producing a ~16 EB write of zeroes at skbtailpointer(skb). KASAN logs it as "Write of size 18446744073709551537 at addr ffff888...".

Check for underflow and return 1. This causes the sendmsg attempt to fail with ENETUNREACH.