A flaw was found in ml-metadata. The statically-linked gRPC stack in ml-metadata is outdated, making it vulnerable to known HTTP/2 denial of service (DoS) issues. An in-cluster attacker, with network access to the MLMD pod, could exploit these vulnerabilities by sending specially crafted HTTP/2 requests. This could lead to a denial of service by crashing the MLMD pod, disrupting all pipeline runs in the affected namespace.

Finding The statically-linked gRPC stack is pinned to v1.46.3 (2022) in the Bazel WORKSPACE file, predating multiple HTTP/2 DoS CVEs that are directly reachable on the network listener:

WORKSPACE:116-117 comgithubgrpcgrpc → gRPC 1.46.3 WORKSPACE:129-130 comgoogleprotobuf → protobuf 3.21.12 WORKSPACE:173-176 zlib → 1.3 The MLMD server is a network-facing gRPC listener (FIND-001: no auth), so gRPC-layer DoS CVEs are directly reachable from any pod that can open a TCP connection to :8080.

The Bazel WORKSPACE pins are content-addressed (sha256), so the issue is staleness, not mutability. Renovate is present (.github/renovate.json) but evidently not covering Bazel httparchive entries.

File: WORKSPACE:116-117,129-130,173-176 Repository: red-hat-data-services/ml-metadata Framework: ASVS V14.2.1; OWASP K8s K07 Vulnerable Components; OpenSSF Scorecard Vulnerabilities CWE: CWE-1395 / CWE-1104 CVSS v3.1: 7.5 (High) AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H (inherits gRPC HTTP/2 DoS vectors)

RHOAI Mitigation The DSPO-deployed NetworkPolicy restricts who can reach MLMD port 8080 to only KFP v2 driver pods and DSP components. This limits the attack surface but does not eliminate it — a compromise of a KFP driver pod or DSP component could exploit these CVEs to crash the MLMD pod and disrupt all pipeline runs in the namespace.

Impact An in-cluster attacker who can reach the MLMD pod (within the NetworkPolicy allowlist) can crash or resource-exhaust the MLMD pod via known gRPC/HTTP2 frame-handling bugs, disrupting all pipeline runs in the namespace.

Context ml-metadata is planned for removal from the product (several months out). The stale dependency risk remains active until removal is complete.

Remediation Bump WORKSPACE pins: gRPC >= 1.62, protobuf >= 25.x, zlib >= 1.3.1. Extend Renovate configuration to cover Bazel httparchive entries.

Denial of Service through Data corruption in gRPC-C++

HPACK table poisoning in gRPC C++, Python & Ruby

HTTP/2 Rapid reset attack The HTTP/2 protocol allows clients to indicate to the server that a previous stream should be canceled by sending a RSTSTREAM frame. The protocol does not require the client and server to coordinate the cancellation in any way, the client may do it unilaterally. The client may also assume that the cancellation will take effect immediately when the server receives the RSTSTREAM frame, before any other data from that TCP connection is processed.

Abuse of this feature is called a Rapid Reset attack because it relies on the ability for an endpoint to send a RSTSTREAM frame immediately after sending a request frame, which makes the other endpoint start working and then rapidly resets the request. The request is canceled, but leaves the HTTP/2 connection open.

The HTTP/2 Rapid Reset attack built on this capability is simple: The client opens a large number of streams at once as in the standard HTTP/2 attack, but rather than waiting for a response to each request stream from the server or proxy, the client cancels each request immediately.

The ability to reset streams immediately allows each connection to have an indefinite number of requests in flight. By explicitly canceling the requests, the attacker never exceeds the limit on the number of concurrent open streams. The number of in-flight requests is no longer dependent on the round-trip time (RTT), but only on the available network bandwidth.

In a typical HTTP/2 server implementation, the server will still have to do significant amounts of work for canceled requests, such as allocating new stream data structures, parsing the query and doing header decompression, and mapping the URL to a resource. For reverse proxy implementations, the request may be proxied to the backend server before the RSTSTREAM frame is processed. The client on the other hand paid almost no costs for sending the requests. This creates an exploitable cost asymmetry between the server and the client.

Multiple software artifacts implementing HTTP/2 are affected. This advisory was originally ingested from the swift-nio-http2 repo advisory and their original conent follows.

swift-nio-http2 specific advisory swift-nio-http2 is vulnerable to a denial-of-service vulnerability in which a malicious client can create and then reset a large number of HTTP/2 streams in a short period of time. This causes swift-nio-http2 to commit to a large amount of expensive work which it then throws away, including creating entirely new Channels to serve the traffic. This can easily overwhelm an EventLoop and prevent it from making forward progress.

swift-nio-http2 1.28 contains a remediation for this issue that applies reset counter using a sliding window. This constrains the number of stream resets that may occur in a given window of time. Clients violating this limit will have their connections torn down. This allows clients to continue to cancel streams for legitimate reasons, while constraining malicious actors.

Denial of Service in gRPC Core

Denial-of-Service in gRPC

gRPC could allow a remote attacker to obtain sensitive information, caused by a flaw when gRPC HTTP2 stack raised a header size exceeded error. By sending a specially crafted request, an attacker could exploit this vulnerability to obtain sensitive information, and use this information to launch further attacks against the affected system.

Denial-of-Service in gRPC

gRPC is vulnerable to a denial of service. By sending a specially crafted header, an attacker could exploit this vulnerability to cause a denial of service.

Google gRPC before 2017-04-05 has an out-of-bounds write caused by a heap-based buffer overflow related to core/lib/iomgr/error.c.

Google gRPC before 2017-02-22 has an out-of-bounds write related to the gprfree function in core/lib/support/alloc.c.

Google gRPC before 2017-02-22 has an out-of-bounds write caused by a heap-based buffer overflow related to the parseunix function in core/ext/clientchannel/parseaddress.c.