CVE-2020-5234: Untrusted data can lead to DoS attack in MessagePack for C# and Unity

Impact

When this library is used to deserialize messagepack data from an untrusted source, there is a risk of a denial of service attack by either of two vectors:

1. hash collisions - leading to large CPU consumption disproportionate to the size of the data being deserialized. 1. stack overflow - leading to the deserializing process crashing.

Patches

The following steps are required to mitigate this risk.

1. Upgrade to a version of the library where a fix is available 1. Add code to your application to put MessagePack into the defensive UntrustedData mode. 1. Identify all MessagePack extensions that implement IMessagePackFormatter<T> implementations that do not ship with the MessagePack library to include the security mitigations. This includes those acquired from 3rd party packages and classes included directly into your project. Any AOT formatters generated with the MPC tool must be regenerated with the patched version of mpc. 1. Review your messagepack-serializable data structures for hash-based collections that use custom or unusual types for the hashed key. See below for details on handling such situations.

Review the MessagePackSecurity class to tweak any settings as necessary to strike the right balance between performance, functionality, and security.

Specialized IEqualityComparer<T> implementations provide the hash collision resistance. Each type of hashed key may require a specialized implementation of its own. The patched MessagePack library includes many such implementations for primitive types commonly used as keys in hash-based collections. If your data structures use custom types as keys in these hash-based collections, putting MessagePack in UntrustedData mode may lead the deserializer to throw an exception because no safe IEqualityComparer<T> is available for your custom T type. You can provide your own safe implementation by deriving from the MessagePackSecurity class and overriding the GetHashCollisionResistantEqualityComparer<T>() method to return your own custom implementation when T matches your type, and fallback to return base.GetHashCollisionResistantEqualityComparer<T>(); for types you do not have custom implementations for.

Unrelated to this advisory, but as general security guidance, you should also avoid the Typeless serializer/formatters/resolvers for untrusted data as that opens the door for the untrusted data to potentially deserialize unanticipated types that can compromise security.

MessagePack 1.x users

1. Upgrade to any 1.9.x version.

1. When deserializing untrusted data, put MessagePack into a more secure mode with:

cs MessagePackSecurity.Active = MessagePackSecurity.UntrustedData;

In MessagePack v1.x this is a static property and thus the security level is shared by the entire process or AppDomain. Use MessagePack v2.1 or later for better control over the security level for your particular use.

1. Any code produced by mpc should be regenerated with the mpc tool with the matching (patched) version. Such generated code usually is written to a file called Generated.cs. A patched Generated.cs file will typically reference the MessagePackSecurity class.

Review any custom-written IMessagePackFormatter<T> implementations in your project or that you might use from 3rd party packages to ensure they also utilize the MessagePackSecurity class as required. In particular, a formatter that deserializes an object (as opposed to a primitive value) should wrap the deserialization in a using (MessagePackSecurity.DepthStep()) block. For example:

cs public MyObject Deserialize(ref MessagePackReader reader, MessagePackSerializerOptions options) { if (reader.TryReadNil()) { return default; } else { using (MessagePackSecurity.DepthStep()) // STACK OVERFLOW MITIGATION { MyObject o = new MyObject(); // deserialize members of the object here. return o; } } }

If your custom formatter creates hash-based collections (e.g. Dictionary<K, V> or HashSet<T>) where the hashed key comes from the messagepack data, always instantiate your collection using MessagePackSecurity.Active.GetEqualityComparer<T>() as the equality comparer:

cs var collection = new HashSet<T>(MessagePackSecurity.Active.GetEqualityComparer<T>());

This ensures that when reading untrusted data, you will be using a collision-resistent hash algorithm.

Learn more about best security practices when reading untrusted data with MessagePack 1.x.

MessagePack 2.x users

1. Upgrade to any 2.1.x or later version.

1. When deserializing untrusted data, put MessagePack into a more secure mode by configuring your MessagePackSerializerOptions.Security property:

cs var options = MessagePackSerializerOptions.Standard .WithSecurity(MessagePackSecurity.UntrustedData);

// Pass the options explicitly for the greatest control. T object = MessagePackSerializer.Deserialize<T>(data, options);

// Or set the security level as the default. MessagePackSerializer.DefaultOptions = options;

1. Any code produced by mpc should be regenerated with the mpc tool with the matching (patched) version. Such generated code usually is written to a file called Generated.cs. A patched Generated.cs file will typically reference the Security member on the MessagePackSerializerOptions parameter.

Review any custom-written IMessagePackFormatter<T> implementations in your project or that you might use from 3rd party packages to ensure they also utilize the MessagePackSecurity class as required. In particular, a formatter that deserializes an object (as opposed to a primitive value) should call options.Security.DepthStep(ref reader); before deserializing the object's members, and be sure to revert the depth step with reader.Depth--; before exiting the method. For example:

cs public MyObject Deserialize(ref MessagePackReader reader, MessagePackSerializerOptions options) { if (reader.TryReadNil()) { return default; } else { options.Security.DepthStep(ref reader); // STACK OVERFLOW MITIGATION, line 1 try { MyObject o = new MyObject(); // deserialize members of the object here. return o; } finally { reader.Depth--; // STACK OVERFLOW MITIGATION, line 2 } } }

If your custom formatter creates hash-based collections (e.g. Dictionary<K, V> or HashSet<T>) where the hashed key comes from the messagepack data, always instantiate your collection using options.Security.GetEqualityComparer<TKey>() as the equality comparer:

cs var collection = new HashSet<T>(options.Security.GetEqualityComparer<T>());

This ensures that when reading untrusted data, you will be using a collision-resistent hash algorithm.

Learn more about best security practices when reading untrusted data with MessagePack 2.x.

Workarounds

The security vulnerabilities are in the formatters. Avoiding the built-in formatters entirely in favor of reading messagepack primitive data directly or relying on carefully written custom formatters can provide a workaround.

MessagePack v1.x users may utilize the MessagePackBinary static class directly to read the data they expect. MessagePack v2.x users may utilize the MessagePackReader struct directly to read the data they expect.

References

Learn more about best security practices when reading untrusted data with MessagePack 1.x or MessagePack 2.x.

For more information

If you have any questions or comments about this advisory:

Open an issue in MessagePack-CSharp Email us