Horovod versions up to and including v0.28.1 are vulnerable to unauthenticated remote code execution. The vulnerability is due to improper handling of base64-encoded data in the ElasticRendezvousHandler, a subclass of KVStoreHandler. Specifically, the putvalue method in ElasticRendezvousHandler calls codec.loadsbase64(value), which eventually invokes cloudpickle.loads(decoded). This allows an attacker to send a malicious pickle object via a PUT request, leading to arbitrary code execution on the server.

Impact The insecure tempfile.mktemp() is used when Horovod is run in an LSF job with jsrun. In that situation, a jsrun rank file is created with mktemp, which could be hijacked by another process to read or manipulate the content.

This issue does not impact the use of MPI, Gloo, Spark or Ray.

Patches The problem has been fixed in b96ecae4.

Workarounds The rank file is not created when bindingargs are provided in the Settings instance.

References Please see https://github.com/horovod/horovod/pull/3358 for details.

For more information If you have any questions or comments about this advisory: Open an issue in https://github.com/horovod/horovod