F5-K000138242: Medium severity F5 BIG-IP Next vulnerability

Issue summary: Generating excessively long X9.42 DH keys or checking excessively long X9.42 DH keys or parameters may be very slow. Impact summary: Applications that use the functions DHgeneratekey() to generate an X9.42 DH key may experience long delays. Likewise, applications that use DHcheckpubkey(), DHcheckpubkeyex() or EVPPKEYpubliccheck() to check an X9.42 DH key or X9.42 DH parameters may experience long delays. Where the key or parameters that are being checked have been obtained from an untrusted source this may lead to a Denial of Service. While DHcheck() performs all the necessary checks (as of CVE-2023-3817), DHcheckpubkey() doesn't make any of these checks, and is therefore vulnerable for excessively large P and Q parameters. Likewise, while DHgeneratekey() performs a check for an excessively large P, it doesn't check for an excessively large Q. An application that calls DHgeneratekey() or DHcheckpubkey() and supplies a key or parameters obtained from an untrusted source could be vulnerable to a Denial of Service attack. DHgeneratekey() and DHcheckpubkey() are also called by a number of other OpenSSL functions. An application calling any of those other functions may similarly be affected. The other functions affected by this are DHcheckpubkeyex(), EVPPKEYpubliccheck(), and EVPPKEYgenerate(). Also vulnerable are the OpenSSL pkey command line application when using the "-pubcheck" option, as well as the OpenSSL genpkey command line application. The OpenSSL SSL/TLS implementation is not affected by this issue. The OpenSSL 3.0 and 3.1 FIPS providers are not affected by this issue.