Specifically, an ASN.1 NumericString in the certificate CN will fail UTF-8 conversion
and result in a negative return value, which the "x509 -checkhost" command-line option
incorrectly interpreted as success.
Also update X509_check_host docs to reflect reality.
Thanks to Sean Burford (Google) for reporting this issue.
Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 0923e7df9e)
Reduces number of silly casts in OpenSSL code and likely most
applications. Consistent with (char *) for "peername" value from
X509_check_host() and X509_VERIFY_PARAM_get0_peername().
(cherry picked from commit 297c67fcd8)
A client reference identity of ".example.com" matches a server
certificate presented identity that is any sub-domain of "example.com"
(e.g. "www.sub.example.com).
With the X509_CHECK_FLAG_SINGLE_LABEL_SUBDOMAINS flag, it matches
only direct child sub-domains (e.g. "www.sub.example.com").
(cherry picked from commit e52c52f10bb8e34aaf8f28f3e5b56939e8f6b357)
Fixes to host checking wild card support and add support for
setting host checking flags when verifying a certificate
chain.
(cherry picked from commit 397a8e747d)
