The problem occurs in EVP_PKEY_sign() when using RSA with X931 padding.
It is only triggered if the RSA key size is smaller than the digest length.
So with SHA512 you can trigger the overflow with anything less than an RSA
512 bit key. I managed to trigger a 62 byte overflow when using a 16 bit RSA
key. This wasn't sufficient to cause a crash, although your mileage may
vary.
In practice RSA keys of this length are never used and X931 padding is very
rare. Even if someone did use an excessively short RSA key, the chances of
them combining that with a longer digest and X931 padding is very
small. For these reasons I do not believe there is a security implication to
this. Thanks to Kevin Wojtysiak (Int3 Solutions) and Paramjot Oberoi (Int3
Solutions) for reporting this issue.
Reviewed-by: Andy Polyakov <appro@openssl.org>
(cherry picked from commit 34166d4189)
Extend RSA ASN1 method to support CMS PSS signatures for both sign
and verify.
For signing the EVP_PKEY_CTX parameters are read and the appropriate
CMS structures set up.
For verification the CMS structures are analysed and the corresponding
parameters in the EVP_PKEY_CTX set.
Also add RSA-OAEP support.
For encrypt the EVP_PKEY_CTX parameters are used.
For decrypt the CMS structure is uses to set the appropriate EVP_PKEY_CTX
parameters.
(cherry picked from commit 0574cadf85)
Also sync error codes with OpenSSL 1.0.1 and add new ones.
Add OAEP ctrls to retrieve MD and label. Return errors if
an attempt is made to set or retrieve OAEP parameters when
padding mode is not OAEP.
(cherry picked from commit 211a14f627)
Extend OAEP support. Generalise the OAEP padding functions to support
arbitrary digests. Extend EVP_PKEY RSA method to handle the new OAEP
padding functions and add ctrls to set the additional parameters.
(cherry picked from commit 271fef0ef3)
Conflicts:
CHANGES
When in FIPS mode the approved implementations are used as normal,
when not in FIPS mode the internal unapproved versions are used instead.
This means that the FIPS capable OpenSSL isn't forced to use the
(often lower perfomance) FIPS implementations outside FIPS mode.
of handshake failure
2. Changes to x509_certificate_type function (crypto/x509/x509type.c) to
make it recognize GOST certificates as EVP_PKT_SIGN|EVP_PKT_EXCH
(required for s3_srvr to accept GOST client certificates).
3. Changes to EVP
- adding of function EVP_PKEY_CTX_get0_peerkey
- Make function EVP_PKEY_derive_set_peerkey work for context with
ENCRYPT operation, because we use peerkey field in the context to
pass non-ephemeral secret key to GOST encrypt operation.
- added EVP_PKEY_CTRL_SET_IV control command. It is really
GOST-specific, but it is used in SSL code, so it has to go
in some header file, available during libssl compilation
4. Fix to HMAC to avoid call of OPENSSL_cleanse on undefined data
5. Include des.h if KSSL_DEBUG is defined into some libssl files, to
make debugging output which depends on constants defined there, work
and other KSSL_DEBUG output fixes
6. Declaration of real GOST ciphersuites, two authentication methods
SSL_aGOST94 and SSL_aGOST2001 and one key exchange method SSL_kGOST
7. Implementation of these methods.
8. Support for sending unsolicited serverhello extension if GOST
ciphersuite is selected. It is require for interoperability with
CryptoPro CSP 3.0 and 3.6 and controlled by
SSL_OP_CRYPTOPRO_TLSEXT_BUG constant.
This constant is added to SSL_OP_ALL, because it does nothing, if
non-GOST ciphersuite is selected, and all implementation of GOST
include compatibility with CryptoPro.
9. Support for CertificateVerify message without length field. It is
another CryptoPro bug, but support is made unconditional, because it
does no harm for draft-conforming implementation.
10. In tls1_mac extra copy of stream mac context is no more done.
When I've written currently commited code I haven't read
EVP_DigestSignFinal manual carefully enough and haven't noticed that
it does an internal digest ctx copying.
This implementation was tested against
1. CryptoPro CSP 3.6 client and server
2. Cryptopro CSP 3.0 server
value of the passed output buffer is NULL.
The old method of using EVP_PKEY_size(pkey) isn't flexible enough to cover all
cases where the output length may depend on the operation or the parameters
associated with it.
