Fix incorrect return code on ECDSA key verification

ECDSA_do_verify() is a function that verifies a ECDSA signature given a hash and a public EC key. The function is supposed to return 1 on valid signature, 0 on invalid signature and -1 on error. Previously, we returned 0 if the key did not have a verify_sig method. This is actually an error case and not an invalid signature. Consequently, this patch updates the return code to -1.

Fixes #8766

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/10693)

(cherry picked from commit 26583f6aa8dc28e3598e61db66e54e2fdf8b195f)
This commit is contained in:
Andrew Hoang 2019-12-23 20:19:24 -08:00 committed by Nicola Tuveri
parent 49847f3c40
commit 0fcc6e70bc

View file

@ -23,7 +23,7 @@ int ECDSA_do_verify(const unsigned char *dgst, int dgst_len,
if (eckey->meth->verify_sig != NULL)
return eckey->meth->verify_sig(dgst, dgst_len, sig, eckey);
ECerr(EC_F_ECDSA_DO_VERIFY, EC_R_OPERATION_NOT_SUPPORTED);
return 0;
return -1;
}
/*-
@ -39,5 +39,5 @@ int ECDSA_verify(int type, const unsigned char *dgst, int dgst_len,
return eckey->meth->verify(type, dgst, dgst_len, sigbuf, sig_len,
eckey);
ECerr(EC_F_ECDSA_VERIFY, EC_R_OPERATION_NOT_SUPPORTED);
return 0;
return -1;
}