felem_neg does not produce an output within the tight bounds suitable
for felem_contract. This affects build configurations which set
enable-ec_nistp_64_gcc_128.
point_double and point_add, in the non-z*_is_zero cases, tolerate and
fix up the wider bounds, so this only affects point_add calls where the
other point is infinity. Thus it only affects the final addition in
arbitrary-point multiplication, giving the wrong y-coordinate. This is a
no-op for ECDH and ECDSA, which only use the x-coordinate of
arbitrary-point operations.
Note: ecp_nistp521.c has the same issue in that the documented
preconditions are violated by the test case. I have not addressed this
in this PR. ecp_nistp521.c does not immediately produce the wrong
answer; felem_contract there appears to be a bit more tolerant than its
documented preconditions. However, I haven't checked the point_add
property above holds. ecp_nistp521.c should either get this same fix, to
be conservative, or have the bounds analysis and comments reworked for
the wider bounds.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5779)
dc55e4f70f