Conceptually, this is a squashed version of:
Revert "Address feedback"
This reverts commit 75551e07bd.
and
Revert "Add CRYPTO_thread_glock_new"
This reverts commit ed6b2c7938.
But there were some intervening commits that made neither revert apply
cleanly, so instead do it all as one shot.
The crypto global locks were an attempt to cope with the awkward
POSIX semantics for pthread_atfork(); its documentation (the "RATIONALE"
section) indicates that the expected usage is to have the prefork handler
lock all "global" locks, and the parent and child handlers release those
locks, to ensure that forking happens with a consistent (lock) state.
However, the set of functions available in the child process is limited
to async-signal-safe functions, and pthread_mutex_unlock() is not on
the list of async-signal-safe functions! The only synchronization
primitives that are async-signal-safe are the semaphore primitives,
which are not really appropriate for general-purpose usage.
However, the state consistency problem that the global locks were
attempting to solve is not actually a serious problem, particularly for
OpenSSL. That is, we can consider four cases of forking application
that might use OpenSSL:
(1) Single-threaded, does not call into OpenSSL in the child (e.g.,
the child calls exec() immediately)
For this class of process, no locking is needed at all, since there is
only ever a single thread of execution and the only reentrancy is due to
signal handlers (which are themselves limited to async-signal-safe
operation and should not be doing much work at all).
(2) Single-threaded, calls into OpenSSL after fork()
The application must ensure that it does not fork() with an unexpected
lock held (that is, one that would get unlocked in the parent but
accidentally remain locked in the child and cause deadlock). Since
OpenSSL does not expose any of its internal locks to the application
and the application is single-threaded, the OpenSSL internal locks
will be unlocked for the fork(), and the state will be consistent.
(OpenSSL will need to reseed its PRNG in the child, but that is
an orthogonal issue.) If the application makes use of locks from
libcrypto, proper handling for those locks is the responsibility of
the application, as for any other locking primitive that is available
for application programming.
(3) Multi-threaded, does not call into OpenSSL after fork()
As for (1), the OpenSSL state is only relevant in the parent, so
no particular fork()-related handling is needed. The internal locks
are relevant, but there is no interaction with the child to consider.
(4) Multi-threaded, calls into OpenSSL after fork()
This is the case where the pthread_atfork() hooks to ensure that all
global locks are in a known state across fork() would come into play,
per the above discussion. However, these "calls into OpenSSL after
fork()" are still subject to the restriction to async-signal-safe
functions. Since OpenSSL uses all sorts of locking and libc functions
that are not on the list of safe functions (e.g., malloc()), this
case is not currently usable and is unlikely to ever be usable,
independently of the locking situation. So, there is no need to
go through contortions to attempt to support this case in the one small
area of locking interaction with fork().
In light of the above analysis (thanks @davidben and @achernya), go
back to the simpler implementation that does not need to distinguish
"library-global" locks or to have complicated atfork handling for locks.
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5089)
Code was added in commit b3c31a65 that overwrote the last ex_data value
using CRYPTO_dup_ex_data() causing a memory leak, and potentially
confusing the ex_data dup() callback.
In ssl_session_dup(), fix error handling (properly reference and up-ref
shared data) and new-up the ex_data before calling CRYPTO_dup_ex_data();
all other structures that dup ex_data have the destination ex_data new'd
before the dup.
Fix up some of the ex_data documentation.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3323)
Fix a strict aliasing issue in ui_dup_method_data.
Add test coverage for CRYPTO_dup_ex_data, use OPENSSL_assert.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2988)
Fixed a memory leak in ASN1_digest and ASN1_item_digest.
Reworked error handling in asn1_item_embed_new.
Fixed error handling in int_ctx_new and EVP_PKEY_CTX_dup.
Fixed a memory leak in CRYPTO_free_ex_data.
Reworked error handing in x509_name_ex_d2i, x509_name_encode and x509_name_canon.
Check for null pointer in tls_process_cert_verify.
Fixes#2103#2104#2105#2109#2111#2115
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2342)
Certain functions are automatically called during auto-deinit in order
to deallocate resources. However, if we have never entered a function which
marks lib crypto as inited then they never get called. This can happen if
the user only ever makes use of a small sub-set of functions that don't hit
the auto-init code.
This commit ensures all such resources deallocated by these functions also
init libcrypto when they are initially allocated.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Ben Laurie <ben@openssl.org>
Travis identified a problem with freeing the ex_data locks which wasn't
quite right in ff2344052. Trying to fix it identified a further problem:
the ex_data locks are cleaned up by OPENSSL_cleanup(), which is called
explicitly by CRYPTO_mem_leaks(), but then later the BIO passed to
CRYPTO_mem_leaks() is freed. An attempt is then made to use the ex_data
lock already freed.
Reviewed-by: Tim Hudson <tjh@openssl.org>
There is a preference for suffixes to indicate that a function is internal
rather than prefixes. Note: the suffix is only required to disambiguate
internal functions and public symbols with the same name (but different
case)
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
There was a lot of naming inconsistency, so we try and standardise on
one form.
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
CRYPTO_cleanup_all_ex_data() should not be called expicitly - we should
leave auto-deinit to clean this up instead.
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
ENGINE_cleanup calls CRYPTO_free_ex_data and therefore,
CRYPTO_cleanup_all_ex_data - which cleans up the method pointers - must
run after ENGINE_cleanup.
Additionally, don't needlessly initialize the EX_CALLBACKS stack during
e.g. CRYPTO_free_ex_data. The only time this is actually needed is when
reserving the first ex data index. Specifically, since sk_num returns -1
on NULL input, the rest of the code already handles a NULL method stack
correctly.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Applications wishing to include their own stacks now just need to include
DEFINE_STACK_OF(foo)
in a header file.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Add CRYPTO_free_ex_index (for shared libraries)
Unify and complete the documentation for all "ex_data" API's and objects.
Replace xxx_get_ex_new_index functions with a macro.
Added an exdata test.
Renamed the ex_data internal datatypes.
Reviewed-by: Matt Caswell <matt@openssl.org>
Removed ability to set ex_data impl at runtime. This removed these
three functions:
const CRYPTO_EX_DATA_IMPL *CRYPTO_get_ex_data_implementation(void);
int CRYPTO_set_ex_data_implementation(const CRYPTO_EX_DATA_IMPL *i);
int CRYPTO_ex_data_new_class(void);
It is no longer possible to change the ex_data implementation at
runtime. (Luckily those functions were never documented :)
Also removed the ability to add new exdata "classes." We don't believe
this received much (if any) use, since you can't add it to OpenSSL objects,
and there are probably better (native) methods for developers to add
their own extensible data, if they really need that.
Replaced the internal hash table (of per-"class" stacks) with a simple
indexed array. Reserved an index for "app" application.
Each API used to take the lock twice; now it only locks once.
Use local stack storage for function pointers, rather than malloc,
if possible (i.e., number of ex_data items is under a dozen).
Make CRYPTO_EX_DATA_FUNCS opaque/internal.
Also fixes RT3710; index zero is reserved.
Reviewed-by: Richard Levitte <levitte@openssl.org>
There are header files in crypto/ that are used by a number of crypto/
submodules. Move those to crypto/include/internal and adapt the
affected source code and Makefiles.
The header files that got moved are:
crypto/cryptolib.h
crypto/md32_common.h
Reviewed-by: Rich Salz <rsalz@openssl.org>
For a local variable:
TYPE *p;
Allocations like this are "risky":
p = OPENSSL_malloc(sizeof(TYPE));
if the type of p changes, and the malloc call isn't updated, you
could get memory corruption. Instead do this:
p = OPENSSL_malloc(sizeof(*p));
Also fixed a few memset() calls that I noticed while doing this.
Reviewed-by: Richard Levitte <levitte@openssl.org>
After the finale, the "real" final part. :) Do a recursive grep with
"-B1 -w [a-zA-Z0-9_]*_free" to see if any of the preceeding lines are
an "if NULL" check that can be removed.
Reviewed-by: Tim Hudson <tjh@openssl.org>
Free up bio_err after memory leak data has been printed to it.
In int_free_ex_data if ex_data is NULL there is nothing to free up
so return immediately and don't reallocate it.
Reviewed-by: Tim Hudson <tjh@openssl.org>
The mkstack.pl script now generates the entire safestack.h file.
It generates output that follows the coding style.
Also, removed all instances of the obsolete IMPLEMENT_STACK_OF
macro.
Reviewed-by: Andy Polyakov <appro@openssl.org>
choice but to have to cast away "const" qualifiers from their prototypes.
This does not remove constification restrictions from hash/compare
callbacks, but allows destructor commands to be run over a tables' elements
without bad casts.
Henson). Also, reverse a previous change that used an implicit function
pointer cast rather than an explicit data pointer cast in the STACK cleanup
code.
Currently, this change merely addresses where ex_data indexes are stored
and managed, and thus fixes the thread-safety issues that existed at that
level. "Class" code (eg. RSA, DSA, etc) no longer store their own STACKS
and per-class index counters - all such data is stored inside ex_data.c. So
rather than passing both STACK+counter to index-management ex_data
functions, a 'class_index' is instead passed to indicate the class (eg.
CRYPTO_EX_INDEX_RSA). New classes can be dynamically registered on-the-fly
and this is also thread-safe inside ex_data.c (though whether the caller
manages the return value in a thread-safe way is not addressed).
This does not change the "get/set" functions on individual "ex_data"
structures, and so thread-safety at that level isn't (yet) assured.
Likewise, the method of getting and storing per-class indexes has not
changed, so locking may still be required at the "caller" end but is
nonetheless thread-safe inside "ex_data"'s internal implementation.
Typically this occurs when code implements a new method of some kind and
stores its own per-class index in a global variable without locking the
setting and usage of that variable. If the code in question is likely to be
used in multiple threads, locking the setting and use of that index is
still up to the code in question. Possible fixes to this are being
sketched, but definitely require more major changes to the API itself than
this change undertakes.
The underlying implementation in ex_data.c has also been modularised so
that alternative "ex_data" implementations (that control all access to
state) can be plugged in. Eg. a loaded module can have its implementation
set to that of the application loaded it - the result being that
thread-safety and consistency of "ex_data" classes and indexes can be
maintained in the same place rather than the loaded module using its own
copy of ex_data support code and state.
Due to the centralisation of "state" with this change, cleanup of all
"ex_data" state can now be performed properly. Previously all allocation of
ex_data state was guaranteed to leak - and MemCheck_off() had been used to
avoid it flagging up the memory debugging. A new function has been added to
perfrom all this cleanup, CRYPTO_cleanup_all_ex_data(). The "openssl"
command(s) have been changed to use this cleanup, as have the relevant test
programs. External application code may want to do so too - failure to
cleanup will not induce more memory leaking than was the case before, but
the memory debugging is not tricked into hiding it any more so it may
"appear" where it previously did not.
