openssl/ssl/record
Matt Caswell e3d0dae7cf DTLSv1_listen rewrite
The existing implementation of DTLSv1_listen() is fundamentally flawed. This
function is used in DTLS solutions to listen for new incoming connections
from DTLS clients. A client will send an initial ClientHello. The server
will respond with a HelloVerifyRequest containing a unique cookie. The
client the responds with a second ClientHello - which this time contains the
cookie.

Once the cookie has been verified then DTLSv1_listen() returns to user code,
which is typically expected to continue the handshake with a call to (for
example) SSL_accept().

Whilst listening for incoming ClientHellos, the underlying BIO is usually in
an unconnected state. Therefore ClientHellos can come in from *any* peer.
The arrival of the first ClientHello without the cookie, and the second one
with it, could be interspersed with other intervening messages from
different clients.

The whole purpose of this mechanism is as a defence against DoS attacks. The
idea is to avoid allocating state on the server until the client has
verified that it is capable of receiving messages at the address it claims
to come from. However the existing DTLSv1_listen() implementation completely
fails to do this. It attempts to super-impose itself on the standard state
machine and reuses all of this code. However the standard state machine
expects to operate in a stateful manner with a single client, and this can
cause various problems.

A second more minor issue is that the return codes from this function are
quite confused, with no distinction made between fatal and non-fatal errors.
Most user code treats all errors as non-fatal, and simply retries the call
to DTLSv1_listen().

This commit completely rewrites the implementation of DTLSv1_listen() and
provides a stand alone implementation that does not rely on the existing
state machine. It also provides more consistent return codes.

Reviewed-by: Andy Polyakov <appro@openssl.org>
2015-09-23 13:53:26 +01:00
..
dtls1_bitmap.c Define SEQ_NUM_SIZE 2015-03-26 17:25:48 +00:00
README Version negotiation rewrite cleanup 2015-05-16 09:20:38 +01:00
rec_layer_d1.c DTLSv1_listen rewrite 2015-09-23 13:53:26 +01:00
rec_layer_s3.c Clean up reset of read/write sequences 2015-08-26 16:22:45 +01:00
record.h DTLSv1_listen rewrite 2015-09-23 13:53:26 +01:00
record_locl.h Fix a memory leak in compression 2015-05-22 08:08:45 +01:00
ssl3_buffer.c Fix DTLS1.2 buffers 2015-09-02 00:34:14 +01:00
ssl3_record.c CCM support. 2015-08-14 06:56:11 +01:00

Record Layer Design
===================

This file provides some guidance on the thinking behind the design of the
record layer code to aid future maintenance.

The record layer is divided into a number of components. At the time of writing
there are four: SSL3_RECORD, SSL3_BUFFER, DLTS1_BITMAP and RECORD_LAYER. Each
of these components is defined by:
1) A struct definition of the same name as the component
2) A set of source files that define the functions for that component
3) A set of accessor macros

All struct definitions are in record.h. The functions and macros are either
defined in record.h or record_locl.h dependent on whether they are intended to
be private to the record layer, or whether they form part of the API to the rest
of libssl.

The source files map to components as follows:

dtls1_bitmap.c                                   -> DTLS1_BITMAP component
ssl3_buffer.c                                    -> SSL3_BUFFER component
ssl3_record.c                                    -> SSL3_RECORD component
rec_layer_s3.c, rec_layer_d1.c                   -> RECORD_LAYER component

The RECORD_LAYER component is a facade pattern, i.e. it provides a simplified
interface to the record layer for the rest of libssl. The other 3 components are
entirely private to the record layer and therefore should never be accessed
directly by libssl.

Any component can directly access its own members - they are private to that
component, e.g. ssl3_buffer.c can access members of the SSL3_BUFFER struct
without using a macro. No component can directly access the members of another
component, e.g. ssl3_buffer cannot reach inside the RECORD_LAYER component to
directly access its members. Instead components use accessor macros, so if code
in ssl3_buffer.c wants to access the members of the RECORD_LAYER it uses the
RECORD_LAYER_* macros.

Conceptually it looks like this:

                        libssl
                           |
---------------------------|-----record.h--------------------------------------
                           |
                    _______V______________
                   |                      |
                   |    RECORD_LAYER      |
                   |                      |
                   |    rec_layer_s3.c    |
                   |          ^           |
                   | _________|__________ |
                   ||                    ||
                   || DTLS1_RECORD_LAYER ||
                   ||                    ||
                   || rec_layer_d1.c     ||
                   ||____________________||
                   |______________________|
        record_locl.h     ^   ^   ^
         _________________|   |   |_________________
        |                     |                     |
   _____V_________      ______V________      _______V________
  |               |    |               |    |                |
  | SSL3_BUFFER   |    | SSL3_RECORD   |    | DTLS1_BITMAP   |
  |               |--->|               |    |                |
  | ssl3_buffer.c |    | ssl3_record.c |    | dtls1_bitmap.c |
  |_______________|    |_______________|    |________________|


The two RECORD_LAYER source files build on each other, i.e.
the main one is rec_layer_s3.c which provides the core SSL/TLS layer. The second
one is rec_layer_d1.c which builds off of the SSL/TLS code to provide DTLS
specific capabilities. It uses some DTLS specific RECORD_LAYER component members
which should only be accessed from rec_layer_d1.c. These are held in the
DTLS1_RECORD_LAYER struct.