openssl/ssl/statem
Matt Caswell c8e2f98c97 Partial revert of "Fix client verify mode to check SSL_VERIFY_PEER"
This partially reverts commit c636c1c47. It also tweaks the documentation
and comments in this area. On the client side the documented interface for
SSL_CTX_set_verify()/SSL_set_verify() is that setting the flag
SSL_VERIFY_PEER causes verfication of the server certificate to take place.
Previously what was implemented was that if *any* flag was set then
verification would take place. The above commit improved the semantics to
be as per the documented interface.

However, we have had a report of at least one application where an
application was incorrectly using the interface and used *only*
SSL_VERIFY_FAIL_IF_NO_PEER_CERT on the client side. In OpenSSL prior to
the above commit this still caused verification of the server certificate
to take place. After this commit the application silently failed to verify
the server certificate.

Ideally SSL_CTX_set_verify()/SSL_set_verify() could be modified to indicate
if invalid flags were being used. However these are void functions!

The simplest short term solution is to revert to the previous behaviour
which at least means we "fail closed" rather than "fail open".

Thanks to Cory Benfield for reporting this issue.

Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-11-07 16:02:50 +00:00
..
README Add a state machine README 2015-10-30 08:38:18 +00:00
statem.c Further libssl size_t-ify of reading 2016-11-04 12:09:45 +00:00
statem.h Indent ssl/ 2016-08-18 14:02:29 +02:00
statem_clnt.c Partial revert of "Fix client verify mode to check SSL_VERIFY_PEER" 2016-11-07 16:02:50 +00:00
statem_dtls.c Rename all "read" variables with "readbytes" 2016-11-04 12:09:46 +00:00
statem_lib.c Always ensure that init_msg is initialised for a CCS 2016-11-07 15:22:33 +00:00
statem_locl.h Further libssl size_t-ify of reading 2016-11-04 12:09:45 +00:00
statem_srvr.c Fix misc size_t issues causing Windows warnings in 64 bit 2016-11-04 12:09:46 +00:00

State Machine Design
====================

This file provides some guidance on the thinking behind the design of the
state machine code to aid future maintenance.

The state machine code replaces an older state machine present in OpenSSL
versions 1.0.2 and below. The new state machine has the following objectives:
    - Remove duplication of state code between client and server
    - Remove duplication of state code between TLS and DTLS
    - Simplify transitions and bring the logic together in a single location
      so that it is easier to validate
    - Remove duplication of code between each of the message handling functions
    - Receive a message first and then work out whether that is a valid
      transition - not the other way around (the other way causes lots of issues
      where we are expecting one type of message next but actually get something
      else)
    - Separate message flow state from handshake state (in order to better
      understand each)
      - message flow state = when to flush buffers; handling restarts in the
        event of NBIO events; handling the common flow of steps for reading a
        message and the common flow of steps for writing a message etc
      - handshake state = what handshake message are we working on now
    - Control complexity: only the state machine can change state: keep all
      the state changes local to the state machine component

The message flow state machine is divided into a reading sub-state machine and a
writing sub-state machine. See the source comments in statem.c for a more
detailed description of the various states and transitions possible.

Conceptually the state machine component is designed as follows:

                        libssl
                           |
---------------------------|-----statem.h--------------------------------------
                           |
                    _______V____________________
                   |                            |
                   |    statem.c                |
                   |                            |
                   |    Core state machine code |
                   |____________________________|
        statem_locl.h     ^          ^
                 _________|          |_______
                |                            |
   _____________|____________   _____________|____________
  |                          | |                          |
  | statem_clnt.c            | | statem_srvr.c            |
  |                          | |                          |
  | TLS/DTLS client specific | | TLS/DTLS server specific |
  | state machine code       | | state machine code       |
  |__________________________| |__________________________|
               |        |_______________|__       |
               |        ________________|  |      |
               |       |                   |      |
   ____________V_______V________   ________V______V_______________
  |                             | |                               |
  | statem_both.c               | | statem_dtls.c                 |
  |                             | |                               |
  | Non core functions common   | | Non core functions common to  |
  | to both servers and clients | | both DTLS servers and clients |
  |_____________________________| |_______________________________|