341 lines
13 KiB
Markdown
341 lines
13 KiB
Markdown
|
Report on the Conclusions of the QUIC DDD Process
|
||
|
|
=================================================
|
||
|
|
|
||
|
|
The [QUIC Demo-Driven Design process](README.md) was undertaken to meet the OMC
|
||
|
|
requirement to develop a QUIC API that required only minimal changes to existing
|
||
|
|
applications to be able to adapt their code to use QUIC. The demo-driven design
|
||
|
|
process developed a set of representative demos modelling a variety of common
|
||
|
|
OpenSSL usage patterns based on analysis of a broad spectrum of open source
|
||
|
|
software projects using OpenSSL.
|
||
|
|
|
||
|
|
As part of this process, a set of proposed diffs were produced. These proposed
|
||
|
|
diffs were the expected changes which would be needed to the baseline demos to
|
||
|
|
support QUIC based on theoretical analysis of the minimum requirements to be
|
||
|
|
able to support QUIC. This analysis concluded that the changes needed to
|
||
|
|
applications could be kept very small in many circumstances, with only minimal
|
||
|
|
diff sizes to the baseline demos.
|
||
|
|
|
||
|
|
Following the development of QUIC MVP, these demos have been revisited and the
|
||
|
|
correspondence of our actual final API and usage patterns with the planned diffs
|
||
|
|
have been reviewed.
|
||
|
|
|
||
|
|
This document discusses the planned changes and the actual changes for each demo
|
||
|
|
and draws conclusions on the level of disparity.
|
||
|
|
|
||
|
|
Since tracking a set of diffs separately is unwieldy, both the planned and
|
||
|
|
unplanned changes have been folded into the original baseline demo files guarded
|
||
|
|
with `#ifdef USE_QUIC`. Viewing these files therefore is informative to
|
||
|
|
application writers as it provides a clear view of what is different when using
|
||
|
|
QUIC. (The originally planned changes, and the final changes, are added in
|
||
|
|
separate, clearly-labelled commits; to view the originally planned changes only,
|
||
|
|
view the commit history for a given demo file.)
|
||
|
|
|
||
|
|
ddd-01-conn-blocking
|
||
|
|
--------------------
|
||
|
|
|
||
|
|
This demo exists to demonstrate the simplest possible usage of OpenSSL, whether
|
||
|
|
with TLS or QUIC.
|
||
|
|
|
||
|
|
### Originally planned changes
|
||
|
|
|
||
|
|
The originally planned change to enable applications for QUIC amounted to just a
|
||
|
|
single line:
|
||
|
|
|
||
|
|
```diff
|
||
|
|
+ ctx = SSL_CTX_new(QUIC_client_method());
|
||
|
|
- ctx = SSL_CTX_new(TLS_client_method());
|
||
|
|
```
|
||
|
|
|
||
|
|
### Actual changes
|
||
|
|
|
||
|
|
The following additional changes needed to be made:
|
||
|
|
|
||
|
|
- `QUIC_client_method` was renamed to `OSSL_QUIC_client_method` for namespacing
|
||
|
|
reasons.
|
||
|
|
|
||
|
|
- A call to `SSL_set_alpn_protos` to configure ALPN was added. This is necessary
|
||
|
|
because QUIC mandates the use of ALPN, and this was not noted during the
|
||
|
|
DDD process.
|
||
|
|
|
||
|
|
ddd-02-conn-nonblocking
|
||
|
|
-----------------------
|
||
|
|
|
||
|
|
This demo exists to demonstrate simple non-blocking usage. As with
|
||
|
|
ddd-01-conn-blocking, the name resolution process is managed by `BIO_s_connect`.
|
||
|
|
|
||
|
|
It also arbitrarily adds a `BIO_f_buffer` pushed onto the BIO stack
|
||
|
|
as this is a common application usage pattern.
|
||
|
|
|
||
|
|
### Originally planned changes
|
||
|
|
|
||
|
|
The originally planned changes to enable applications for QUIC amounted to:
|
||
|
|
|
||
|
|
- Change of method (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- Use of a `BIO_f_dgram_buffer` BIO method instead of a `BIO_f_buffer`;
|
||
|
|
|
||
|
|
- Use of a `BIO_get_poll_fd` function to get the FD to poll rather than
|
||
|
|
`BIO_get_fd`;
|
||
|
|
|
||
|
|
- A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2)
|
||
|
|
need to be determined.
|
||
|
|
|
||
|
|
- Additional functions in application code to determine event handling
|
||
|
|
timeouts related to QUIC (`get_conn_pump_timeout`) and to pump
|
||
|
|
the QUIC event loop (`pump`).
|
||
|
|
|
||
|
|
- Timeout computation code which involves merging and comparing different
|
||
|
|
timeouts and calling `pump` as needed, based on deadlines reported
|
||
|
|
by libssl.
|
||
|
|
|
||
|
|
Note that some of these changes are unnecessary when using the thread assisted
|
||
|
|
mode (see the variant ddd-02-conn-nonblocking-threads below).
|
||
|
|
|
||
|
|
### Actual changes
|
||
|
|
|
||
|
|
The following additional changes needed to be made:
|
||
|
|
|
||
|
|
- Change of method name (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- Use of ALPN (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- The strategy for how to expose pollable OS resource handles
|
||
|
|
to applications to determine I/O readiness has changed substantially since the
|
||
|
|
original DDD process. As such, applications now use `BIO_get_rpoll_descriptor`
|
||
|
|
and `BIO_get_wpoll_descriptor` to determine I/O readiness, rather than the
|
||
|
|
originally hypothesised `SSL_get_poll_fd`.
|
||
|
|
|
||
|
|
- The strategy for how to determine when to poll for `POLLIN`, when to
|
||
|
|
poll for `POLLOUT`, etc. has changed since the original DDD process.
|
||
|
|
This information is now exposed via `SSL_net_read_desired` and
|
||
|
|
`SSL_net_write_desired`.
|
||
|
|
|
||
|
|
- The API to expose the event handling deadline for the QUIC engine
|
||
|
|
has evolved since the original DDD process. The new API
|
||
|
|
`SSL_get_event_timeout` is used, rather than the originally hypothesised
|
||
|
|
`BIO_get_timeout`/`SSL_get_timeout`.
|
||
|
|
|
||
|
|
- The API to perform QUIC event processing has been renamed to be
|
||
|
|
more descriptive. It is now called `SSL_handle_events` rather than
|
||
|
|
the originally hypothesised `BIO_pump`/`SSL_pump`.
|
||
|
|
|
||
|
|
The following changes were foreseen to be necessary, but turned out to actually
|
||
|
|
not be necessary:
|
||
|
|
|
||
|
|
- The need to change code which pushes a `BIO_f_buffer()` after a SSL BIO
|
||
|
|
was foreseen as use of buffering on the network side is unworkable with
|
||
|
|
QUIC. This turned out not to be necessary since we can just reject the
|
||
|
|
BIO_push() call. The buffer should still be freed eventually when the
|
||
|
|
SSL BIO is freed. The buffer is not used and is unnecessary, so it is
|
||
|
|
still desirable for applications to remove this code.
|
||
|
|
|
||
|
|
ddd-02-conn-nonblocking-threads
|
||
|
|
-------------------------------
|
||
|
|
|
||
|
|
This is a variant of the ddd-02-conn-nonblocking demo. The base is the same, but
|
||
|
|
the changes made are different. The use of thread-assisted mode, in which an
|
||
|
|
internal assist thread is used to perform QUIC event handling, enables an
|
||
|
|
application to make fewer changes than are needed in the ddd-02-conn-nonblocking
|
||
|
|
demo.
|
||
|
|
|
||
|
|
### Originally planned changes
|
||
|
|
|
||
|
|
The originally planned changes to enable applications for QUIC amounted to:
|
||
|
|
|
||
|
|
- Change of method, this time using method `QUIC_client_thread_method` rather
|
||
|
|
than `QUIC_client_method`;
|
||
|
|
|
||
|
|
- Use of a `BIO_get_poll_fd` function to get the FD to poll rather than
|
||
|
|
`BIO_get_fd`;
|
||
|
|
|
||
|
|
- A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2)
|
||
|
|
need to be determined.
|
||
|
|
|
||
|
|
Note that this is a substantially smaller list of changes than for
|
||
|
|
ddd-02-conn-nonblocking.
|
||
|
|
|
||
|
|
### Actual changes
|
||
|
|
|
||
|
|
The following additional changes needed to be made:
|
||
|
|
|
||
|
|
- Change of method name (`QUIC_client_thread_method` was renamed to
|
||
|
|
`OSSL_QUIC_client_thread_method` for namespacing reasons);
|
||
|
|
|
||
|
|
- Use of ALPN (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- Use of `BIO_get_rpoll_descriptor` rather than `BIO_get_poll_fd` (as for
|
||
|
|
ddd-02-conn-nonblocking).
|
||
|
|
|
||
|
|
- Use of `SSL_net_read_desired` and `SSL_net_write_desired` (as for
|
||
|
|
ddd-02-conn-nonblocking).
|
||
|
|
|
||
|
|
ddd-03-fd-blocking
|
||
|
|
------------------
|
||
|
|
|
||
|
|
This demo is similar to ddd-01-conn-blocking but uses a file descriptor passed
|
||
|
|
directly by the application rather than BIO_s_connect.
|
||
|
|
|
||
|
|
### Originally planned changes
|
||
|
|
|
||
|
|
- Change of method (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- The arguments to the `socket(2)` call are changed from `(AF_INET, SOCK_STREAM,
|
||
|
|
IPPROTO_TCP)` to `(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`.
|
||
|
|
|
||
|
|
### Actual changes
|
||
|
|
|
||
|
|
The following additional changes needed to be made:
|
||
|
|
|
||
|
|
- Change of method name (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- Use of ALPN (as for ddd-01-conn-blocking).
|
||
|
|
|
||
|
|
ddd-04-fd-nonblocking
|
||
|
|
---------------------
|
||
|
|
|
||
|
|
This demo is similar to ddd-01-conn-nonblocking but uses a file descriptor
|
||
|
|
passed directly by the application rather than BIO_s_connect.
|
||
|
|
|
||
|
|
### Originally planned changes
|
||
|
|
|
||
|
|
- Change of method (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- The arguments to the `socket(2)` call are changed from `(AF_INET, SOCK_STREAM,
|
||
|
|
IPPROTO_TCP)` to `(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`;
|
||
|
|
|
||
|
|
- A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2)
|
||
|
|
need to be determined.
|
||
|
|
|
||
|
|
- Additional functions in application code to determine event handling
|
||
|
|
timeouts related to QUIC (`get_conn_pump_timeout`) and to pump
|
||
|
|
the QUIC event loop (`pump`).
|
||
|
|
|
||
|
|
- Timeout computation code which involves merging and comparing different
|
||
|
|
timeouts and calling `pump` as needed, based on deadlines reported
|
||
|
|
by libssl.
|
||
|
|
|
||
|
|
### Actual changes
|
||
|
|
|
||
|
|
The following additional changes needed to be made:
|
||
|
|
|
||
|
|
- Change of method name (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- Use of ALPN (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- `SSL_get_timeout` replaced with `SSL_get_event_timeout` (as for
|
||
|
|
ddd-02-conn-nonblocking);
|
||
|
|
|
||
|
|
- `SSL_pump` renamed to `SSL_handle_events` (as for ddd-02-conn-nonblocking);
|
||
|
|
|
||
|
|
- The strategy for how to determine when to poll for `POLLIN`, when to
|
||
|
|
poll for `POLLOUT`, etc. has changed since the original DDD process.
|
||
|
|
This information is now exposed via `SSL_net_read_desired` and
|
||
|
|
`SSL_net_write_desired` (as for ddd-02-conn-nonblocking).
|
||
|
|
|
||
|
|
ddd-05-mem-nonblocking
|
||
|
|
----------------------
|
||
|
|
|
||
|
|
This demo is more elaborate. It uses memory buffers created and managed by an
|
||
|
|
application as an intermediary between libssl and the network, which is a common
|
||
|
|
usage pattern for applications. Managing this pattern for QUIC is more elaborate
|
||
|
|
since datagram semantics on the network channel need to be maintained.
|
||
|
|
|
||
|
|
### Originally planned changes
|
||
|
|
|
||
|
|
- Change of method (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- Call to `BIO_new_bio_pair` is changed to `BIO_new_dgram_pair`, which
|
||
|
|
provides a bidirectional memory buffer BIO with datagram semantics.
|
||
|
|
|
||
|
|
- A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2)
|
||
|
|
need to be determined.
|
||
|
|
|
||
|
|
- Potential changes to buffer sizes used by applications to buffer
|
||
|
|
datagrams, if those buffers are smaller than 1472 bytes.
|
||
|
|
|
||
|
|
- The arguments to the `socket(2)` call are changed from `(AF_INET, SOCK_STREAM,
|
||
|
|
IPPROTO_TCP)` to `(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`;
|
||
|
|
|
||
|
|
### Actual changes
|
||
|
|
|
||
|
|
The following additional changes needed to be made:
|
||
|
|
|
||
|
|
- Change of method name (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- Use of ALPN (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- The API to construct a `BIO_s_dgram_pair` ended up being named
|
||
|
|
`BIO_new_bio_dgram_pair` rather than `BIO_new_dgram_pair`;
|
||
|
|
|
||
|
|
- Use of `SSL_net_read_desired` and `SSL_net_write_desired` (as for
|
||
|
|
ddd-02-conn-nonblocking).
|
||
|
|
|
||
|
|
ddd-06-mem-uv
|
||
|
|
-------------
|
||
|
|
|
||
|
|
This demo is the most elaborate of the set. It uses a real-world asynchronous
|
||
|
|
I/O reactor, namely libuv (the engine used by Node.js). In doing so it seeks to
|
||
|
|
demonstrate and prove the viability of our API design with a real-world
|
||
|
|
asynchronous I/O system. It operates wholly in non-blocking mode and uses memory
|
||
|
|
buffers on either side of the QUIC stack to feed data to and from the
|
||
|
|
application and the network.
|
||
|
|
|
||
|
|
### Originally planned changes
|
||
|
|
|
||
|
|
- Change of method (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- Various changes to use of libuv needed to switch to using UDP;
|
||
|
|
|
||
|
|
- Additional use of libuv to configure a timer event;
|
||
|
|
|
||
|
|
- Call to `BIO_new_bio_pair` is changed to `BIO_new_dgram_pair`
|
||
|
|
(as for ddd-05-mem-nonblocking);
|
||
|
|
|
||
|
|
- Some reordering of code required by the design of libuv.
|
||
|
|
|
||
|
|
### Actual changes
|
||
|
|
|
||
|
|
The following additional changes needed to be made:
|
||
|
|
|
||
|
|
- Change of method name (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- Use of ALPN (as for ddd-01-conn-blocking);
|
||
|
|
|
||
|
|
- `BIO_new_dgram_pair` renamed to `BIO_new_bio_dgram_pair` (as for
|
||
|
|
ddd-05-mem-nonblocking);
|
||
|
|
|
||
|
|
- `SSL_get_timeout` replaced with `SSL_get_event_timeout` (as for
|
||
|
|
ddd-02-conn-nonblocking);
|
||
|
|
|
||
|
|
- `SSL_pump` renamed to `SSL_handle_events` (as for ddd-02-conn-nonblocking);
|
||
|
|
|
||
|
|
- Fixes to use of libuv based on a corrected understanding
|
||
|
|
of its operation, and changes that necessarily ensue.
|
||
|
|
|
||
|
|
Conclusions
|
||
|
|
-----------
|
||
|
|
|
||
|
|
The DDD process has successfully delivered on the objective of delivering a QUIC
|
||
|
|
API which can be used with only minimal API changes. The additional changes on
|
||
|
|
top of those originally planned which were required to successfully execute the
|
||
|
|
demos using QUIC were highly limited in scope and mostly constituted only minor
|
||
|
|
changes. The sum total of the changes required for each demo (both planned and
|
||
|
|
additional), as denoted in each DDD demo file under `#ifdef USE_QUIC` guards,
|
||
|
|
are both minimal and limited in scope.
|
||
|
|
|
||
|
|
“Minimal” and “limited” are distinct criteria. If inexorable technical
|
||
|
|
requirements dictate, an enormous set of changes to an application could be
|
||
|
|
considered “minimal”. The changes required to representative applications, as
|
||
|
|
demonstrated by the DDD demos, are not merely minimal but also limited.
|
||
|
|
|
||
|
|
For example, while the extent of these necessary changes varies by the
|
||
|
|
sophistication of each demo and the kind of application usage pattern it
|
||
|
|
represents, some demos in particular demonstrate exceptionally small changesets;
|
||
|
|
for example, ddd-01-conn-blocking and ddd-02-conn-nonblocking-threads, with
|
||
|
|
ddd-01-conn-blocking literally being enabled by a single line change assuming
|
||
|
|
ALPN is already configured.
|
||
|
|
|
||
|
|
This report concludes the DDD process for the single-stream QUIC client API
|
||
|
|
design process, which sought to validate our API design and API ease of use for
|
||
|
|
existing applications seeking to adopt QUIC.
|