Dual HTTP/1.1 and HTTP/2 support at M5

[rjb1000]

Context

In order to unblock development of the Media Session Handler, issue #10 temporarily removed HTTP/2 support at reference point M5 in favour of HTTP/1.1. However, TS 26.512 clause 6.2.1.1 requires support for both versions of HTTP at this reference point (and at reference point M1).

The Open5GS framework currently supports only HTTP/2, but was patched to support HTTP/1.1 instead in #10.

This issue modifies the implementation so that the HTTP client is able to choose which of the two protocols it wishes to use, bringing it more in line with the requirements of TS 26.512.

Specification

Adjust lib/sbi/server.c behaviour in the Open5GS framework to support on the same server listening port:

• HTTP/1.1.
• HTTP/2 (cleartext h2c only) with prior knowledge, according to the procedure specified in section 3.4 of RFC 7540.

Detection of the HTTP/2 connection preface byte string (see section 3.5 of RFC 7540) moves the server into HTTP/2 (h2c) mode. Otherwise, it assumes HTTP/1.1 operation.

Support for protocol upgrade to HTTP/2 (h2c) operation after a period of HTTP/1.1 operation of an HTTP session (per section 6.7 of RFC 7230) is not required.

Adding TLS support (on a different server listening port) is not in the scope of this issue and will be tackled separately, possibly within the Open5GS project itself after discussion with the maintainer of that project.

Implementation notes

HTTP/1.1 support is of little use to the main Open5GS project because all service-based interactions in the Control Plane use HTTP/2 per TS 33.501 clause 13.1.0: this change to support both HTTP versions is only applicable when interacting with Application Functions in the Data Network.

It is therefore unclear whether it is best to change the server behaviour locally in this repository, or to clone the Open5GS repository and make it a submodule of this repository.

And there may be no value in contributing this change back to the Open5GS project.

[davidjwbbc]

Open5GS already links in the microhttpd, nghttp2, gnutls and curl libraries (even though it doesn't use the microhttpd library anymore). Therefore I've been looking at how we can support HTTP/1.1 and HTTP/2 with and without TLS using these libraries.

Microhttpd likes to manage its own sockets and select/epoll/poll loop, however it is possible to tell it that you wish to use external functions to manage some of these things. For example it is possible to allow the application to listen to a socket for connections and just pass the accepted client connections onto microhttpd to manage. You can also opt to run your own select/poll/epoll loop and feed the results to microhttpd for processing. If you are using TLS, then microhttpd will feed the data into a gnuTLS session. It is possible to obtain the gnuTLS session context from the microhttpd connection information. Microhttpd errors when it sees a PRI request (HTTP/2 prior knowledge) as the HTTP version in the request is not 1.x.

nghttp2 prefers the application to manage sockets and uses callbacks to read and write data. This makes it easy to insert a TLS layer (or any other transport method you care to add).

Given all this it looks like the best way for full integration is:

non-TLS

• Use our own listening socket and "external" polling loop for the socket fds. When a connection is accepted it is added to a list of possible intercept fds and included in the polling loop.
• If data arrives on a possible intercept socket it is snooped for the PRI * HTTP/2\r\n\r\n string and if the string is present the fd is moved to the http2 fds list, an http2 session started to handle the incoming data and the first data is pushed to the session. If the PRI ... string was not present then the MHD_add_connection is used to add the fd to microhttpd, the fd is moved to an http1 list of fds and the select result is passed to microhttpd to start processing the response.
• If data arrives for fds on the http1 list then the select response is passed to microhttpd to deal with.
• If data arrives on fds on the http2 list then the nghttp2 session is informed that data is ready to read and the data is passed through the callback.
• If microhttpd notices a Connection: upgrade request for h2c then the 101 response is sent, the socket is suspended in microhttpd, a session is started in nghttp2 and the fd is moved from the http1 list to the http2 list.
• The rest of the sessions are handled using the nghttp2 or microhttpd callbacks as appropriate.

This should allow h2c prior knowledge and connection upgrades to work when TLS is not in use.

TLS

• Allow microhttpd to start the server in TLS mode but use "external" select on the fds to allow transfer to nghttp2 if we need to. Get the gnuTLS session and setup the ALPN list and an ALPN callback.
• If the ALPN callback sees an h2 protocol request it can suspend microhttpd operation on the socket, set up a TLS nghttp2 session (alternative call so that nghttp2 isn't expecting the PRI ... message) and use the gnuTLS session with the socket fd to transfer the connection to nghttp2, passing the data along via the select/poll/epoll loop. Any other ALPN request is left to microhttpd to handle.

I think between these two methods we should be able to provide full HTTP/1.1 and HTTP/2 with optional TLS for the application function.

Note: The open5gs PR open5gs/open5gs#1865 has now been merged and introduces the OpenSSL library alongside gnuTLS. It works by using OpenSSL SSL/TLS routines to manage a socket if tls parameters have been set in the configuration. Although the basics of the operation for this patch can be used (tls parameters in the configuration files and using the TLS routines to pass data to nghttp2) we are stuck with microhttpd taking a lot of the control and using gnuTLS in order to support HTTP/1.1, so this PR is not useful to us.