mirror of
https://github.com/yhirose/cpp-httplib.git
synced 2026-11-22 20:11:46 +00:00
A kept-alive connection holds a worker thread while it waits for its next request, so connections beyond the thread count wait for one to be closed, and some clients get nothing until another connection ends. With keep-alive parking, listen() runs an event loop on the detail::Poller instead of the blocking accept loop. A connection waiting for its next request is parked there and goes back to a worker through the TaskQueue when the request arrives. The handlers, content providers and WebSocket keep their blocking API and still hold a thread while they run. SSLServer supports it too: a worker runs the TLS handshake once the client starts it, and the session stays with the parked connection. It is opt-in and off by default. With it on: - consecutive requests on a connection may run on different threads; - when the queue is full, a connection that has not been served (or has not finished its TLS handshake) is closed as before, while an established one waits in a backlog instead of losing its request; - a parked connection frees its read buffer, and with OpenSSL 3.0.14, 3.1.6, 3.2.2, 3.3.1 or later also its TLS record buffers; - handing a connection over adds roughly 10-20 us to a request; - a subclass that overrides the private process_and_close_socket() is bypassed. On two CPUs with two workers, 50 keep-alive clients of a 2.7 ms handler were all served evenly (the slowest response took about 70 ms instead of 5 s, with 10 clients getting nothing), light requests from 50 clients went from 79k to about 240k per second, and with 10000 idle connections parked, 10 active clients got 95k requests per second where the old loop served none. An idle connection costs under 1 KB, or about 15 KB with TLS. The old accept loop moved unchanged into accept_loop(). KeepAliveConnection holds a parked connection, PlainConnection and SSLServer::TlsConnection add the transport, and SSLServer::start_session() holds the session setup and handshake that both paths use.