Files
cpp-httplib/test
yhirose b2a81c0019 Add keep-alive parking (Server::set_keep_alive_parking())
A kept-alive connection holds a worker thread while it waits for its
next request, so connections beyond the thread count wait for one to be
closed, and some clients get nothing until another connection ends.

With keep-alive parking, listen() runs an event loop on the
detail::Poller instead of the blocking accept loop. A connection waiting
for its next request is parked there and goes back to a worker through
the TaskQueue when the request arrives. The handlers, content providers
and WebSocket keep their blocking API and still hold a thread while they
run. SSLServer supports it too: a worker runs the TLS handshake once the
client starts it, and the session stays with the parked connection.

It is opt-in and off by default. With it on:
- consecutive requests on a connection may run on different threads;
- when the queue is full, a connection that has not been served (or has
  not finished its TLS handshake) is closed as before, while an
  established one waits in a backlog instead of losing its request;
- a parked connection frees its read buffer, and with OpenSSL 3.0.14,
  3.1.6, 3.2.2, 3.3.1 or later also its TLS record buffers;
- handing a connection over adds roughly 10-20 us to a request;
- a subclass that overrides the private process_and_close_socket() is
  bypassed.

On two CPUs with two workers, 50 keep-alive clients of a 2.7 ms handler
were all served evenly (the slowest response took about 70 ms instead of
5 s, with 10 clients getting nothing), light requests from 50 clients
went from 79k to about 240k per second, and with 10000 idle connections
parked, 10 active clients got 95k requests per second where the old loop
served none. An idle connection costs under 1 KB, or about 15 KB with
TLS.

The old accept loop moved unchanged into accept_loop(). KeepAliveConnection
holds a parked connection, PlainConnection and SSLServer::TlsConnection
add the transport, and SSLServer::start_session() holds the session setup
and handshake that both paths use.
2026-11-21 21:23:22 -05:00
..
2021-09-11 14:26:48 -04:00
2021-09-11 14:26:48 -04:00
2017-12-29 22:34:59 -05:00
2013-07-04 18:18:52 -04:00
2024-11-16 11:14:13 -05:00