Compare commits

..
Author SHA1 Message Date
yhirose a8d58085cb Run the test suite with keep-alive parking on in CI, and mark it experimental
CPPHTTPLIB_KEEPALIVE_PARKING makes every Server start with parking on.
The OpenSSL jobs on Ubuntu (epoll) and macOS (kqueue) now also run the
whole suite built with it, which used to be done only locally. The two
tests that check the blocking path turn parking off explicitly. README
and the cookbook call the feature experimental and say that a server
whose event loop cannot be set up serves as it does without it.
2026-11-21 20:47:45 -05:00
yhirose 00e2fd89e8 Describe keep-alive parking in the Keep-Alive cookbook page 2026-11-21 20:47:45 -05:00
yhirose 530afcc12e Add keep-alive parking (Server::set_keep_alive_parking())
A kept-alive connection holds a worker thread while it waits for its
next request, so connections beyond the thread count wait for one to be
closed, and some clients get nothing until another connection ends.

With keep-alive parking, listen() runs an event loop on the
detail::Poller instead of the blocking accept loop. A connection waiting
for its next request is parked there and goes back to a worker through
the TaskQueue when the request arrives. The handlers, content providers
and WebSocket keep their blocking API and still hold a thread while they
run. SSLServer supports it too: a worker runs the TLS handshake once the
client starts it, and the session stays with the parked connection.

It is opt-in and off by default. With it on:
- consecutive requests on a connection may run on different threads;
- when the queue is full, a connection that has not been served (or has
  not finished its TLS handshake) is closed as before, while an
  established one waits in a backlog instead of losing its request;
- a parked connection frees its read buffer, and with OpenSSL 3.0.14,
  3.1.6, 3.2.2, 3.3.1 or later also its TLS record buffers;
- handing a connection over adds roughly 10-20 us to a request;
- a subclass that overrides the private process_and_close_socket() is
  bypassed.

On two CPUs with two workers, 50 keep-alive clients of a 2.7 ms handler
were all served evenly (the slowest response took about 70 ms instead of
5 s, with 10 clients getting nothing), light requests from 50 clients
went from 79k to about 240k per second, and with 10000 idle connections
parked, 10 active clients got 95k requests per second where the old loop
served none. An idle connection costs under 1 KB, or about 15 KB with
TLS.

The old accept loop moved unchanged into accept_loop(). KeepAliveConnection
holds a parked connection, PlainConnection and SSLServer::TlsConnection
add the transport, and SSLServer::start_session() holds the session setup
and handshake that both paths use.
2026-11-21 20:47:45 -05:00
yhirose da42c375ca Pass the keep-alive wait and the served count into the server loop
process_server_socket_core() now takes the function that waits for the
next request and the number of requests served so far, instead of
calling keep_alive() and counting on its own stack. Both callers pass
keep_alive() and a fresh counter, so nothing changes yet; this lets a
caller stop waiting and resume the same connection later without losing
track of keep_alive_max_count.
2026-11-21 20:22:05 -05:00
yhirose a694b8c306 Add a one-shot Poller with poll, epoll and kqueue backends
Groundwork for keep-alive parking, an opt-in event loop that parks idle
keep-alive connections instead of blocking a worker thread on each.

detail::Poller delivers readiness once per arm(), so a descriptor is
either armed (owned by the poller) or owned by whoever got the event.
wake() ends a wait() from any thread, and an armed descriptor must be
disarmed before it is closed.

- PollPoller (poll() / WSAPoll()) works everywhere.
- EpollPoller (Linux) uses EPOLLONESHOT.
- KqueuePoller (macOS, FreeBSD) uses EV_ONESHOT.
- PollerWaker carries wake() for all three: an eventfd on Linux, a socket
  pair elsewhere. detail::create_socket_pair() emulates socketpair() on
  Windows with sockets that child processes do not inherit.

make_poller() picks the best one and falls back to poll(). The contract
tests run on every backend that builds. Nothing uses the poller yet.
2026-11-21 20:22:05 -05:00
2 changed files with 13 additions and 51 deletions

No files matched your search

+13 -23
View File
@@ -4034,7 +4034,8 @@ private:
PollerWaker waker_;
// fds_[0] is the waker's, always armed; tokens_[i] belongs to fds_[i].
// fds_[0] is the waker's, which is always armed; tokens_[i] belongs to
// fds_[i].
std::vector<struct pollfd> fds_;
std::vector<Token> tokens_;
std::unordered_map<socket_t, size_t> index_;
@@ -4087,11 +4088,6 @@ private:
// The best poller for this platform, or nullptr when it cannot be created.
std::unique_ptr<Poller> make_poller();
// The Poller::wait() timeout until `deadline`: -1 for time_point::max(),
// otherwise whole milliseconds, rounded up and capped at INT_MAX.
int poll_timeout_until(std::chrono::steady_clock::time_point deadline,
std::chrono::steady_clock::time_point now);
class BufferStream final : public Stream {
public:
BufferStream() = default;
@@ -7383,7 +7379,7 @@ inline void drain_and_close_socket(socket_t sock,
const auto n = read_socket(sock, buf, sizeof(buf), CPPHTTPLIB_RECV_FLAGS);
if (n <= 0) { break; }
total += static_cast<size_t>(n);
} while (total < size_t(1024u * 1024u) && usec_left() > 0); // bound #2
} while (usec_left() > 0 && total < size_t(1024u * 1024u)); // bound #2
shutdown_socket(sock);
close_socket(sock);
@@ -8266,18 +8262,6 @@ inline std::unique_ptr<Poller> make_poller() {
return PollPoller::create();
}
inline int poll_timeout_until(std::chrono::steady_clock::time_point deadline,
std::chrono::steady_clock::time_point now) {
using namespace std::chrono;
if (deadline == (steady_clock::time_point::max)()) { return -1; }
if (deadline <= now) { return 0; }
const auto left = deadline - now;
auto msec = duration_cast<milliseconds>(left);
if (msec < left) { msec += milliseconds(1); }
const auto cap = milliseconds((std::numeric_limits<int>::max)());
return static_cast<int>((std::min)(msec, cap).count());
}
inline bool bind_ip_address(socket_t sock, const std::string &host) {
struct addrinfo hints;
struct addrinfo *result;
@@ -16592,10 +16576,16 @@ inline bool Server::parking_loop(TaskQueue &task_queue) {
poller.arm(listen_sock, listen_token) ? never : now + check_interval;
}
auto until =
(std::min)({st.next_deadline(), listen_rearm_at, next_idle,
st.backlog.empty() ? never : now + check_interval});
poller.wait(detail::poll_timeout_until(until, now), ready);
// At most a minute, which keeps the timeout below within an int.
auto until = (std::min)({st.next_deadline(), listen_rearm_at, next_idle,
st.backlog.empty() ? never : now + check_interval,
now + std::chrono::seconds(60)});
// Rounded up, so that the wait does not end just short of the deadline.
auto timeout = std::chrono::duration_cast<std::chrono::milliseconds>(
until - now + std::chrono::microseconds(999));
poller.wait(static_cast<int>((std::max)(timeout.count(),
std::chrono::milliseconds::rep(0))),
ready);
if (svr_sock_ != listen_sock) { break; }
now = Clock::now();
-28
View File
@@ -34049,20 +34049,6 @@ TEST_P(PollerTest, WaitHonoursItsTimeout) {
EXPECT_LT(elapsed, std::chrono::seconds(1));
}
TEST_P(PollerTest, NegativeTimeoutWaitsUntilReady) {
TestSocketPair sp;
ASSERT_TRUE(sp.ok());
ASSERT_TRUE(poller_->arm(sp.reader(), 3));
std::thread writer([&] {
std::this_thread::sleep_for(std::chrono::milliseconds(20));
sp.make_readable();
});
auto ready = wait_for(-1);
writer.join();
EXPECT_EQ(Tokens{3}, ready);
}
TEST_P(PollerTest, OnlyTheReadyDescriptorsAreDelivered) {
std::vector<std::unique_ptr<TestSocketPair>> sps;
for (detail::Poller::Token i = 0; i < 8; i++) {
@@ -34181,20 +34167,6 @@ TEST(PollerFactoryTest, MakePollerWorks) {
EXPECT_TRUE(poller->wait(1000, ready));
}
TEST(PollTimeoutTest, LastsUntilTheDeadline) {
using namespace std::chrono;
const auto now = steady_clock::now();
auto timeout = [&](steady_clock::time_point deadline) {
return detail::poll_timeout_until(deadline, now);
};
EXPECT_EQ(-1, timeout((steady_clock::time_point::max)()));
EXPECT_EQ(0, timeout(now - milliseconds(5)));
EXPECT_EQ(1, timeout(now + nanoseconds(1)));
EXPECT_EQ(1, timeout(now + milliseconds(1)));
EXPECT_EQ(2, timeout(now + microseconds(1001)));
EXPECT_EQ((std::numeric_limits<int>::max)(), timeout(now + hours(1000)));
}
// ---------------------------------------------------------------------------
// Keep-alive parking
// ---------------------------------------------------------------------------