Compare commits

...
Author SHA1 Message Date
yhirose 907ac7c546 Retry a parking backlog on a timer only, and tidy up its tests
The wake a worker sent when it started a job only shortened the wait of a
backlog from at most 10 ms, and needed an atomic flag and a second enqueue
try to be safe. The 10 ms retry, which works with any TaskQueue, now does
it alone, and one submit() decides who may enter the queue. The listen
retries use the same interval, and the park failure is logged in one
place. The zero-timeout test sends its request before the server accepts,
which removes its race with the 10 ms floor.
2026-11-21 14:57:24 -05:00
yhirose dac2f48003 Make two keep-alive parking tests hold up on busy CI runners
The backlog test queued two jobs before a worker could take the first,
which a queue with room for one turns away. The zero-timeout test now
tries a few connections, since one can miss the 10 ms floor on a busy
machine.
2026-11-21 13:58:31 -05:00
yhirose e1bc9c0140 Fix keep-alive parking with a zero timeout and a waiting backlog
- A keep-alive timeout of 0 closed every new connection before its first
  request. The wait now has the same 10 ms floor as keep_alive().
- An established connection in the backlog could wait for an unrelated
  event although the queue had room. A worker woke the loop when it
  finished a job, before ThreadPool made room by taking the next one, and
  the loop announced the backlog only after the failed try. A worker now
  wakes the loop when it starts a job, the loop announces the backlog
  before a second try, and retries it every 10 ms while it is not empty,
  for task queues that make room later.
- A new connection could take the room the backlog waited for. It is now
  turned away while the backlog is not empty.
- A failed arm() of the listening socket stopped accepting for good, and
  one of a connection left it unwatched. The first is retried after 10 ms,
  the second closes the connection.
2026-11-21 13:46:34 -05:00
yhirose 114e3d0e46 Free OpenSSL's buffers only where CVE-2024-4741 is fixed
Before 3.0.14, 3.1.6, 3.2.2 and 3.3.1, SSL_free_buffers() could free the
read buffer while a record whose header had been read still waited for its
body. A client can bring that about before its connection is parked, since
writing a response peeks at the socket. Check the OpenSSL that runs, not
the one built against.
2026-11-21 13:46:34 -05:00
yhirose 7e32283df3 Free OpenSSL's record buffers of a parked TLS connection
An idle parked TLS connection took about 24 KB, most of it OpenSSL's read
and write buffers. SSL_free_buffers() before parking brings it to about
15 KB; OpenSSL keeps them when a record is pending and makes new ones on
the next read. The request rate is unchanged. BoringSSL, LibreSSL, Mbed TLS
and wolfSSL are left as they are.
2026-11-21 11:47:48 -05:00
yhirose 76e1a9c4c9 Say what handing a parked connection over costs 2026-11-21 11:18:32 -05:00
yhirose 0289c0f528 Describe keep-alive parking in the Keep-Alive cookbook page 2026-11-21 10:27:07 -05:00
yhirose b99ccee3db Support keep-alive parking in SSLServer
SSLServer::TlsConnection plugs the TLS session into the parked connection.
A worker runs the handshake when the event loop sees the ClientHello, so a
connection that has not started it, or has gone quiet after it, waits in
the event loop instead of holding a worker. The handshake counts as having
read what the loop saw, and a connection that has finished it waits for
room like a served one when the queue is full. close_notify sent from the
event loop does not block.

SSLServer::start_session() holds the session setup and handshake that both
the blocking path and TlsConnection use, and supports_keep_alive_parking()
goes away.
2026-11-21 10:19:26 -05:00
yhirose d730627815 Give a parked connection transport hooks
KeepAliveConnection becomes a base that parking_loop() and
resume_connection() drive without knowing the transport. PlainConnection
holds the SocketStream, and Server::new_keep_alive_connection() makes one
for each accepted socket, so that SSLServer can add a TLS connection next.

A connection now counts as established for a full queue once a worker has
opened it, which for plain HTTP is when it has been served, as before.
Closing happens explicitly in the event loop and on the worker; the
destructor is only a net. No change in behaviour.
2026-11-21 08:30:47 -05:00
yhirose 2a6854c935 Write epoll_event without struct so that clang-format versions agree
clang-format 14 puts a space before {} after 'struct epoll_event ev',
and the newer one in CI's style check does not.
2026-11-21 00:41:51 -05:00
yhirose ddaeb5d83d Add a kqueue poller for keep-alive parking on macOS and FreeBSD
KqueuePoller keeps the Poller contract with EV_ONESHOT, which deletes an
event once it is delivered, so arm() adds it each time. wake() uses the
socket pair that PollPoller uses, since EVFILT_USER is not on every BSD.
make_poller() picks it on macOS and FreeBSD and falls back to poll()
elsewhere.

The contract tests now run on it as well.
2026-11-21 00:26:30 -05:00
yhirose f92079e633 Add an epoll poller for keep-alive parking on Linux
EpollPoller keeps the Poller contract with EPOLLONESHOT: a delivered
descriptor stays registered but disabled, so arm() modifies it, and adds
it when it was closed in between. wake() writes to an eventfd that stays
armed. make_poller() picks it on Linux and falls back to poll().

poll() goes over every parked connection on each wait, and epoll only
over the ready ones. With 10000 idle keep-alive connections parked, 10
active clients got 95k requests per second instead of 2.4k, at 0.1 ms
instead of 4 ms each. With few connections nothing changed.

The contract tests now run on both backends.
2026-11-21 00:25:50 -05:00
yhirose 7b6af186af Free the read buffer of a parked connection
SocketStream now allocates its 4KB read buffer on the first read that
needs it, and a connection gives the buffer back when it is parked. An
idle connection that has sent nothing never allocates one. With 10000
idle keep-alive connections parked, the server's RSS went from 47MB to
8MB; requests per second did not change.
2026-11-21 00:01:05 -05:00
yhirose 58bd9a1900 Add Server::set_keep_alive_parking()
A kept-alive connection holds a worker thread while it waits for its
next request, so connections beyond the thread count wait for one to be
closed, and with a pool smaller than the number of clients some clients
get nothing until the keep-alive timeout or the max count ends another
connection.

With keep-alive parking, listen() runs an event loop on the detail::Poller
instead of the blocking accept loop. A connection waiting for its next
request is parked there, and goes back to a worker through the TaskQueue
when the request arrives. The handlers, content providers and WebSocket
keep their blocking API and still hold a thread while they run.

It is opt-in and off by default. With it on:
- consecutive requests on a connection may run on different threads;
- when the queue is full, a connection that has not been served yet is
  closed as before, but an established one waits in a backlog instead of
  losing its request;
- the loop owns the listening socket while it watches it, and stop()
  hands it over instead of closing it under the poller;
- SSLServer ignores the setting for now, and an override of
  process_and_close_socket() is not called.

On two CPUs with two workers, 50 keep-alive clients of a 2.7 ms handler
were all served evenly (the slowest response took 70 ms instead of 5 s
with 10 clients getting nothing), light requests from 50 clients went
from 82k to 248k per second, and 10 clients were served after 10000
idle connections that had stopped the old loop. With no more clients
than workers, handing the connection over costs about half of the
requests per second of a very light handler.

The old accept loop moved unchanged into accept_loop().
process_server_socket_core() now takes the stream, and
drain_and_close_socket() takes how long to wait for the peer.
2026-11-20 23:39:33 -05:00
yhirose 1b914e1597 Fix a lost wake() in PollPoller
wait() cleared the pending flag before reading the wake byte. A wake()
landing in between sent a second byte, the read took both, and the flag
stayed set with nothing left to read, so every later wake() was skipped
and the waiting thread slept on. Clear the flag after the read instead,
as libuv does for uv_async_send(): a wake() that lands in between finds
the flag still set and sends nothing, which loses nothing because the
caller looks at its shared state after wait() returns.
2026-11-20 23:39:33 -05:00
yhirose c51417f458 Pass the keep-alive wait and the served count into the server loop
process_server_socket_core() now takes the function that waits for the
next request and the number of requests served so far, instead of
calling keep_alive() and counting on its own stack. Both callers pass
keep_alive() and a fresh counter, so nothing changes yet; this lets a
caller stop waiting and resume the same connection later without losing
track of keep_alive_max_count.
2026-11-20 23:39:32 -05:00
yhirose a69d6a0a35 Add a one-shot Poller and create_socket_pair()
Groundwork for an opt-in reactor that parks idle keep-alive connections
instead of blocking a worker thread on each of them.

detail::Poller delivers readiness once per arm(), so a descriptor is
either armed (owned by the poller) or owned by whoever got the event.
The same contract maps onto poll(), epoll, kqueue and Windows' AFD_POLL,
so a backend can be added without touching its users. wake() ends a
wait() from any thread; an armed descriptor must be disarmed before it
is closed, which the poll() backend asserts.

detail::PollPoller is the portable backend (poll() / WSAPoll()), and
detail::create_socket_pair() emulates socketpair() on Windows with
sockets that child processes do not inherit.

Nothing uses them yet.
2026-11-20 23:39:32 -05:00
yhirose 726e4583ef Accept an uppercase URL scheme
parse_url rejected any scheme containing an uppercase letter, so
Client("HTTPS://..."), WebSocketClient("WSS://...") and a redirect to
"Location: HTTPS://..." failed, although the scheme is case-insensitive
(RFC 3986 3.1). Lowercase the scheme in parse_url so that every caller
compares one canonical form and selects TLS for it.
2026-11-20 17:47:57 -05:00
yhirose a6ab971a3f Verify WebSocketClient certificates with CryptoAPI on Windows
SSLClient verifies the server certificate chain with CryptoAPI on
Windows by default, while WebSocketClient always left it to the TLS
backend, whose snapshot of the Windows store lacks the roots Windows
fetches on demand. A wss:// connection could fail against a host that
SSLClient connects to.

WebSocketClient now makes the same decision through a shared helper,
and gets enable_windows_certificate_verification() to turn it off.

Fix #2685
2026-11-20 12:36:14 -05:00
yhirose 9458c27c7c WebSocket: a read timeout can land inside a message; TCP_NODELAY by default
A read timeout set with set_read_timeout() was reported only on a frame
boundary. One that elapsed inside a frame, or between the fragments of a
message, failed the read and closed the connection, as what had been read
could not be resumed. A small frame written in pieces often arrives in
pieces on Linux, so a server polling with a short timeout lost the
connection on its first message.

A frame is now read by ws::impl::FrameReader, which keeps what it has read
across a timeout, and WebSocket keeps the message being assembled, so
read() returns Timeout wherever the timeout lands and goes on from there at
the next call. close() waits for the peer's Close through the same reader.
read_websocket_frame() keeps its one-shot meaning.

write_websocket_frame() writes the header with the payload's first chunk
in one write, and WebSocket connections set TCP_NODELAY unless
CPPHTTPLIB_WEBSOCKET_TCP_NODELAY is false: on Linux, small messages each
way otherwise waited on Nagle's algorithm for the peer's delayed ACK, a
few in fifty around 30-40 ms late.
2026-11-20 08:18:40 -05:00
metsw24-max e7a004f509 Refuse a bare host:port Location in ClientImpl::redirect (#2684)
ClientImpl::redirect() resolved a Location with resolve_relative_location(), which hands a value with a ':' in its first segment back unchanged as having a scheme, and then parsed it with parse_url(), which also accepts a bare host[:port] so that Client("localhost:8080") works. A Location such as "internal:8080" or "127.0.0.1:6379" therefore came out with an empty scheme and a host, slipped past the http/https check, and was followed as a cross-host redirect to that host and port. A value such as "localhost:8080/x" was sent to the same host as a path without a leading '/'.

Follow only http/https URLs and references starting with '/', and refuse anything else, as an unsupported scheme already was. Network-path references such as "//host:port/x" are still followed.
2026-11-19 16:32:58 -05:00
yhirose 3544f47edd Shorten the comment on the close() check in WebSocket::read() 2026-11-18 12:39:02 -05:00
8 changed files with 2528 additions and 394 deletions

No files matched your search

+6 -2
View File
@@ -66,7 +66,7 @@ enum ReadResult : int {
Returned by `read()`. Since `Fail` is `0`, the result works naturally in boolean contexts — `while (ws.read(msg))` continues until the connection closes. When you need to distinguish text from binary, check the return value directly. Returned by `read()`. Since `Fail` is `0`, the result works naturally in boolean contexts — `while (ws.read(msg))` continues until the connection closes. When you need to distinguish text from binary, check the return value directly.
`Timeout` is only returned for a read timeout you set yourself with `set_read_timeout()`. It means the timeout elapsed on a message boundary: nothing was consumed and the connection is still open, so you can send on it and read again. The compile-time defaults (`CPPHTTPLIB_WEBSOCKET_SERVER_READ_TIMEOUT_SECOND`, 300 seconds on the server; a client waits forever) are a backstop against a peer that has gone quiet, not a request for control: when one of them elapses, `read()` returns `Fail` and closes the connection, so code that never calls `set_read_timeout()` can keep using `while (ws.read(msg))`. `Timeout` is only returned for a read timeout you set yourself with `set_read_timeout()`. It means the timeout elapsed and the connection is still open, so you can send on it and read again. It can land anywhere, inside a message too: what was read of the message is kept, and the next `read()` goes on from there. The compile-time defaults (`CPPHTTPLIB_WEBSOCKET_SERVER_READ_TIMEOUT_SECOND`, 300 seconds on the server; a client waits forever) are a backstop against a peer that has gone quiet, not a request for control: when one of them elapses, `read()` returns `Fail` and closes the connection, so code that never calls `set_read_timeout()` can keep using `while (ws.read(msg))`.
**`msg` is left untouched on `Timeout`.** Because `Timeout` is non-zero, `while (ws.read(msg))` keeps looping — with the *previous* message still in `msg`. Once you set a read timeout, test the result instead: **`msg` is left untouched on `Timeout`.** Because `Timeout` is non-zero, `while (ws.read(msg))` keeps looping — with the *previous* message still in `msg`. Once you set a read timeout, test the result instead:
@@ -231,7 +231,7 @@ void set_connection_timeout(const std::chrono::duration<Rep, Period> &duration);
void set_websocket_ping_interval(time_t sec); void set_websocket_ping_interval(time_t sec);
void set_websocket_max_missed_pongs(int count); void set_websocket_max_missed_pongs(int count);
// Socket - the same setters Client has // Socket - the same setters Client has (TCP_NODELAY is on by default here)
void set_tcp_nodelay(bool on); void set_tcp_nodelay(bool on);
void set_address_family(int family); void set_address_family(int family);
void set_ipv6_v6only(bool on); void set_ipv6_v6only(bool on);
@@ -252,6 +252,9 @@ void enable_system_ca(bool enabled);
void set_server_certificate_verifier(tls::VerifyCallback verifier); void set_server_certificate_verifier(tls::VerifyCallback verifier);
void set_session_verifier( void set_session_verifier(
std::function<SSLVerifierResponse(tls::session_t)> verifier); std::function<SSLVerifierResponse(tls::session_t)> verifier);
// Windows only, as on SSLClient (see "Platform-specific Certificate Handling"
// in README.md)
void enable_windows_certificate_verification(bool enabled);
``` ```
## Examples ## Examples
@@ -511,6 +514,7 @@ if (ws.connect()) {
| `CPPHTTPLIB_WEBSOCKET_CLOSE_TIMEOUT_SECOND` | `5` | Timeout for waiting peer's Close response (seconds) | | `CPPHTTPLIB_WEBSOCKET_CLOSE_TIMEOUT_SECOND` | `5` | Timeout for waiting peer's Close response (seconds) |
| `CPPHTTPLIB_WEBSOCKET_PING_INTERVAL_SECOND` | `30` | Automatic Ping interval for heartbeat (seconds) | | `CPPHTTPLIB_WEBSOCKET_PING_INTERVAL_SECOND` | `30` | Automatic Ping interval for heartbeat (seconds) |
| `CPPHTTPLIB_WEBSOCKET_MAX_MISSED_PONGS` | `0` (disabled) | Close the connection after N consecutive unacked pings | | `CPPHTTPLIB_WEBSOCKET_MAX_MISSED_PONGS` | `0` (disabled) | Close the connection after N consecutive unacked pings |
| `CPPHTTPLIB_WEBSOCKET_TCP_NODELAY` | `true` | Set TCP_NODELAY on WebSocket connections, so small messages do not wait on Nagle's algorithm |
### Runtime Ping Interval ### Runtime Ping Interval
+19 -2
View File
@@ -252,7 +252,7 @@ On Windows, the CryptoAPI check can also be turned off at runtime. The TLS backe
cli.enable_windows_certificate_verification(false); cli.enable_windows_certificate_verification(false);
``` ```
`WebSocketClient` does not use CryptoAPI; the TLS backend always verifies its chain. `WebSocketClient` verifies the same way and has the same `enable_windows_certificate_verification()`.
> [!NOTE] > [!NOTE]
> When using SSL, it seems impossible to avoid SIGPIPE in all cases, since on some operating systems, SIGPIPE can only be suppressed on a per-message basis, but there is no way to make the OpenSSL library do so for its internal communications. If your program needs to avoid being terminated on SIGPIPE, the only fully general way might be to set up a signal handler for SIGPIPE to handle or ignore it yourself. > When using SSL, it seems impossible to avoid SIGPIPE in all cases, since on some operating systems, SIGPIPE can only be suppressed on a per-message basis, but there is no way to make the OpenSSL library do so for its internal communications. If your program needs to avoid being terminated on SIGPIPE, the only fully general way might be to set up a signal handler for SIGPIPE to handle or ignore it yourself.
@@ -919,6 +919,23 @@ svr.set_keep_alive_max_count(2); // Default is 100, 0 for no limit
svr.set_keep_alive_timeout(10); // Default is 5 svr.set_keep_alive_timeout(10); // Default is 5
``` ```
#### Keep-alive parking
```cpp
svr.set_keep_alive_parking(true); // Call it before listen()
```
By default, a kept-alive connection holds a worker thread while it waits for its next request, so connections beyond the thread count wait for one to be closed. With keep-alive parking, a connection waiting for its next request is handed to an event loop instead, and goes back to a worker when the request arrives. An idle connection then costs a socket and a few kilobytes instead of a thread, and connections that outnumber the threads take turns.
With it:
- Consecutive requests on a connection may run on different threads.
- When the [max queued requests](#max-queued-requests) limit is reached, only a connection that has not been served yet (or, with `SSLServer`, has not finished its TLS handshake) is closed. An established one waits for room.
- Handing the connection over to a worker adds roughly 10-20 µs to each request. It only shows when the client is as near as loopback and the handlers are very light: in our measurements, 2 connections on 2 threads served about a third fewer requests per second over loopback, but only a few percent fewer with a 250 µs round trip.
- A handler, a content provider and a WebSocket still hold their thread while they run.
- A subclass that overrides the private `process_and_close_socket()` is bypassed.
- `SSLServer` runs the TLS handshake on a worker when the client starts it. Until then the connection waits in the event loop and is closed at the keep-alive timeout, not the read timeout, and `ssl_last_error()` is not set for it.
### Client address behind a reverse proxy ### Client address behind a reverse proxy
```cpp ```cpp
@@ -962,7 +979,7 @@ Please see [Server example](https://github.com/yhirose/cpp-httplib/blob/master/e
When a new task arrives and the queued tasks outnumber the idle threads, a temporary thread is spawned (up to the maximum). A dynamic thread exits automatically once it has waited for the idle timeout without receiving a task. The idle timeout defaults to 3 seconds, configurable via `CPPHTTPLIB_THREAD_POOL_IDLE_TIMEOUT`. When a new task arrives and the queued tasks outnumber the idle threads, a temporary thread is spawned (up to the maximum). A dynamic thread exits automatically once it has waited for the idle timeout without receiving a task. The idle timeout defaults to 3 seconds, configurable via `CPPHTTPLIB_THREAD_POOL_IDLE_TIMEOUT`.
A connection holds its thread while it is served: between requests up to the keep-alive timeout, through a request body sent slowly but within the read timeout, and for the whole life of a WebSocket. Enough of them leave no thread for anyone else, so in front of untrusted clients, put a reverse proxy that limits connections per client, and reject unwanted WebSocket upgrades in the pre-request handler. A connection holds its thread while it is served: between requests up to the keep-alive timeout (unless [keep-alive parking](#keep-alive-parking) is on), through a request body sent slowly but within the read timeout, and for the whole life of a WebSocket. Enough of them leave no thread for anyone else, so in front of untrusted clients, put a reverse proxy that limits connections per client, and reject unwanted WebSocket upgrades in the pre-request handler.
If you want to set the thread counts at runtime: If you want to set the thread counts at runtime:
+18 -2
View File
@@ -34,7 +34,7 @@ svr.set_keep_alive_timeout(10s);
## Tuning ideas ## Tuning ideas
**Too many idle connections eating resources** **Too many idle connections eating resources**
Shorten the timeout so idle connections drop and release their worker threads. Shorten the timeout so idle connections drop and release their worker threads. If you want to keep them open, turn on keep-alive parking instead (see below).
```cpp ```cpp
svr.set_keep_alive_timeout(2s); svr.set_keep_alive_timeout(2s);
@@ -52,6 +52,22 @@ Set `set_keep_alive_max_count(1)` and every request gets its own connection. Mos
## Relationship with the thread pool ## Relationship with the thread pool
A Keep-Alive connection holds a worker thread for its entire lifetime. The pool adds threads under load, up to its maximum thread count (4× the base count by default). If open connections outnumber that maximum, new connections wait in the queue until one closes. For thread counts, see [S21. Configure the thread pool](../s21-thread-pool). By default, a Keep-Alive connection holds a worker thread for its entire lifetime. The pool adds threads under load, up to its maximum thread count (4× the base count by default). If open connections outnumber that maximum, new connections wait in the queue until one closes. For thread counts, see [S21. Configure the thread pool](../s21-thread-pool).
## Keep idle connections off the threads
```cpp
svr.set_keep_alive_parking(true); // call it before listen()
```
With keep-alive parking, a connection waiting for its next request is handed to an event loop instead of keeping its worker thread, and goes back to a worker when the request arrives. An idle connection then costs a socket and some memory (a few kilobytes, more with TLS) instead of a thread, and connections that outnumber the threads take turns instead of waiting for one to close. `SSLServer` supports it too.
What changes with it:
- Consecutive requests on one connection may run on different threads, so don't keep per-connection state in `thread_local` variables.
- Handing the connection over to a worker adds roughly 10-20 µs to each request. It only shows with very light handlers and a client as near as loopback.
- A handler, a content provider and a WebSocket still hold their thread while they run.
The full list is in the [README](https://github.com/yhirose/cpp-httplib/blob/master/README.md#keep-alive-parking).
> **Note:** For the client side, see [C14. Understand connection reuse and Keep-Alive behavior](../c14-keep-alive). Even when the server closes the connection on timeout, the client reconnects automatically. > **Note:** For the client side, see [C14. Understand connection reuse and Keep-Alive behavior](../c14-keep-alive). Even when the server closes the connection on timeout, the client reconnects automatically.
@@ -42,7 +42,7 @@ ws.set_write_timeout(10s);
## What the read timeout means ## What the read timeout means
`set_read_timeout()` applies to a single `read()` call. If no message arrives within that time, `read()` returns `ReadResult::Timeout`: **the connection is still open** and nothing was consumed, so you can send on it and read again. That is what separates it from `ReadResult::Fail`, which means the connection is gone. `set_read_timeout()` applies to a single `read()` call. If no whole message arrives within that time, `read()` returns `ReadResult::Timeout`: **the connection is still open**, and what was read of a message is kept for the next `read()`, so you can send on it and read again. That is what separates it from `ReadResult::Fail`, which means the connection is gone.
This is what lets one thread own a connection in both directions: This is what lets one thread own a connection in both directions:
@@ -67,7 +67,7 @@ Without a read timeout, `read()` blocks until a message arrives, so the thread h
Two things to know about `Timeout`: Two things to know about `Timeout`:
- It leaves `msg` untouched, and it is non-zero. So `while (ws.read(msg))` is not usable once a read timeout is set — the loop would keep running with the *previous* message still in `msg`. - It leaves `msg` untouched, and it is non-zero. So `while (ws.read(msg))` is not usable once a read timeout is set — the loop would keep running with the *previous* message still in `msg`.
- It is only reported on a message boundary. If the timeout elapses partway through a fragmented message, that message cannot be resumed and `read()` returns `Fail`. - It can land anywhere, partway through a message too. What was read of the message is kept, and the next `read()` goes on from there, so a message that arrives slowly or in fragments still comes out whole.
For connections where long idle periods are normal — waiting on notifications, for example — either leave the read timeout unset, or treat `Timeout` as the no-op it is and keep looping. For connections where long idle periods are normal — waiting on notifications, for example — either leave the read timeout unset, or treat `Timeout` as the no-op it is and keep looping.
+18 -2
View File
@@ -34,7 +34,7 @@ svr.set_keep_alive_timeout(10s);
## チューニングの目安 ## チューニングの目安
**アイドル接続が多くてリソースを食う** **アイドル接続が多くてリソースを食う**
タイムアウトを短めに設定すると、遊んでいる接続がすぐに切れてスレッドが解放されます。 タイムアウトを短めに設定すると、遊んでいる接続がすぐに切れてスレッドが解放されます。接続は残したままにしたい場合は、代わりにKeep-Aliveのパーキング(後述)を有効にします。
```cpp ```cpp
svr.set_keep_alive_timeout(2s); svr.set_keep_alive_timeout(2s);
@@ -52,6 +52,22 @@ svr.set_keep_alive_max_count(1000);
## スレッドプールとの関係 ## スレッドプールとの関係
Keep-Aliveでつながりっぱなしの接続は、その間ずっとワーカースレッドを1つ占有します。スレッドプールは負荷に応じて最大スレッド数(デフォルトはベースの4倍)までスレッドを増やしますが、開いている接続数がそれを超えると、新しい接続はどれかが閉じるまでキューで待たされます。スレッド数の調整は[S21. マルチスレッド数を設定する](../s21-thread-pool)を参照してください。 デフォルトでは、Keep-Aliveでつながりっぱなしの接続は、その間ずっとワーカースレッドを1つ占有します。スレッドプールは負荷に応じて最大スレッド数(デフォルトはベースの4倍)までスレッドを増やしますが、開いている接続数がそれを超えると、新しい接続はどれかが閉じるまでキューで待たされます。スレッド数の調整は[S21. マルチスレッド数を設定する](../s21-thread-pool)を参照してください。
## アイドル接続にスレッドを持たせない
```cpp
svr.set_keep_alive_parking(true); // listen()の前に呼ぶ
```
Keep-Aliveのパーキングを有効にすると、次のリクエストを待っている接続はワーカースレッドを手放してイベントループに預けられ、リクエストが届くとまたワーカーに戻ります。アイドル接続のコストはスレッド1つではなく、ソケットと少しのメモリ(数KB、TLSではもう少し多い)になり、スレッド数より多い接続も、どれかが閉じるのを待たずに順番に処理されます。`SSLServer`でも使えます。
有効にすると変わること:
- 同じ接続の連続するリクエストが、別々のスレッドで処理されることがあります。接続ごとの状態を`thread_local`変数に持たせないでください。
- ワーカーへの接続の受け渡しで、1リクエストあたり10〜20µsほど時間が増えます。目立つのは、ハンドラがとても軽く、クライアントがループバック並みに近いときだけです。
- ハンドラ、コンテンツプロバイダ、WebSocketは、実行中は今までどおりスレッドを占有します。
すべての変更点は[README](https://github.com/yhirose/cpp-httplib/blob/master/README.md#keep-alive-parking)にあります。
> **Note:** クライアント側の挙動は[C14. 接続の再利用とKeep-Aliveの挙動を理解する](../c14-keep-alive)を参照してください。サーバーがタイムアウトで接続を切っても、クライアントは自動で再接続します。 > **Note:** クライアント側の挙動は[C14. 接続の再利用とKeep-Aliveの挙動を理解する](../c14-keep-alive)を参照してください。サーバーがタイムアウトで接続を切っても、クライアントは自動で再接続します。
@@ -42,7 +42,7 @@ ws.set_write_timeout(10s);
## 読み取りタイムアウトの意味 ## 読み取りタイムアウトの意味
`set_read_timeout()`は「1回の`read()`呼び出し」に対するタイムアウトです。メッセージが届かないまま指定時間が経過すると`read()`は`ReadResult::Timeout`を返します。このとき**接続は開いたまま**で、1バイトも読み進めていないので、そのまま送信して読み直せます。接続が失われたことを意味する`ReadResult::Fail`とはここが違います。 `set_read_timeout()`は「1回の`read()`呼び出し」に対するタイムアウトです。メッセージがそろわないまま指定時間が経過すると`read()`は`ReadResult::Timeout`を返します。このとき**接続は開いたまま**で、途中まで読んだ分は次の`read()`に引き継がれるので、そのまま送信して読み直せます。接続が失われたことを意味する`ReadResult::Fail`とはここが違います。
1本の接続を1つのスレッドで双方向に扱えるのはこのためです。 1本の接続を1つのスレッドで双方向に扱えるのはこのためです。
@@ -67,7 +67,7 @@ while (ws.is_open()) {
`Timeout`について2点あります。 `Timeout`について2点あります。
- `msg`は書き換えられません。値も0以外なので、読み取りタイムアウトを設定した状態で`while (ws.read(msg))`と書くと、**前回のメッセージ**が`msg`に残ったままループが回り続けます。 - `msg`は書き換えられません。値も0以外なので、読み取りタイムアウトを設定した状態で`while (ws.read(msg))`と書くと、**前回のメッセージ**が`msg`に残ったままループが回り続けます。
- 報告されるのはメッセージの境界だけです。分割されたメッセージの途中でタイムアウトした場合、そのメッセージは再開できないので`read()`は`Fail`を返します。 - メッセージの途中でも報告されます。それまでに読んだ分は残り、次の`read()`はその続きから読むので、ゆっくり届くメッセージや分割されたメッセージも、そろった形で受け取れます。
通知の待受のように長時間メッセージが来ないことが正常な接続では、読み取りタイムアウトを設定しないままにするか、`Timeout`を「まだ何も来ていない」印として扱ってループを続けてください。 通知の待受のように長時間メッセージが来ないことが正常な接続では、読み取りタイムアウトを設定しないままにするか、`Timeout`を「まだ何も来ていない」印として扱ってループを続けてください。
+1387 -377
View File
File diff suppressed because it is too large. Load diff
+1076 -5
View File
File diff suppressed because it is too large. Load diff