diff --git a/README.md b/README.md index 8abfdc3d..ccce594f 100644 --- a/README.md +++ b/README.md @@ -931,7 +931,7 @@ With it: - Consecutive requests on a connection may run on different threads. - When the [max queued requests](#max-queued-requests) limit is reached, only a connection that has not been served yet (or, with `SSLServer`, has not finished its TLS handshake) is closed. An established one waits for room. -- Each request pays for handing the connection over, so with no more connections than threads and very light handlers, it can serve fewer requests per second. +- Handing the connection over to a worker adds roughly 10-20 µs to each request. It only shows when the client is as near as loopback and the handlers are very light: in our measurements, 2 connections on 2 threads served about a third fewer requests per second over loopback, but only a few percent fewer with a 250 µs round trip. - A handler, a content provider and a WebSocket still hold their thread while they run. - A subclass that overrides the private `process_and_close_socket()` is bypassed. - `SSLServer` runs the TLS handshake on a worker when the client starts it. Until then the connection waits in the event loop and is closed at the keep-alive timeout, not the read timeout, and `ssl_last_error()` is not set for it. diff --git a/docs-src/pages/en/cookbook/s20-keep-alive.md b/docs-src/pages/en/cookbook/s20-keep-alive.md index 05dd43fb..95aff5dc 100644 --- a/docs-src/pages/en/cookbook/s20-keep-alive.md +++ b/docs-src/pages/en/cookbook/s20-keep-alive.md @@ -34,7 +34,7 @@ svr.set_keep_alive_timeout(10s); ## Tuning ideas **Too many idle connections eating resources** -Shorten the timeout so idle connections drop and release their worker threads. +Shorten the timeout so idle connections drop and release their worker threads. If you want to keep them open, turn on keep-alive parking instead (see below). ```cpp svr.set_keep_alive_timeout(2s); @@ -52,6 +52,22 @@ Set `set_keep_alive_max_count(1)` and every request gets its own connection. Mos ## Relationship with the thread pool -A Keep-Alive connection holds a worker thread for its entire lifetime. The pool adds threads under load, up to its maximum thread count (4× the base count by default). If open connections outnumber that maximum, new connections wait in the queue until one closes. For thread counts, see [S21. Configure the thread pool](../s21-thread-pool). +By default, a Keep-Alive connection holds a worker thread for its entire lifetime. The pool adds threads under load, up to its maximum thread count (4× the base count by default). If open connections outnumber that maximum, new connections wait in the queue until one closes. For thread counts, see [S21. Configure the thread pool](../s21-thread-pool). + +## Keep idle connections off the threads + +```cpp +svr.set_keep_alive_parking(true); // call it before listen() +``` + +With keep-alive parking, a connection waiting for its next request is handed to an event loop instead of keeping its worker thread, and goes back to a worker when the request arrives. An idle connection then costs a socket and some memory (a few kilobytes, more with TLS) instead of a thread, and connections that outnumber the threads take turns instead of waiting for one to close. `SSLServer` supports it too. + +What changes with it: + +- Consecutive requests on one connection may run on different threads, so don't keep per-connection state in `thread_local` variables. +- Handing the connection over to a worker adds roughly 10-20 µs to each request. It only shows with very light handlers and a client as near as loopback. +- A handler, a content provider and a WebSocket still hold their thread while they run. + +The full list is in the [README](https://github.com/yhirose/cpp-httplib/blob/master/README.md#keep-alive-parking). > **Note:** For the client side, see [C14. Understand connection reuse and Keep-Alive behavior](../c14-keep-alive). Even when the server closes the connection on timeout, the client reconnects automatically. diff --git a/docs-src/pages/ja/cookbook/s20-keep-alive.md b/docs-src/pages/ja/cookbook/s20-keep-alive.md index c19f958b..4cedd4f4 100644 --- a/docs-src/pages/ja/cookbook/s20-keep-alive.md +++ b/docs-src/pages/ja/cookbook/s20-keep-alive.md @@ -34,7 +34,7 @@ svr.set_keep_alive_timeout(10s); ## チューニングの目安 **アイドル接続が多くてリソースを食う** -タイムアウトを短めに設定すると、遊んでいる接続がすぐに切れてスレッドが解放されます。 +タイムアウトを短めに設定すると、遊んでいる接続がすぐに切れてスレッドが解放されます。接続は残したままにしたい場合は、代わりにKeep-Aliveのパーキング(後述)を有効にします。 ```cpp svr.set_keep_alive_timeout(2s); @@ -52,6 +52,22 @@ svr.set_keep_alive_max_count(1000); ## スレッドプールとの関係 -Keep-Aliveでつながりっぱなしの接続は、その間ずっとワーカースレッドを1つ占有します。スレッドプールは負荷に応じて最大スレッド数(デフォルトはベースの4倍)までスレッドを増やしますが、開いている接続数がそれを超えると、新しい接続はどれかが閉じるまでキューで待たされます。スレッド数の調整は[S21. マルチスレッド数を設定する](../s21-thread-pool)を参照してください。 +デフォルトでは、Keep-Aliveでつながりっぱなしの接続は、その間ずっとワーカースレッドを1つ占有します。スレッドプールは負荷に応じて最大スレッド数(デフォルトはベースの4倍)までスレッドを増やしますが、開いている接続数がそれを超えると、新しい接続はどれかが閉じるまでキューで待たされます。スレッド数の調整は[S21. マルチスレッド数を設定する](../s21-thread-pool)を参照してください。 + +## アイドル接続にスレッドを持たせない + +```cpp +svr.set_keep_alive_parking(true); // listen()の前に呼ぶ +``` + +Keep-Aliveのパーキングを有効にすると、次のリクエストを待っている接続はワーカースレッドを手放してイベントループに預けられ、リクエストが届くとまたワーカーに戻ります。アイドル接続のコストはスレッド1つではなく、ソケットと少しのメモリ(数KB、TLSではもう少し多い)になり、スレッド数より多い接続も、どれかが閉じるのを待たずに順番に処理されます。`SSLServer`でも使えます。 + +有効にすると変わること: + +- 同じ接続の連続するリクエストが、別々のスレッドで処理されることがあります。接続ごとの状態を`thread_local`変数に持たせないでください。 +- ワーカーへの接続の受け渡しで、1リクエストあたり10〜20µsほど時間が増えます。目立つのは、ハンドラがとても軽く、クライアントがループバック並みに近いときだけです。 +- ハンドラ、コンテンツプロバイダ、WebSocketは、実行中は今までどおりスレッドを占有します。 + +すべての変更点は[README](https://github.com/yhirose/cpp-httplib/blob/master/README.md#keep-alive-parking)にあります。 > **Note:** クライアント側の挙動は[C14. 接続の再利用とKeep-Aliveの挙動を理解する](../c14-keep-alive)を参照してください。サーバーがタイムアウトで接続を切っても、クライアントは自動で再接続します。