mirror of
https://github.com/yhirose/cpp-httplib.git
synced 2026-11-21 20:01:46 +00:00
Compare commits
17
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
907ac7c546 | ||
|
|
dac2f48003 | ||
|
|
e1bc9c0140 | ||
|
|
114e3d0e46 | ||
|
|
7e32283df3 | ||
|
|
76e1a9c4c9 | ||
|
|
0289c0f528 | ||
|
|
b99ccee3db | ||
|
|
d730627815 | ||
|
|
2a6854c935 | ||
|
|
ddaeb5d83d | ||
|
|
f92079e633 | ||
|
|
7b6af186af | ||
|
|
58bd9a1900 | ||
|
|
1b914e1597 | ||
|
|
c51417f458 | ||
|
|
a69d6a0a35 |
No files matched your search
@@ -919,6 +919,23 @@ svr.set_keep_alive_max_count(2); // Default is 100, 0 for no limit
|
||||
svr.set_keep_alive_timeout(10); // Default is 5
|
||||
```
|
||||
|
||||
#### Keep-alive parking
|
||||
|
||||
```cpp
|
||||
svr.set_keep_alive_parking(true); // Call it before listen()
|
||||
```
|
||||
|
||||
By default, a kept-alive connection holds a worker thread while it waits for its next request, so connections beyond the thread count wait for one to be closed. With keep-alive parking, a connection waiting for its next request is handed to an event loop instead, and goes back to a worker when the request arrives. An idle connection then costs a socket and a few kilobytes instead of a thread, and connections that outnumber the threads take turns.
|
||||
|
||||
With it:
|
||||
|
||||
- Consecutive requests on a connection may run on different threads.
|
||||
- When the [max queued requests](#max-queued-requests) limit is reached, only a connection that has not been served yet (or, with `SSLServer`, has not finished its TLS handshake) is closed. An established one waits for room.
|
||||
- Handing the connection over to a worker adds roughly 10-20 µs to each request. It only shows when the client is as near as loopback and the handlers are very light: in our measurements, 2 connections on 2 threads served about a third fewer requests per second over loopback, but only a few percent fewer with a 250 µs round trip.
|
||||
- A handler, a content provider and a WebSocket still hold their thread while they run.
|
||||
- A subclass that overrides the private `process_and_close_socket()` is bypassed.
|
||||
- `SSLServer` runs the TLS handshake on a worker when the client starts it. Until then the connection waits in the event loop and is closed at the keep-alive timeout, not the read timeout, and `ssl_last_error()` is not set for it.
|
||||
|
||||
### Client address behind a reverse proxy
|
||||
|
||||
```cpp
|
||||
@@ -962,7 +979,7 @@ Please see [Server example](https://github.com/yhirose/cpp-httplib/blob/master/e
|
||||
|
||||
When a new task arrives and the queued tasks outnumber the idle threads, a temporary thread is spawned (up to the maximum). A dynamic thread exits automatically once it has waited for the idle timeout without receiving a task. The idle timeout defaults to 3 seconds, configurable via `CPPHTTPLIB_THREAD_POOL_IDLE_TIMEOUT`.
|
||||
|
||||
A connection holds its thread while it is served: between requests up to the keep-alive timeout, through a request body sent slowly but within the read timeout, and for the whole life of a WebSocket. Enough of them leave no thread for anyone else, so in front of untrusted clients, put a reverse proxy that limits connections per client, and reject unwanted WebSocket upgrades in the pre-request handler.
|
||||
A connection holds its thread while it is served: between requests up to the keep-alive timeout (unless [keep-alive parking](#keep-alive-parking) is on), through a request body sent slowly but within the read timeout, and for the whole life of a WebSocket. Enough of them leave no thread for anyone else, so in front of untrusted clients, put a reverse proxy that limits connections per client, and reject unwanted WebSocket upgrades in the pre-request handler.
|
||||
|
||||
If you want to set the thread counts at runtime:
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ svr.set_keep_alive_timeout(10s);
|
||||
## Tuning ideas
|
||||
|
||||
**Too many idle connections eating resources**
|
||||
Shorten the timeout so idle connections drop and release their worker threads.
|
||||
Shorten the timeout so idle connections drop and release their worker threads. If you want to keep them open, turn on keep-alive parking instead (see below).
|
||||
|
||||
```cpp
|
||||
svr.set_keep_alive_timeout(2s);
|
||||
@@ -52,6 +52,22 @@ Set `set_keep_alive_max_count(1)` and every request gets its own connection. Mos
|
||||
|
||||
## Relationship with the thread pool
|
||||
|
||||
A Keep-Alive connection holds a worker thread for its entire lifetime. The pool adds threads under load, up to its maximum thread count (4× the base count by default). If open connections outnumber that maximum, new connections wait in the queue until one closes. For thread counts, see [S21. Configure the thread pool](../s21-thread-pool).
|
||||
By default, a Keep-Alive connection holds a worker thread for its entire lifetime. The pool adds threads under load, up to its maximum thread count (4× the base count by default). If open connections outnumber that maximum, new connections wait in the queue until one closes. For thread counts, see [S21. Configure the thread pool](../s21-thread-pool).
|
||||
|
||||
## Keep idle connections off the threads
|
||||
|
||||
```cpp
|
||||
svr.set_keep_alive_parking(true); // call it before listen()
|
||||
```
|
||||
|
||||
With keep-alive parking, a connection waiting for its next request is handed to an event loop instead of keeping its worker thread, and goes back to a worker when the request arrives. An idle connection then costs a socket and some memory (a few kilobytes, more with TLS) instead of a thread, and connections that outnumber the threads take turns instead of waiting for one to close. `SSLServer` supports it too.
|
||||
|
||||
What changes with it:
|
||||
|
||||
- Consecutive requests on one connection may run on different threads, so don't keep per-connection state in `thread_local` variables.
|
||||
- Handing the connection over to a worker adds roughly 10-20 µs to each request. It only shows with very light handlers and a client as near as loopback.
|
||||
- A handler, a content provider and a WebSocket still hold their thread while they run.
|
||||
|
||||
The full list is in the [README](https://github.com/yhirose/cpp-httplib/blob/master/README.md#keep-alive-parking).
|
||||
|
||||
> **Note:** For the client side, see [C14. Understand connection reuse and Keep-Alive behavior](../c14-keep-alive). Even when the server closes the connection on timeout, the client reconnects automatically.
|
||||
@@ -34,7 +34,7 @@ svr.set_keep_alive_timeout(10s);
|
||||
## チューニングの目安
|
||||
|
||||
**アイドル接続が多くてリソースを食う**
|
||||
タイムアウトを短めに設定すると、遊んでいる接続がすぐに切れてスレッドが解放されます。
|
||||
タイムアウトを短めに設定すると、遊んでいる接続がすぐに切れてスレッドが解放されます。接続は残したままにしたい場合は、代わりにKeep-Aliveのパーキング(後述)を有効にします。
|
||||
|
||||
```cpp
|
||||
svr.set_keep_alive_timeout(2s);
|
||||
@@ -52,6 +52,22 @@ svr.set_keep_alive_max_count(1000);
|
||||
|
||||
## スレッドプールとの関係
|
||||
|
||||
Keep-Aliveでつながりっぱなしの接続は、その間ずっとワーカースレッドを1つ占有します。スレッドプールは負荷に応じて最大スレッド数(デフォルトはベースの4倍)までスレッドを増やしますが、開いている接続数がそれを超えると、新しい接続はどれかが閉じるまでキューで待たされます。スレッド数の調整は[S21. マルチスレッド数を設定する](../s21-thread-pool)を参照してください。
|
||||
デフォルトでは、Keep-Aliveでつながりっぱなしの接続は、その間ずっとワーカースレッドを1つ占有します。スレッドプールは負荷に応じて最大スレッド数(デフォルトはベースの4倍)までスレッドを増やしますが、開いている接続数がそれを超えると、新しい接続はどれかが閉じるまでキューで待たされます。スレッド数の調整は[S21. マルチスレッド数を設定する](../s21-thread-pool)を参照してください。
|
||||
|
||||
## アイドル接続にスレッドを持たせない
|
||||
|
||||
```cpp
|
||||
svr.set_keep_alive_parking(true); // listen()の前に呼ぶ
|
||||
```
|
||||
|
||||
Keep-Aliveのパーキングを有効にすると、次のリクエストを待っている接続はワーカースレッドを手放してイベントループに預けられ、リクエストが届くとまたワーカーに戻ります。アイドル接続のコストはスレッド1つではなく、ソケットと少しのメモリ(数KB、TLSではもう少し多い)になり、スレッド数より多い接続も、どれかが閉じるのを待たずに順番に処理されます。`SSLServer`でも使えます。
|
||||
|
||||
有効にすると変わること:
|
||||
|
||||
- 同じ接続の連続するリクエストが、別々のスレッドで処理されることがあります。接続ごとの状態を`thread_local`変数に持たせないでください。
|
||||
- ワーカーへの接続の受け渡しで、1リクエストあたり10〜20µsほど時間が増えます。目立つのは、ハンドラがとても軽く、クライアントがループバック並みに近いときだけです。
|
||||
- ハンドラ、コンテンツプロバイダ、WebSocketは、実行中は今までどおりスレッドを占有します。
|
||||
|
||||
すべての変更点は[README](https://github.com/yhirose/cpp-httplib/blob/master/README.md#keep-alive-parking)にあります。
|
||||
|
||||
> **Note:** クライアント側の挙動は[C14. 接続の再利用とKeep-Aliveの挙動を理解する](../c14-keep-alive)を参照してください。サーバーがタイムアウトで接続を切っても、クライアントは自動で再接続します。
|
||||
+699
@@ -76,8 +76,13 @@ ctx_t create_server_context();
|
||||
void free_context(ctx_t ctx);
|
||||
bool set_server_cert_file(ctx_t ctx, const char *cert_path,
|
||||
const char *key_path, const char *password);
|
||||
ctx_t create_client_context();
|
||||
void set_verify_client(ctx_t ctx, bool require);
|
||||
session_t create_session(ctx_t ctx, socket_t sock);
|
||||
void free_session(session_t session);
|
||||
bool connect_nonblocking(session_t session, socket_t sock, time_t timeout_sec,
|
||||
time_t timeout_usec, TlsError *err,
|
||||
const std::function<bool()> &interrupted);
|
||||
bool accept_nonblocking(session_t session, socket_t sock, time_t timeout_sec,
|
||||
time_t timeout_usec, TlsError *err);
|
||||
ssize_t read(session_t session, void *buf, size_t len, TlsError &err);
|
||||
@@ -90,6 +95,11 @@ macos_ssl_trust_settings_result(CFArrayRef trust_settings);
|
||||
} // namespace impl
|
||||
#endif
|
||||
} // namespace tls
|
||||
#ifdef CPPHTTPLIB_OPENSSL_SUPPORT
|
||||
namespace detail {
|
||||
bool ssl_free_buffers_is_safe(unsigned long version);
|
||||
} // namespace detail
|
||||
#endif
|
||||
} // namespace httplib
|
||||
#endif
|
||||
|
||||
@@ -33916,3 +33926,692 @@ TEST(WebSocketProxyTest, WssTunnelsThroughConnect) {
|
||||
EXPECT_EQ(1, proxy.connect_hits());
|
||||
}
|
||||
#endif
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Poller
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
namespace poller_test {
|
||||
|
||||
struct PollerBackend {
|
||||
const char *name;
|
||||
std::unique_ptr<detail::Poller> (*create)();
|
||||
};
|
||||
|
||||
void PrintTo(const PollerBackend &backend, std::ostream *os) {
|
||||
*os << backend.name;
|
||||
}
|
||||
|
||||
// Every backend runs the same contract tests. A new one is added here only.
|
||||
std::vector<PollerBackend> poller_backends() {
|
||||
return {
|
||||
{"poll", &detail::PollPoller::create},
|
||||
#ifdef __linux__
|
||||
{"epoll", &detail::EpollPoller::create},
|
||||
#endif
|
||||
#ifdef CPPHTTPLIB_KQUEUE_POLLER
|
||||
{"kqueue", &detail::KqueuePoller::create},
|
||||
#endif
|
||||
};
|
||||
}
|
||||
|
||||
// A descriptor whose readability the test controls.
|
||||
struct TestSocketPair {
|
||||
socket_t fd[2] = {INVALID_SOCKET, INVALID_SOCKET};
|
||||
|
||||
TestSocketPair() { detail::create_socket_pair(fd); }
|
||||
~TestSocketPair() {
|
||||
for (auto s : fd) {
|
||||
if (s != INVALID_SOCKET) { detail::close_socket(s); }
|
||||
}
|
||||
}
|
||||
|
||||
bool ok() const { return fd[0] != INVALID_SOCKET; }
|
||||
socket_t reader() const { return fd[0]; }
|
||||
void make_readable() {
|
||||
const char c = 1;
|
||||
detail::send_socket(fd[1], &c, 1, 0);
|
||||
}
|
||||
void close_writer() {
|
||||
detail::close_socket(fd[1]);
|
||||
fd[1] = INVALID_SOCKET;
|
||||
}
|
||||
};
|
||||
|
||||
using Tokens = std::vector<detail::Poller::Token>;
|
||||
|
||||
class PollerTest : public ::testing::TestWithParam<PollerBackend> {
|
||||
protected:
|
||||
void SetUp() override {
|
||||
poller_ = GetParam().create();
|
||||
ASSERT_TRUE(poller_ != nullptr);
|
||||
}
|
||||
|
||||
Tokens wait_for(int timeout_ms) {
|
||||
Tokens ready;
|
||||
poller_->wait(timeout_ms, ready);
|
||||
std::sort(ready.begin(), ready.end());
|
||||
return ready;
|
||||
}
|
||||
|
||||
std::unique_ptr<detail::Poller> poller_;
|
||||
};
|
||||
|
||||
} // namespace poller_test
|
||||
|
||||
using poller_test::PollerTest;
|
||||
using poller_test::TestSocketPair;
|
||||
using poller_test::Tokens;
|
||||
|
||||
INSTANTIATE_TEST_SUITE_P(
|
||||
Backends, PollerTest, ::testing::ValuesIn(poller_test::poller_backends()),
|
||||
[](const ::testing::TestParamInfo<poller_test::PollerBackend> &p) {
|
||||
return std::string(p.param.name);
|
||||
});
|
||||
|
||||
TEST_P(PollerTest, DeliversOnceThenStaysDisarmedUntilRearmed) {
|
||||
TestSocketPair sp;
|
||||
ASSERT_TRUE(sp.ok());
|
||||
ASSERT_TRUE(poller_->arm(sp.reader(), 7));
|
||||
EXPECT_EQ(Tokens{}, wait_for(0));
|
||||
|
||||
sp.make_readable();
|
||||
EXPECT_EQ(Tokens{7}, wait_for(1000));
|
||||
EXPECT_EQ(Tokens{}, wait_for(0)); // still readable, but disarmed
|
||||
|
||||
ASSERT_TRUE(poller_->arm(sp.reader(), 8));
|
||||
EXPECT_EQ(Tokens{8}, wait_for(1000));
|
||||
}
|
||||
|
||||
TEST_P(PollerTest, DisarmedDescriptorIsNotDelivered) {
|
||||
TestSocketPair sp;
|
||||
ASSERT_TRUE(sp.ok());
|
||||
ASSERT_TRUE(poller_->arm(sp.reader(), 1));
|
||||
poller_->disarm(sp.reader());
|
||||
poller_->disarm(sp.reader()); // disarming twice is harmless
|
||||
sp.make_readable();
|
||||
EXPECT_EQ(Tokens{}, wait_for(0));
|
||||
}
|
||||
|
||||
TEST_P(PollerTest, WaitHonoursItsTimeout) {
|
||||
TestSocketPair sp;
|
||||
ASSERT_TRUE(sp.ok());
|
||||
ASSERT_TRUE(poller_->arm(sp.reader(), 1));
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
EXPECT_EQ(Tokens{}, wait_for(30));
|
||||
auto elapsed = std::chrono::steady_clock::now() - start;
|
||||
EXPECT_GE(elapsed, std::chrono::milliseconds(25));
|
||||
EXPECT_LT(elapsed, std::chrono::seconds(1));
|
||||
}
|
||||
|
||||
TEST_P(PollerTest, OnlyTheReadyDescriptorsAreDelivered) {
|
||||
std::vector<std::unique_ptr<TestSocketPair>> sps;
|
||||
for (detail::Poller::Token i = 0; i < 8; i++) {
|
||||
sps.emplace_back(new TestSocketPair());
|
||||
ASSERT_TRUE(sps.back()->ok());
|
||||
ASSERT_TRUE(poller_->arm(sps.back()->reader(), i));
|
||||
}
|
||||
|
||||
sps[0]->make_readable();
|
||||
sps[3]->make_readable();
|
||||
sps[7]->make_readable();
|
||||
EXPECT_EQ((Tokens{0, 3, 7}), wait_for(1000));
|
||||
|
||||
// The others are still armed, the delivered ones are not.
|
||||
sps[1]->make_readable();
|
||||
EXPECT_EQ(Tokens{1}, wait_for(1000));
|
||||
EXPECT_EQ(Tokens{}, wait_for(0));
|
||||
}
|
||||
|
||||
TEST_P(PollerTest, StaysConsistentAfterOtherDescriptorsWereDelivered) {
|
||||
TestSocketPair s0, s1, s2;
|
||||
ASSERT_TRUE(s0.ok() && s1.ok() && s2.ok());
|
||||
ASSERT_TRUE(poller_->arm(s0.reader(), 0));
|
||||
ASSERT_TRUE(poller_->arm(s1.reader(), 1));
|
||||
ASSERT_TRUE(poller_->arm(s2.reader(), 2));
|
||||
|
||||
// A delivery may move another entry inside the poller.
|
||||
s0.make_readable();
|
||||
EXPECT_EQ(Tokens{0}, wait_for(1000));
|
||||
|
||||
// The entries left must still be found by descriptor.
|
||||
poller_->disarm(s2.reader());
|
||||
s1.make_readable();
|
||||
s2.make_readable();
|
||||
EXPECT_EQ(Tokens{1}, wait_for(1000));
|
||||
}
|
||||
|
||||
TEST_P(PollerTest, PeerHangUpCountsAsReadable) {
|
||||
TestSocketPair sp;
|
||||
ASSERT_TRUE(sp.ok());
|
||||
ASSERT_TRUE(poller_->arm(sp.reader(), 5));
|
||||
sp.close_writer();
|
||||
EXPECT_EQ(Tokens{5}, wait_for(1000));
|
||||
|
||||
char c;
|
||||
EXPECT_EQ(0, detail::read_socket(sp.reader(), &c, 1, 0)); // EOF
|
||||
}
|
||||
|
||||
TEST_P(PollerTest, WakeEndsABlockedWait) {
|
||||
std::thread waker([&] {
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(20));
|
||||
poller_->wake();
|
||||
});
|
||||
|
||||
Tokens ready;
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
EXPECT_TRUE(poller_->wait(5000, ready)); // only the waker ends this early
|
||||
waker.join();
|
||||
EXPECT_TRUE(ready.empty());
|
||||
EXPECT_LT(std::chrono::steady_clock::now() - start, std::chrono::seconds(1));
|
||||
}
|
||||
|
||||
TEST_P(PollerTest, WakeBeforeWaitIsNotLost) {
|
||||
poller_->wake(); // e.g. stop() running before the reactor reaches wait()
|
||||
Tokens ready;
|
||||
EXPECT_TRUE(poller_->wait(1000, ready));
|
||||
}
|
||||
|
||||
TEST_P(PollerTest, WakesCoalesceAndAreConsumed) {
|
||||
for (int i = 0; i < 1000; i++) {
|
||||
poller_->wake();
|
||||
}
|
||||
Tokens ready;
|
||||
EXPECT_TRUE(poller_->wait(0, ready));
|
||||
EXPECT_FALSE(poller_->wait(0, ready));
|
||||
|
||||
poller_->wake();
|
||||
EXPECT_TRUE(poller_->wait(0, ready));
|
||||
}
|
||||
|
||||
TEST_P(PollerTest, ConcurrentWakesAreNeverLost) {
|
||||
// Two threads publish changes and wake() after each, as a reactor's workers
|
||||
// do. Every change must be seen without a wait() timing out. A lost wake()
|
||||
// needs an unlucky interleaving, hence the rounds.
|
||||
for (int round = 0; round < 5; round++) {
|
||||
SCOPED_TRACE(round);
|
||||
const int per_thread = 20000;
|
||||
std::atomic<int> posted{0};
|
||||
auto post = [&] {
|
||||
for (int i = 0; i < per_thread; i++) {
|
||||
posted++;
|
||||
poller_->wake();
|
||||
}
|
||||
};
|
||||
std::thread t1(post), t2(post);
|
||||
|
||||
auto lost = false;
|
||||
Tokens ready;
|
||||
for (auto seen = 0; seen < 2 * per_thread; seen = posted) {
|
||||
if (!poller_->wait(1000, ready)) {
|
||||
lost = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
t1.join();
|
||||
t2.join();
|
||||
ASSERT_FALSE(lost);
|
||||
}
|
||||
}
|
||||
|
||||
TEST(PollerFactoryTest, MakePollerWorks) {
|
||||
auto poller = detail::make_poller();
|
||||
ASSERT_TRUE(poller != nullptr);
|
||||
poller->wake();
|
||||
std::vector<detail::Poller::Token> ready;
|
||||
EXPECT_TRUE(poller->wait(1000, ready));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Keep-alive parking
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
namespace parking_test {
|
||||
|
||||
// Runs `svr` on a free port and stops it when it goes away. `before_listen`
|
||||
// runs once the port is bound and listening, before the server accepts.
|
||||
struct RunningServer {
|
||||
explicit RunningServer(
|
||||
Server &svr, const std::function<void(int)> &before_listen = nullptr)
|
||||
: svr_(svr) {
|
||||
port = svr_.bind_to_any_port("127.0.0.1");
|
||||
if (before_listen) { before_listen(port); }
|
||||
thread = std::thread([this] { svr_.listen_after_bind(); });
|
||||
svr_.wait_until_ready();
|
||||
}
|
||||
~RunningServer() {
|
||||
svr_.stop();
|
||||
if (thread.joinable()) { thread.join(); }
|
||||
}
|
||||
|
||||
Server &svr_;
|
||||
int port = 0;
|
||||
std::thread thread;
|
||||
};
|
||||
|
||||
struct Sockets {
|
||||
std::vector<socket_t> socks;
|
||||
~Sockets() {
|
||||
for (auto s : socks) {
|
||||
detail::close_socket(s);
|
||||
}
|
||||
}
|
||||
socket_t connect(int port) {
|
||||
auto error = Error::Success;
|
||||
auto s = detail::create_client_socket("127.0.0.1", "", port, AF_UNSPEC,
|
||||
false, false, nullptr, 2, 0, 2, 0, 2,
|
||||
0, std::string(), error);
|
||||
if (s != INVALID_SOCKET) { socks.push_back(s); }
|
||||
return s;
|
||||
}
|
||||
};
|
||||
|
||||
std::chrono::steady_clock::time_point in_ms(int ms) {
|
||||
return std::chrono::steady_clock::now() + std::chrono::milliseconds(ms);
|
||||
}
|
||||
|
||||
bool readable_by(socket_t sock,
|
||||
std::chrono::steady_clock::time_point deadline) {
|
||||
auto left = std::chrono::duration_cast<std::chrono::microseconds>(
|
||||
deadline - std::chrono::steady_clock::now())
|
||||
.count();
|
||||
return left > 0 && detail::select_read(sock, 0, left) > 0;
|
||||
}
|
||||
|
||||
bool send_get(socket_t sock, const std::string &path = "/") {
|
||||
auto req = "GET " + path + " HTTP/1.1\r\nHost: localhost\r\n\r\n";
|
||||
return detail::send_socket(sock, req.data(), req.size(), 0) ==
|
||||
static_cast<ssize_t>(req.size());
|
||||
}
|
||||
|
||||
// Reads one response whose body is "ok".
|
||||
bool receives_ok(socket_t sock, int timeout_ms) {
|
||||
auto deadline = in_ms(timeout_ms);
|
||||
std::string buf;
|
||||
while (readable_by(sock, deadline)) {
|
||||
char tmp[1024];
|
||||
auto n = detail::read_socket(sock, tmp, sizeof(tmp), 0);
|
||||
if (n <= 0) { return false; }
|
||||
buf.append(tmp, static_cast<size_t>(n));
|
||||
auto end = buf.find("\r\n\r\n");
|
||||
if (end != std::string::npos && buf.size() >= end + 4 + 2) {
|
||||
return buf.compare(0, 12, "HTTP/1.1 200") == 0 &&
|
||||
buf.compare(end + 4, 2, "ok") == 0;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
bool get_ok(socket_t sock, int timeout_ms) {
|
||||
return send_get(sock) && receives_ok(sock, timeout_ms);
|
||||
}
|
||||
|
||||
// Whether the server closes `sock` by `deadline`.
|
||||
bool sees_eof(socket_t sock, std::chrono::steady_clock::time_point deadline) {
|
||||
char c;
|
||||
return readable_by(sock, deadline) &&
|
||||
detail::read_socket(sock, &c, 1, 0) <= 0;
|
||||
}
|
||||
|
||||
bool wait_until(const std::function<bool()> &cond, int timeout_ms) {
|
||||
auto deadline = in_ms(timeout_ms);
|
||||
while (!cond()) {
|
||||
if (std::chrono::steady_clock::now() > deadline) { return false; }
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(1));
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
void serve_ok(Server &svr, std::function<TaskQueue *(void)> new_task_queue) {
|
||||
svr.new_task_queue = std::move(new_task_queue);
|
||||
svr.set_keep_alive_parking(true);
|
||||
svr.Get("/", [](const Request &, Response &res) {
|
||||
res.set_content("ok", "text/plain");
|
||||
});
|
||||
}
|
||||
|
||||
std::function<TaskQueue *(void)> threads(size_t n) {
|
||||
return [n] { return new ThreadPool(n); };
|
||||
}
|
||||
|
||||
// Counts what the event loop managed to queue, so that a test can wait for it.
|
||||
class CountingTaskQueue final : public TaskQueue {
|
||||
public:
|
||||
CountingTaskQueue(size_t threads, size_t max_queued, std::atomic<int> &ok,
|
||||
std::atomic<int> &full)
|
||||
: pool_(threads, threads, max_queued), ok_(ok), full_(full) {}
|
||||
|
||||
bool enqueue(std::function<void()> fn) override {
|
||||
auto queued = pool_.enqueue([this, fn]() {
|
||||
fn();
|
||||
// Holds the worker back from taking the next job.
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(pause_after_ms));
|
||||
});
|
||||
(queued ? ok_ : full_)++;
|
||||
return queued;
|
||||
}
|
||||
void shutdown() override { pool_.shutdown(); }
|
||||
|
||||
std::atomic<int> pause_after_ms{0};
|
||||
|
||||
private:
|
||||
ThreadPool pool_;
|
||||
std::atomic<int> &ok_;
|
||||
std::atomic<int> &full_;
|
||||
};
|
||||
|
||||
// Holds handlers until it opens, or for 3 seconds at most.
|
||||
class Gate {
|
||||
public:
|
||||
void wait() {
|
||||
std::unique_lock<std::mutex> lock(m_);
|
||||
entered_++;
|
||||
cv_.notify_all();
|
||||
cv_.wait_for(lock, std::chrono::seconds(3), [&] { return open_; });
|
||||
}
|
||||
void open() {
|
||||
std::lock_guard<std::mutex> guard(m_);
|
||||
open_ = true;
|
||||
cv_.notify_all();
|
||||
}
|
||||
int entered() {
|
||||
std::lock_guard<std::mutex> guard(m_);
|
||||
return entered_;
|
||||
}
|
||||
|
||||
private:
|
||||
std::mutex m_;
|
||||
std::condition_variable cv_;
|
||||
bool open_ = false;
|
||||
int entered_ = 0;
|
||||
};
|
||||
|
||||
} // namespace parking_test
|
||||
|
||||
TEST(KeepAliveParkingTest, IdleConnectionsDoNotHoldTheWorkers) {
|
||||
using namespace parking_test;
|
||||
Server svr;
|
||||
serve_ok(svr, threads(1));
|
||||
RunningServer running(svr);
|
||||
Sockets socks;
|
||||
|
||||
// One kept-alive connection that has been served, and many that have not
|
||||
// sent anything yet. None of them may keep the only worker.
|
||||
auto served = socks.connect(running.port);
|
||||
ASSERT_TRUE(get_ok(served, 2000));
|
||||
for (int i = 0; i < 49; i++) {
|
||||
ASSERT_NE(INVALID_SOCKET, socks.connect(running.port));
|
||||
}
|
||||
|
||||
EXPECT_TRUE(get_ok(socks.connect(running.port), 1000));
|
||||
EXPECT_TRUE(get_ok(served, 1000));
|
||||
}
|
||||
|
||||
TEST(KeepAliveParkingTest, IdleConnectionsTimeOutTogether) {
|
||||
using namespace parking_test;
|
||||
Server svr;
|
||||
serve_ok(svr, threads(1));
|
||||
svr.set_keep_alive_timeout(1);
|
||||
RunningServer running(svr);
|
||||
Sockets socks;
|
||||
|
||||
for (int i = 0; i < 3; i++) {
|
||||
ASSERT_NE(INVALID_SOCKET, socks.connect(running.port));
|
||||
}
|
||||
// One worker waiting on each in turn would take about 3 seconds.
|
||||
auto deadline = in_ms(2000);
|
||||
for (auto s : socks.socks) {
|
||||
EXPECT_TRUE(sees_eof(s, deadline));
|
||||
}
|
||||
}
|
||||
|
||||
TEST(KeepAliveParkingTest, EstablishedConnectionWaitsForAFullQueue) {
|
||||
using namespace parking_test;
|
||||
std::atomic<int> queued{0}, full{0};
|
||||
Server svr;
|
||||
// One worker and room for one queued task.
|
||||
serve_ok(svr, [&] { return new CountingTaskQueue(1, 1, queued, full); });
|
||||
|
||||
Gate gate;
|
||||
svr.Get("/block", [&](const Request &, Response &res) {
|
||||
gate.wait();
|
||||
res.set_content("ok", "text/plain");
|
||||
});
|
||||
|
||||
RunningServer running(svr);
|
||||
// Before the server stops, so that a failed run does not wait for the
|
||||
// handler's timeout.
|
||||
auto open_gate = detail::scope_exit([&] { gate.open(); });
|
||||
Sockets socks;
|
||||
|
||||
auto a = socks.connect(running.port);
|
||||
ASSERT_TRUE(get_ok(a, 2000));
|
||||
|
||||
// B keeps the only worker busy.
|
||||
auto b = socks.connect(running.port);
|
||||
ASSERT_TRUE(send_get(b, "/block"));
|
||||
ASSERT_TRUE(wait_until([&] { return gate.entered() == 1; }, 2000));
|
||||
|
||||
// A new connection C fills the queue.
|
||||
auto c = socks.connect(running.port);
|
||||
ASSERT_TRUE(send_get(c));
|
||||
ASSERT_TRUE(wait_until([&] { return queued == 3; }, 2000));
|
||||
|
||||
// A's next request finds the queue full, but A has been served before, so it
|
||||
// waits instead of being cut off.
|
||||
ASSERT_TRUE(send_get(a));
|
||||
ASSERT_TRUE(wait_until([&] { return full >= 1; }, 2000));
|
||||
|
||||
gate.open();
|
||||
EXPECT_TRUE(receives_ok(b, 2000));
|
||||
EXPECT_TRUE(receives_ok(c, 2000));
|
||||
EXPECT_TRUE(receives_ok(a, 2000));
|
||||
}
|
||||
|
||||
TEST(KeepAliveParkingTest, BacklogIsRetriedOnceTheQueueHasRoom) {
|
||||
using namespace parking_test;
|
||||
std::atomic<int> queued{0}, full{0};
|
||||
CountingTaskQueue *queue = nullptr;
|
||||
Server svr;
|
||||
// Two workers and room for one queued task.
|
||||
serve_ok(svr,
|
||||
[&] { return queue = new CountingTaskQueue(2, 1, queued, full); });
|
||||
Gate first, second;
|
||||
svr.Get("/first", [&](const Request &, Response &res) {
|
||||
first.wait();
|
||||
res.set_content("ok", "text/plain");
|
||||
});
|
||||
svr.Get("/second", [&](const Request &, Response &res) {
|
||||
second.wait();
|
||||
res.set_content("ok", "text/plain");
|
||||
});
|
||||
|
||||
RunningServer running(svr);
|
||||
auto open_gates = detail::scope_exit([&] {
|
||||
first.open();
|
||||
second.open();
|
||||
});
|
||||
Sockets socks;
|
||||
|
||||
auto a = socks.connect(running.port);
|
||||
ASSERT_TRUE(get_ok(a, 2000));
|
||||
|
||||
// B and D take both workers, and C fills the queue. One at a time: the
|
||||
// queue has room for one job only until a worker takes it.
|
||||
ASSERT_TRUE(send_get(socks.connect(running.port), "/first"));
|
||||
ASSERT_TRUE(wait_until([&] { return first.entered() == 1; }, 2000));
|
||||
ASSERT_TRUE(send_get(socks.connect(running.port), "/second"));
|
||||
ASSERT_TRUE(wait_until([&] { return second.entered() == 1; }, 2000));
|
||||
ASSERT_TRUE(send_get(socks.connect(running.port), "/second"));
|
||||
ASSERT_TRUE(wait_until([&] { return queued == 4; }, 2000));
|
||||
|
||||
// A's next request waits in the backlog.
|
||||
ASSERT_TRUE(send_get(a));
|
||||
ASSERT_TRUE(wait_until([&] { return full >= 1; }, 2000));
|
||||
|
||||
// B's worker finishes, then takes C off the queue only later, which makes
|
||||
// room for A.
|
||||
queue->pause_after_ms = 200;
|
||||
first.open();
|
||||
EXPECT_TRUE(wait_until([&] { return queued == 5; }, 1000));
|
||||
queue->pause_after_ms = 0;
|
||||
|
||||
second.open();
|
||||
EXPECT_TRUE(receives_ok(a, 2000));
|
||||
}
|
||||
|
||||
TEST(KeepAliveParkingTest, ZeroKeepAliveTimeoutServesTheFirstRequest) {
|
||||
using namespace parking_test;
|
||||
Server svr;
|
||||
serve_ok(svr, threads(1));
|
||||
svr.set_keep_alive_timeout(0);
|
||||
|
||||
// The request is on the socket before the server accepts the connection,
|
||||
// so it does not race the 10 ms floor. Without the floor the connection is
|
||||
// closed unread.
|
||||
Sockets socks;
|
||||
auto sock = INVALID_SOCKET;
|
||||
RunningServer running(svr, [&](int port) {
|
||||
sock = socks.connect(port);
|
||||
send_get(sock);
|
||||
});
|
||||
EXPECT_TRUE(receives_ok(sock, 1000));
|
||||
}
|
||||
|
||||
TEST(KeepAliveParkingTest, StopClosesParkedConnectionsPromptly) {
|
||||
using namespace parking_test;
|
||||
for (int round = 0; round < 5; round++) {
|
||||
SCOPED_TRACE(round);
|
||||
Server svr;
|
||||
serve_ok(svr, threads(2));
|
||||
RunningServer running(svr);
|
||||
Sockets socks;
|
||||
|
||||
for (int i = 0; i < 20; i++) {
|
||||
ASSERT_TRUE(get_ok(socks.connect(running.port), 1000));
|
||||
}
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
svr.stop();
|
||||
running.thread.join();
|
||||
EXPECT_LT(std::chrono::steady_clock::now() - start,
|
||||
std::chrono::milliseconds(500));
|
||||
|
||||
auto deadline = in_ms(1000);
|
||||
for (auto s : socks.socks) {
|
||||
EXPECT_TRUE(sees_eof(s, deadline));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#ifdef CPPHTTPLIB_SSL_ENABLED
|
||||
namespace parking_test {
|
||||
|
||||
// Completes a TLS handshake on `sock` and sends nothing more, like a browser
|
||||
// that has opened a connection ahead of time. Freeing the session sends no
|
||||
// close_notify, so the connection stays quiet.
|
||||
bool handshake_only(socket_t sock) {
|
||||
auto ctx = tls::create_client_context();
|
||||
tls::set_verify_client(ctx, false);
|
||||
auto session = tls::create_session(ctx, sock);
|
||||
auto se = detail::scope_exit([&] {
|
||||
tls::free_session(session);
|
||||
tls::free_context(ctx);
|
||||
});
|
||||
tls::TlsError err;
|
||||
return session &&
|
||||
tls::connect_nonblocking(session, sock, 2, 0, &err, nullptr);
|
||||
}
|
||||
|
||||
Client tls_client(int port) {
|
||||
Client cli("https://127.0.0.1:" + std::to_string(port));
|
||||
cli.enable_server_certificate_verification(false);
|
||||
cli.set_keep_alive(true);
|
||||
cli.set_connection_timeout(1);
|
||||
cli.set_read_timeout(1);
|
||||
return cli;
|
||||
}
|
||||
|
||||
bool tls_get_ok(Client &cli) {
|
||||
auto res = cli.Get("/");
|
||||
return res && res->body == "ok";
|
||||
}
|
||||
|
||||
} // namespace parking_test
|
||||
|
||||
#ifdef CPPHTTPLIB_OPENSSL_SUPPORT
|
||||
TEST(KeepAliveParkingTest, SslFreeBuffersOnlyWhereTheCveIsFixed) {
|
||||
struct {
|
||||
unsigned long version;
|
||||
bool safe;
|
||||
} cases[] = {
|
||||
{0x30000020UL, false}, {0x300000d0UL, false}, {0x300000e0UL, true},
|
||||
{0x300000f0UL, true}, {0x30100050UL, false}, {0x30100060UL, true},
|
||||
{0x30200010UL, false}, {0x30200020UL, true}, {0x30300000UL, false},
|
||||
{0x30300010UL, true}, {0x30400000UL, true}, {0x40000000UL, true},
|
||||
};
|
||||
for (const auto &c : cases) {
|
||||
EXPECT_EQ(c.safe, detail::ssl_free_buffers_is_safe(c.version))
|
||||
<< std::hex << c.version;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
TEST(KeepAliveParkingTest, IdleTlsConnectionsDoNotHoldTheWorkers) {
|
||||
using namespace parking_test;
|
||||
SSLServer svr(SERVER_CERT_FILE, SERVER_PRIVATE_KEY_FILE);
|
||||
ASSERT_TRUE(svr.is_valid());
|
||||
serve_ok(svr, threads(1));
|
||||
svr.set_read_timeout(1); // so that a failing run does not take long
|
||||
RunningServer running(svr);
|
||||
Sockets socks;
|
||||
|
||||
// One served connection, some that have not started their handshake, and
|
||||
// one that has finished it and gone quiet. None may keep the only worker.
|
||||
auto served = tls_client(running.port);
|
||||
ASSERT_TRUE(tls_get_ok(served));
|
||||
for (int i = 0; i < 10; i++) {
|
||||
ASSERT_NE(INVALID_SOCKET, socks.connect(running.port));
|
||||
}
|
||||
EXPECT_TRUE(handshake_only(socks.connect(running.port)));
|
||||
|
||||
auto other = tls_client(running.port);
|
||||
EXPECT_TRUE(tls_get_ok(other));
|
||||
EXPECT_TRUE(tls_get_ok(served));
|
||||
}
|
||||
|
||||
TEST(KeepAliveParkingTest, FailedTlsHandshakeClosesTheConnection) {
|
||||
using namespace parking_test;
|
||||
SSLServer svr(SERVER_CERT_FILE, SERVER_PRIVATE_KEY_FILE);
|
||||
ASSERT_TRUE(svr.is_valid());
|
||||
serve_ok(svr, threads(1));
|
||||
RunningServer running(svr);
|
||||
Sockets socks;
|
||||
|
||||
auto plain = socks.connect(running.port);
|
||||
ASSERT_TRUE(send_get(plain));
|
||||
EXPECT_TRUE(sees_eof(plain, in_ms(1000)));
|
||||
EXPECT_NE(0, svr.ssl_last_error());
|
||||
}
|
||||
|
||||
TEST(KeepAliveParkingTest, StopClosesParkedTlsConnections) {
|
||||
using namespace parking_test;
|
||||
SSLServer svr(SERVER_CERT_FILE, SERVER_PRIVATE_KEY_FILE);
|
||||
ASSERT_TRUE(svr.is_valid());
|
||||
serve_ok(svr, threads(2));
|
||||
RunningServer running(svr);
|
||||
|
||||
std::vector<Client> clients;
|
||||
for (int i = 0; i < 5; i++) {
|
||||
clients.push_back(tls_client(running.port));
|
||||
ASSERT_TRUE(tls_get_ok(clients.back()));
|
||||
}
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
svr.stop();
|
||||
running.thread.join();
|
||||
EXPECT_LT(std::chrono::steady_clock::now() - start,
|
||||
std::chrono::milliseconds(500));
|
||||
}
|
||||
#endif
|
||||
Reference in new issue
Block a user