yhirose
9bfd2ddf1c
Bring the READMEs and the docs pages (en and ja) in line with the current header: - Fix the client sample that does not compile, the Transfer-Encoding statement for default headers, the verifier callback comment, the thread pool spawn rule, the sanitize_filename description, and the inverted Accept-Encoding sentence. - Describe recent behaviour: TLS errors behind Read/Write failures, Host validation, Connection: close for a body without length, content_reader rules, refused CONNECT, WebSocketClient proxies, CPPHTTPLIB_INTERIM_RESPONSE_MAX_COUNT, the SSE event size limit, and the payload limit of the Stream API. - Complete the API references of WebSocket, WebSocketClient and the stream:: helpers, and the SSE reconnect rules. - Fix cookbook pages: keychain loading on every backend, received Close status, a multipart sample that trusted the client's file name, an echo sample that sent binary as text, SNI, pre-request handler scope, stop() with WebSocket handlers, and the thread pool notes. - Restore the mutex in the LLM app's Chapter 5 code.
2.3 KiB
title, order, status
| title | order | status |
|---|---|---|
| S21. Configure the Thread Pool | 40 | draft |
cpp-httplib serves requests from a thread pool. By default, the base thread count is the greater of std::thread::hardware_concurrency() - 1 and 8, and it can scale up dynamically to 4× that. To set thread counts explicitly, provide your own factory via new_task_queue.
Set thread counts
httplib::Server svr;
svr.new_task_queue = [] {
return new httplib::ThreadPool(/*base_threads=*/8, /*max_threads=*/64);
};
svr.listen("0.0.0.0", 8080);
The factory is a lambda returning a TaskQueue*. Pass base_threads and max_threads to ThreadPool and the pool scales between them based on load. Idle threads exit after a timeout (3 seconds by default).
Also cap the queue
The pending queue can eat memory if it grows unchecked. You can cap it too.
svr.new_task_queue = [] {
return new httplib::ThreadPool(
/*base_threads=*/12,
/*max_threads=*/0, // disable dynamic scaling
/*max_queued_requests=*/18);
};
max_threads=0 disables dynamic scaling — you get a fixed base_threads. Requests that don't fit in max_queued_requests are rejected.
Use your own thread pool
You can plug in a fully custom thread pool by subclassing TaskQueue and returning it from the factory.
class MyTaskQueue : public httplib::TaskQueue {
public:
MyTaskQueue(size_t n) { pool_.start_with_thread_count(n); }
bool enqueue(std::function<void()> fn) override { return pool_.post(std::move(fn)); }
void shutdown() override { pool_.shutdown(); }
private:
MyThreadPool pool_;
};
svr.new_task_queue = [] { return new MyTaskQueue(12); };
Handy when you already have a thread pool in your project and want to keep thread management unified.
Compile-time tuning
You can set the defaults with macros if you want compile-time configuration.
#define CPPHTTPLIB_THREAD_POOL_COUNT 16 // base thread count
#define CPPHTTPLIB_THREAD_POOL_MAX_COUNT 128 // max thread count
#define CPPHTTPLIB_THREAD_POOL_IDLE_TIMEOUT 5 // seconds before idle threads exit
#include <httplib.h>
Note: A WebSocket connection holds a worker thread for its entire lifetime. For lots of simultaneous WebSocket connections, raise the maximum thread count (e.g.
ThreadPool(8, 64)).