KqueuePoller keeps the Poller contract with EV_ONESHOT, which deletes an
event once it is delivered, so arm() adds it each time. wake() uses the
socket pair that PollPoller uses, since EVFILT_USER is not on every BSD.
make_poller() picks it on macOS and FreeBSD and falls back to poll()
elsewhere.
The contract tests now run on it as well.
EpollPoller keeps the Poller contract with EPOLLONESHOT: a delivered
descriptor stays registered but disabled, so arm() modifies it, and adds
it when it was closed in between. wake() writes to an eventfd that stays
armed. make_poller() picks it on Linux and falls back to poll().
poll() goes over every parked connection on each wait, and epoll only
over the ready ones. With 10000 idle keep-alive connections parked, 10
active clients got 95k requests per second instead of 2.4k, at 0.1 ms
instead of 4 ms each. With few connections nothing changed.
The contract tests now run on both backends.
SocketStream now allocates its 4KB read buffer on the first read that
needs it, and a connection gives the buffer back when it is parked. An
idle connection that has sent nothing never allocates one. With 10000
idle keep-alive connections parked, the server's RSS went from 47MB to
8MB; requests per second did not change.
A kept-alive connection holds a worker thread while it waits for its
next request, so connections beyond the thread count wait for one to be
closed, and with a pool smaller than the number of clients some clients
get nothing until the keep-alive timeout or the max count ends another
connection.
With keep-alive parking, listen() runs an event loop on the detail::Poller
instead of the blocking accept loop. A connection waiting for its next
request is parked there, and goes back to a worker through the TaskQueue
when the request arrives. The handlers, content providers and WebSocket
keep their blocking API and still hold a thread while they run.
It is opt-in and off by default. With it on:
- consecutive requests on a connection may run on different threads;
- when the queue is full, a connection that has not been served yet is
closed as before, but an established one waits in a backlog instead of
losing its request;
- the loop owns the listening socket while it watches it, and stop()
hands it over instead of closing it under the poller;
- SSLServer ignores the setting for now, and an override of
process_and_close_socket() is not called.
On two CPUs with two workers, 50 keep-alive clients of a 2.7 ms handler
were all served evenly (the slowest response took 70 ms instead of 5 s
with 10 clients getting nothing), light requests from 50 clients went
from 82k to 248k per second, and 10 clients were served after 10000
idle connections that had stopped the old loop. With no more clients
than workers, handing the connection over costs about half of the
requests per second of a very light handler.
The old accept loop moved unchanged into accept_loop().
process_server_socket_core() now takes the stream, and
drain_and_close_socket() takes how long to wait for the peer.
wait() cleared the pending flag before reading the wake byte. A wake()
landing in between sent a second byte, the read took both, and the flag
stayed set with nothing left to read, so every later wake() was skipped
and the waiting thread slept on. Clear the flag after the read instead,
as libuv does for uv_async_send(): a wake() that lands in between finds
the flag still set and sends nothing, which loses nothing because the
caller looks at its shared state after wait() returns.
process_server_socket_core() now takes the function that waits for the
next request and the number of requests served so far, instead of
calling keep_alive() and counting on its own stack. Both callers pass
keep_alive() and a fresh counter, so nothing changes yet; this lets a
caller stop waiting and resume the same connection later without losing
track of keep_alive_max_count.
Groundwork for an opt-in reactor that parks idle keep-alive connections
instead of blocking a worker thread on each of them.
detail::Poller delivers readiness once per arm(), so a descriptor is
either armed (owned by the poller) or owned by whoever got the event.
The same contract maps onto poll(), epoll, kqueue and Windows' AFD_POLL,
so a backend can be added without touching its users. wake() ends a
wait() from any thread; an armed descriptor must be disarmed before it
is closed, which the poll() backend asserts.
detail::PollPoller is the portable backend (poll() / WSAPoll()), and
detail::create_socket_pair() emulates socketpair() on Windows with
sockets that child processes do not inherit.
Nothing uses them yet.
parse_url rejected any scheme containing an uppercase letter, so
Client("HTTPS://..."), WebSocketClient("WSS://...") and a redirect to
"Location: HTTPS://..." failed, although the scheme is case-insensitive
(RFC 3986 3.1). Lowercase the scheme in parse_url so that every caller
compares one canonical form and selects TLS for it.
SSLClient verifies the server certificate chain with CryptoAPI on
Windows by default, while WebSocketClient always left it to the TLS
backend, whose snapshot of the Windows store lacks the roots Windows
fetches on demand. A wss:// connection could fail against a host that
SSLClient connects to.
WebSocketClient now makes the same decision through a shared helper,
and gets enable_windows_certificate_verification() to turn it off.
Fix#2685
A read timeout set with set_read_timeout() was reported only on a frame
boundary. One that elapsed inside a frame, or between the fragments of a
message, failed the read and closed the connection, as what had been read
could not be resumed. A small frame written in pieces often arrives in
pieces on Linux, so a server polling with a short timeout lost the
connection on its first message.
A frame is now read by ws::impl::FrameReader, which keeps what it has read
across a timeout, and WebSocket keeps the message being assembled, so
read() returns Timeout wherever the timeout lands and goes on from there at
the next call. close() waits for the peer's Close through the same reader.
read_websocket_frame() keeps its one-shot meaning.
write_websocket_frame() writes the header with the payload's first chunk
in one write, and WebSocket connections set TCP_NODELAY unless
CPPHTTPLIB_WEBSOCKET_TCP_NODELAY is false: on Linux, small messages each
way otherwise waited on Nagle's algorithm for the peer's delayed ACK, a
few in fifty around 30-40 ms late.
ClientImpl::redirect() resolved a Location with resolve_relative_location(), which hands a value with a ':' in its first segment back unchanged as having a scheme, and then parsed it with parse_url(), which also accepts a bare host[:port] so that Client("localhost:8080") works. A Location such as "internal:8080" or "127.0.0.1:6379" therefore came out with an empty scheme and a host, slipped past the http/https check, and was followed as a cross-host redirect to that host and port. A value such as "localhost:8080/x" was sent to the same host as a path without a leading '/'.
Follow only http/https URLs and references starting with '/', and refuse anything else, as an unsupported scheme already was. Network-path references such as "//host:port/x" are still followed.
A read() assembling a fragmented message looped over continuation
frames without looking at closed_, which the loop over frames checks.
A peer that kept sending fragments, empty ones included, never let the
read reach a wait, so a close() from another thread could not end it.
The read now ends with Fail once the connection is closed.
stop() takes the listening socket away, and listen() then waits for every
handler to return. A WebSocket handler waiting in read() on a client that
stays connected never learned of the stop: with no read timeout it waited
for the 300 s backstop, and with one it got Timeout back on an open
connection, so a `while (ws.read(msg) != Fail)` loop went on forever and
listen() with it.
The server now hands each WebSocket a predicate for its own stop, and the
WebSocket's read interrupt answers to it as it does to a pong timeout. A
read cut short while the server stops, between messages or in the middle
of a fragmented one, sends a Close with 1001 Going Away and returns Fail.
That close does not wait for the peer's reply, so a stop is not held up
per connection.
A server WebSocket's reads now wait in the interruptible slices whatever
the heartbeat settings, as keep-alive connections already do for the same
check.
* Let ClientImpl::stop() end a connect or a TLS handshake
stop() took socket_mutex_, which send_() and open_stream() hold while
they open the connection, so a stop() made during a connect, a TLS
handshake or the CONNECT of a proxy tunnel waited for the connection
to open or to time out, 300 seconds by default, for a server that
never answers.
stop() no longer waits for the mutex while a connection opens: it
cancels the connection phase by a compare-exchange from Connecting,
and the end of the phase reads the cancel in the same exchange that
takes the state back to Idle, so a cancel stop() accepts is never
lost. stop() also sets stop_requested_: the wait of the connect and
the waits of the handshake, in every TLS backend, poll it in slices of
100 ms, as do the reads of the CONNECT of a proxy tunnel. A request or
a stream canceled there reports Error::Canceled, before it sends
anything or takes the socket. A request opened after stop() clears the
flag as before, and a stream now does too. A negative connection
timeout still waits without end in the interruptible connect wait.
* Add ClientImpl::set_interrupt(), a stop the caller holds
stop() ends what is in flight, and a request or a stream opened after
it clears the flag, so a stop() made just before a request goes out is
lost.
set_interrupt() gives the client a predicate its caller owns, polled
wherever stop_requested_ is: the connect, the TLS handshake, the
CONNECT of a proxy tunnel and the reads of a request, and the head of
a stream too. The client of a cross-host redirect keeps it. A request
or a stream refuses to go out once it holds, so a stop the caller
holds is never lost, whenever it comes. A call blocked in the system,
a send() of a large body or a TLS read inside a record, still ends
through stop() alone, which shuts the socket down.
A timeout past what the clock reaches now waits without end instead of
expiring at once, so such a wait still ends on the interrupt.
* test: create the backlog sockets with SOCK_NONBLOCK
detail::set_nonblocking lives in httplib.cc in the split build, so
test_split did not compile. The test only runs on Linux, where
SOCK_NONBLOCK makes the socket nonblocking at creation.
* Open the connection without socket_mutex_ instead
Rework the change so that send_() and open_stream_() open the
connection without holding socket_mutex_, as requests are already
serialized by request_mutex_. stop() then never waits for a connection
being opened: it always sets stop_requested_, which the connect wait,
the TLS handshake, the CONNECT and the wait for the head of a stream
poll, and the call ends with Error::Canceled before it sends anything.
This replaces the connection state machine and the try_lock loop in
stop().
The flag is cleared once, as a send() or an open_stream() starts, so a
retry inside the call no longer drops a stop() it already got.
Drop set_interrupt() for now: a predicate held per client cannot be
shared, e.g. by an SSEClient that borrows the caller's Client. Also drop
StopEndsATlsReadInsideARecord, which checked existing behavior that
does not hold on Windows, and share the listener setup of the tests.
Keep the deadline_after() fix: a timeout past what the clock reaches
waits forever instead of overflowing into an immediate timeout.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* drop a passed-through Content-Length when the client reframes the body
send_with_content_provider_and_receiver() kept a Content-Length the
caller passed through after the library reframed the body: it compresses
a buffered body under set_compress(true), or streams a provider without a
length as chunked. prepare_default_headers() only adds a Content-Length
when none is present, so the stale value went out describing the original
body. With compression the server reads only that many bytes of the
longer gzip stream and the rest desyncs the next request on a kept-alive
connection; with a chunked provider the client emits both Content-Length
and Transfer-Encoding. Erase it once the library owns the framing so
write_request() emits the matching length, or none for a chunked body.
A plain, uncompressed body keeps the caller's value.
* Document the dropped Content-Length and tidy the new tests
Note in the README that a per-request Content-Length is dropped when the library compresses or chunks the body; a default header is not. Shorten the comments, use HOST like the neighbouring tests, and start the chunked test's Content-Length count at a value the handler has to overwrite.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
close() waited for the peer's Close reply with a read timeout that
applied to each read, so a peer that kept sending other frames, or sent
one frame a byte at a time, held close() for as long as it liked.
Bound the wait as a whole. close() sets the stream's max timeout, which
shortens each wait to what is left, and stops reading frames at the
deadline, which matters when the peer keeps the socket full and no wait
happens at all. WebSocketSSLStream gains set_max_timeout(), the one
stream a WebSocket uses that lacked it.
A read() running on another thread when close() is called still waits
for the reply up to its own read timeout, as documented. The docs now
say how to bound it.
Server::process_and_close_socket() half-closes the socket and drains what
the peer is still sending before the final close (#2534), so that a close
with unread data, or with bytes still in flight, is not answered with a
RST. A RST can make the peer's stack discard a response it has received
but not yet read, as Windows does.
SSLServer::process_and_close_socket() still closed outright. After a
completed handshake, shut down TLS as before (sending close_notify when
the exchange completed) and then drain the same way. A failed handshake
closes outright as before.
Fix#2672
A kept-alive connection was reused while bytes that belong to no
request were waiting on it, and the next request read them as its
response. Such bytes are not a response (RFC 9112 9.2). A server may,
for one, send a 408 before it closes an idle connection.
Close the connection when they show up: after a response, if bytes
past it are already buffered in the stream or the TLS library, and
before reuse, if anything is readable on the socket. As in curl, the
reuse check no longer asks the TLS library whether the peer closed, so
it is the same for every TLS backend. A TLS message on an idle
connection, such as a late session ticket, now costs a new connection.
SSLClient::setup_proxy_connection() left the socket open when the TLS
handshake inside the CONNECT tunnel failed, so the client kept a
connection with no TLS session on it. Only the next request's reuse
check closed it, because tls::is_peer_closed() reports a missing
session as closed. Close it where the handshake fails, as
ensure_socket_connection() does for a direct connection.
* take the request lock in open_stream_ before reusing the socket
* Hold the request lock only while open_stream_ takes the socket
The handle owns the connection once the socket is transferred to it, so
holding request_mutex_ while the stream wrote its request and read the
response head only made requests on other threads wait for the stream.
Also shorten the comments.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* resolve junctions and symlinks in canonicalize_path on Windows
* Shorten the canonicalize_path comment
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
Move the line splitting and event assembly out of
SSEClient::run_event_loop() into a parser that is fed chunks and reports
events, ids and retry values through handlers. The event loop is left
with I/O and reconnection, and the parser can be driven without a
server, which the following test and fuzzer changes rely on.
No behavior change.
ThreadPool::enqueue constructed the dynamic thread without handling
std::system_error. When thread creation failed (e.g. EAGAIN under a
pids limit), the exception escaped the accept loop and destroying the
pool's joinable threads called std::terminate. Treat the failure as
"no spare capacity" and leave the job for an existing worker.
A client using set_address_family(AF_UNIX) took the host of an absolute Location as a socket path, so a redirect to http://localhost/next failed and one to http://other.sock/next went to ./other.sock. Every hop now stays on the client's socket, as with curl's --unix-socket.
The same host is still decided from the Location, as curl does: only localhost (the Host this client sends) counts as this client, so a redirect to any other host goes through the cross-host client and drops Authorization and Cookie. That client reaches the original socket through addr_map_, and a host the caller mapped with set_direct_hostname_addr_map keeps its mapping. An https Location runs TLS over the socket with the Location's host.
The mapped-target lookup shared with make_connect_request_target is factored into detail::addr_map_target.
A request sent with send(Request) and params got those params appended
to the Location of a redirect that had no query, on the same origin or
another one. The Location alone names the next target, as it already
did for Get/Post(path, params).
RFC 9110 14.2 says an origin server MUST ignore a Range header field
with a range unit it does not understand. parse_range_header() accepts
only bytes=, so a request such as "Range: items=1-2" got a 416 instead
of the full response. The unit is now checked first, and only a bytes
range is parsed and answered with a 416 when it is invalid.
* Cap a send() or wolfSSL_read() length at INT_MAX
On 64-bit Windows SocketStream::write() handed size to send(), whose
length is an int, so a single write of 4 GiB or more went out as its low
32 bits: a 4 GiB + 1 byte write sent one byte, and write_data() then
retried the remaining 4 GiB as zero-byte sends forever. The INT_MAX cap
that read() has was limited to 32-bit Windows in df20c27 (#1294), when
the body was still written with one unchecked write(); write_data() now
sends the rest, so the cap applies to every Windows build again.
The wolfSSL tls::read() had the same cast, so a read into a buffer over
INT_MAX failed. Cap it as the OpenSSL backend has done since f80864c.
* Drop the 4 GiB write test and shorten comments
The Windows-only test allocated 4 GiB and took about 5 s, which is too
heavy for CI.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* keep a relative Location on its host in resolve_relative_location
* Shorten the comment on on_this_host in resolve_relative_location
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
stop() shuts the socket down to end a request in flight, but on Windows
that does not wake a read waiting in WSAPoll (#1396). The request then
waited until the server answered, the read timeout passed, or the OS
dropped the connection.
stop() now also sets a flag, and send_() gives the request's stream an
interrupt check that reads it, as SSEClient::stop() does since 801bf2e.
The wait for data then ends within 100 ms on every platform. A request
started after stop() clears the flag. A redirect or a digest retry sent
inside the request keeps it.
select_read_interruptible() now rounds its last wait up to a whole
millisecond. poll() takes milliseconds, and the wait ended up to 1 ms
before the deadline, which MaxTimeoutTest caught once every client
request waited through this function.
ClientImpl and SSLClient let callers set origin credentials with
set_basic_auth()/set_bearer_token_auth(), but WebSocketClient could only
take a fixed Authorization header through its constructor. Add both
setters: each connect() adds the Authorization header to the upgrade
request from the current credentials (Basic over Bearer), unless the
constructor headers already carry one, and never to the proxy's CONNECT
request.
Through a CONNECT proxy it is the proxy that connects to the origin, and the CONNECT target was always the original hostname, so there was no way to send the tunnel to another address. set_tunnel_hostname_addr_map() sets that target on SSLClient, Client and WebSocketClient. The CONNECT request's Host follows it, including the Digest retry, while the Host header, the SNI and the certificate check inside the tunnel keep the original hostname. The map is also copied to a redirect client.
set_hostname_addr_map() is deprecated in favor of set_direct_hostname_addr_map(), which behaves the same and names the connection it applies to. The two maps are independent and correspond to curl's --resolve and --connect-to; neither applies to the proxy's own address.
Mbed TLS and wolfSSL read a TLS EOF without close_notify as a clean end, so a response delimited by the connection close could be cut short and still come back complete. Both reads now end cleanly only on a close_notify, as on OpenSSL.
On Windows a ':' ends a drive prefix or starts an NTFS stream name, so a filename such as "C:.." or "a.txt:stream" survived sanitize_filename and could replace the upload directory or write to an alternate data stream. Cut at ':' on Windows as well.
resolve_relative_location removed "." and ".." segments only from a relative-path Location. A Location that starts with "/" or an absolute URI was requested as is, so a redirect to /dir/../target went out as /dir/../target and a server that does not normalize the path answered 404.
RFC 3986 section 5.2.2 applies remove_dot_segments to the target path of every reference that has a path, and Go and urllib3 do the same. The loop now lives in detail::remove_dot_segments, and ClientImpl::redirect applies it to an absolute path as well. Relative locations resolve as before.
On NTFS a ':' in a path names a stream of the file, so /dir/test.html::$DATA opened the default stream of dir/test.html and passed a set_pre_routing_handler() guard on /dir/test.html, and /setup.exe:Zone.Identifier served the stream Windows attaches to downloaded files. is_valid_path() now refuses a ':' on Windows.
Elsewhere a ':' is an ordinary file name character, so it is still served: a file such as 2024-01-01T10:00:00.log would otherwise answer 404. The test pins both sides.
last_event_id() returned a reference to a string that run_event_loop() assigns on its own thread under start_async(), so a caller reading it from another thread raced the writes. The member is now guarded by a mutex and the getter returns a copy.
The return type changes from const std::string & to std::string. Code that binds the result to a const reference or copies it still compiles.
write_websocket_frame and the request-body write in open_stream_ called
Stream::write once and only failed on a negative return, so a short
write (one record per mbedtls_ssl_write call, or SO_SNDTIMEO expiring
after part of the data went out) dropped the rest of the frame or body.
Route both through write_data, as the other body writers do.
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
The macOS system CA loader took every certificate that has Trust Settings, in the system, admin or user domain, as a trusted root without reading those settings. A root set to "Never Trust" in Keychain Access, or trusted only for S/MIME or code signing, still verified TLS servers.
The loader now evaluates the settings for the SSL policy. A per-user setting overrides the admin domain, which overrides the system domain, and only TrustRoot and TrustAsRoot are loaded. The OpenSSL path shares the helper that the Mbed TLS and wolfSSL paths already use.
open_stream_() created a decoder whatever set_decompress() said, so a response stream decoded the body when decompression was disabled, and failed with UnsupportedContentEncoding in a build without that decoder. The buffered path already checks the option; the streaming path now does too. The new test covers open_stream() and stream::Get().
Fixes#2639.
OpenSSL 4.1 no longer falls back to the subject Common Name when it
checks a host name (RFC 9525), so SSLClientServerTest.TlsVerifyHostname
began failing in the Windows CI job that installs the newest OpenSSL 4.x.
Make every backend agree instead of following whichever OpenSSL is
installed. OpenSSL passes X509_CHECK_FLAG_NEVER_CHECK_SUBJECT to both the
handshake check and verify_hostname(), and Mbed TLS and wolfSSL drop their
own CN fallback. A certificate that names its host only in the CN is now
rejected on all backends, as it already was on BoringSSL.
is_redirect treated any 3xx with a Location as a redirect, so with set_follow_location(true) a 304 or 305 that carried a Location was followed. Only 301, 302, 303, 307 and 308 are redirects a client follows (RFC 9110 section 15.4), as in Go, requests and urllib3. A Location is still required. The 304 check in process_request that is_redirect now covers is dropped.
Without keep-alive, the client sends "Connection: close", but before
following a redirect it disconnected only when the response repeated
the option. RFC 9112 (section 9.6) lets a server close the connection
without repeating it, and a client that sent "close" must not send
another request on that connection. The redirected request was written
to the closing connection and failed with "Failed to read connection"
whenever the server closed it first. The client now disconnects before
a redirect or an auth retry whenever the request asked for the close.
Co-authored-by: Wleeaf <179125857+wleeaf@users.noreply.github.com>
In absolute-form the path follows the authority, so a path such as "@evil.example/x" made the proxy connect to another host and forward the origin's credentials. Both handle_request and open_stream_ now refuse such a path before anything is written.