An idle parked TLS connection took about 24 KB, most of it OpenSSL's read
and write buffers. SSL_free_buffers() before parking brings it to about
15 KB; OpenSSL keeps them when a record is pending and makes new ones on
the next read. The request rate is unchanged. BoringSSL, LibreSSL, Mbed TLS
and wolfSSL are left as they are.
SSLServer::TlsConnection plugs the TLS session into the parked connection.
A worker runs the handshake when the event loop sees the ClientHello, so a
connection that has not started it, or has gone quiet after it, waits in
the event loop instead of holding a worker. The handshake counts as having
read what the loop saw, and a connection that has finished it waits for
room like a served one when the queue is full. close_notify sent from the
event loop does not block.
SSLServer::start_session() holds the session setup and handshake that both
the blocking path and TlsConnection use, and supports_keep_alive_parking()
goes away.
KeepAliveConnection becomes a base that parking_loop() and
resume_connection() drive without knowing the transport. PlainConnection
holds the SocketStream, and Server::new_keep_alive_connection() makes one
for each accepted socket, so that SSLServer can add a TLS connection next.
A connection now counts as established for a full queue once a worker has
opened it, which for plain HTTP is when it has been served, as before.
Closing happens explicitly in the event loop and on the worker; the
destructor is only a net. No change in behaviour.
KqueuePoller keeps the Poller contract with EV_ONESHOT, which deletes an
event once it is delivered, so arm() adds it each time. wake() uses the
socket pair that PollPoller uses, since EVFILT_USER is not on every BSD.
make_poller() picks it on macOS and FreeBSD and falls back to poll()
elsewhere.
The contract tests now run on it as well.
EpollPoller keeps the Poller contract with EPOLLONESHOT: a delivered
descriptor stays registered but disabled, so arm() modifies it, and adds
it when it was closed in between. wake() writes to an eventfd that stays
armed. make_poller() picks it on Linux and falls back to poll().
poll() goes over every parked connection on each wait, and epoll only
over the ready ones. With 10000 idle keep-alive connections parked, 10
active clients got 95k requests per second instead of 2.4k, at 0.1 ms
instead of 4 ms each. With few connections nothing changed.
The contract tests now run on both backends.
SocketStream now allocates its 4KB read buffer on the first read that
needs it, and a connection gives the buffer back when it is parked. An
idle connection that has sent nothing never allocates one. With 10000
idle keep-alive connections parked, the server's RSS went from 47MB to
8MB; requests per second did not change.
A kept-alive connection holds a worker thread while it waits for its
next request, so connections beyond the thread count wait for one to be
closed, and with a pool smaller than the number of clients some clients
get nothing until the keep-alive timeout or the max count ends another
connection.
With keep-alive parking, listen() runs an event loop on the detail::Poller
instead of the blocking accept loop. A connection waiting for its next
request is parked there, and goes back to a worker through the TaskQueue
when the request arrives. The handlers, content providers and WebSocket
keep their blocking API and still hold a thread while they run.
It is opt-in and off by default. With it on:
- consecutive requests on a connection may run on different threads;
- when the queue is full, a connection that has not been served yet is
closed as before, but an established one waits in a backlog instead of
losing its request;
- the loop owns the listening socket while it watches it, and stop()
hands it over instead of closing it under the poller;
- SSLServer ignores the setting for now, and an override of
process_and_close_socket() is not called.
On two CPUs with two workers, 50 keep-alive clients of a 2.7 ms handler
were all served evenly (the slowest response took 70 ms instead of 5 s
with 10 clients getting nothing), light requests from 50 clients went
from 82k to 248k per second, and 10 clients were served after 10000
idle connections that had stopped the old loop. With no more clients
than workers, handing the connection over costs about half of the
requests per second of a very light handler.
The old accept loop moved unchanged into accept_loop().
process_server_socket_core() now takes the stream, and
drain_and_close_socket() takes how long to wait for the peer.
wait() cleared the pending flag before reading the wake byte. A wake()
landing in between sent a second byte, the read took both, and the flag
stayed set with nothing left to read, so every later wake() was skipped
and the waiting thread slept on. Clear the flag after the read instead,
as libuv does for uv_async_send(): a wake() that lands in between finds
the flag still set and sends nothing, which loses nothing because the
caller looks at its shared state after wait() returns.
process_server_socket_core() now takes the function that waits for the
next request and the number of requests served so far, instead of
calling keep_alive() and counting on its own stack. Both callers pass
keep_alive() and a fresh counter, so nothing changes yet; this lets a
caller stop waiting and resume the same connection later without losing
track of keep_alive_max_count.
Groundwork for an opt-in reactor that parks idle keep-alive connections
instead of blocking a worker thread on each of them.
detail::Poller delivers readiness once per arm(), so a descriptor is
either armed (owned by the poller) or owned by whoever got the event.
The same contract maps onto poll(), epoll, kqueue and Windows' AFD_POLL,
so a backend can be added without touching its users. wake() ends a
wait() from any thread; an armed descriptor must be disarmed before it
is closed, which the poll() backend asserts.
detail::PollPoller is the portable backend (poll() / WSAPoll()), and
detail::create_socket_pair() emulates socketpair() on Windows with
sockets that child processes do not inherit.
Nothing uses them yet.
parse_url rejected any scheme containing an uppercase letter, so
Client("HTTPS://..."), WebSocketClient("WSS://...") and a redirect to
"Location: HTTPS://..." failed, although the scheme is case-insensitive
(RFC 3986 3.1). Lowercase the scheme in parse_url so that every caller
compares one canonical form and selects TLS for it.
SSLClient verifies the server certificate chain with CryptoAPI on
Windows by default, while WebSocketClient always left it to the TLS
backend, whose snapshot of the Windows store lacks the roots Windows
fetches on demand. A wss:// connection could fail against a host that
SSLClient connects to.
WebSocketClient now makes the same decision through a shared helper,
and gets enable_windows_certificate_verification() to turn it off.
Fix#2685
A read timeout set with set_read_timeout() was reported only on a frame
boundary. One that elapsed inside a frame, or between the fragments of a
message, failed the read and closed the connection, as what had been read
could not be resumed. A small frame written in pieces often arrives in
pieces on Linux, so a server polling with a short timeout lost the
connection on its first message.
A frame is now read by ws::impl::FrameReader, which keeps what it has read
across a timeout, and WebSocket keeps the message being assembled, so
read() returns Timeout wherever the timeout lands and goes on from there at
the next call. close() waits for the peer's Close through the same reader.
read_websocket_frame() keeps its one-shot meaning.
write_websocket_frame() writes the header with the payload's first chunk
in one write, and WebSocket connections set TCP_NODELAY unless
CPPHTTPLIB_WEBSOCKET_TCP_NODELAY is false: on Linux, small messages each
way otherwise waited on Nagle's algorithm for the peer's delayed ACK, a
few in fifty around 30-40 ms late.
ClientImpl::redirect() resolved a Location with resolve_relative_location(), which hands a value with a ':' in its first segment back unchanged as having a scheme, and then parsed it with parse_url(), which also accepts a bare host[:port] so that Client("localhost:8080") works. A Location such as "internal:8080" or "127.0.0.1:6379" therefore came out with an empty scheme and a host, slipped past the http/https check, and was followed as a cross-host redirect to that host and port. A value such as "localhost:8080/x" was sent to the same host as a path without a leading '/'.
Follow only http/https URLs and references starting with '/', and refuse anything else, as an unsupported scheme already was. Network-path references such as "//host:port/x" are still followed.
A read() assembling a fragmented message looped over continuation
frames without looking at closed_, which the loop over frames checks.
A peer that kept sending fragments, empty ones included, never let the
read reach a wait, so a close() from another thread could not end it.
The read now ends with Fail once the connection is closed.
stop() takes the listening socket away, and listen() then waits for every
handler to return. A WebSocket handler waiting in read() on a client that
stays connected never learned of the stop: with no read timeout it waited
for the 300 s backstop, and with one it got Timeout back on an open
connection, so a `while (ws.read(msg) != Fail)` loop went on forever and
listen() with it.
The server now hands each WebSocket a predicate for its own stop, and the
WebSocket's read interrupt answers to it as it does to a pong timeout. A
read cut short while the server stops, between messages or in the middle
of a fragmented one, sends a Close with 1001 Going Away and returns Fail.
That close does not wait for the peer's reply, so a stop is not held up
per connection.
A server WebSocket's reads now wait in the interruptible slices whatever
the heartbeat settings, as keep-alive connections already do for the same
check.
* Let ClientImpl::stop() end a connect or a TLS handshake
stop() took socket_mutex_, which send_() and open_stream() hold while
they open the connection, so a stop() made during a connect, a TLS
handshake or the CONNECT of a proxy tunnel waited for the connection
to open or to time out, 300 seconds by default, for a server that
never answers.
stop() no longer waits for the mutex while a connection opens: it
cancels the connection phase by a compare-exchange from Connecting,
and the end of the phase reads the cancel in the same exchange that
takes the state back to Idle, so a cancel stop() accepts is never
lost. stop() also sets stop_requested_: the wait of the connect and
the waits of the handshake, in every TLS backend, poll it in slices of
100 ms, as do the reads of the CONNECT of a proxy tunnel. A request or
a stream canceled there reports Error::Canceled, before it sends
anything or takes the socket. A request opened after stop() clears the
flag as before, and a stream now does too. A negative connection
timeout still waits without end in the interruptible connect wait.
* Add ClientImpl::set_interrupt(), a stop the caller holds
stop() ends what is in flight, and a request or a stream opened after
it clears the flag, so a stop() made just before a request goes out is
lost.
set_interrupt() gives the client a predicate its caller owns, polled
wherever stop_requested_ is: the connect, the TLS handshake, the
CONNECT of a proxy tunnel and the reads of a request, and the head of
a stream too. The client of a cross-host redirect keeps it. A request
or a stream refuses to go out once it holds, so a stop the caller
holds is never lost, whenever it comes. A call blocked in the system,
a send() of a large body or a TLS read inside a record, still ends
through stop() alone, which shuts the socket down.
A timeout past what the clock reaches now waits without end instead of
expiring at once, so such a wait still ends on the interrupt.
* test: create the backlog sockets with SOCK_NONBLOCK
detail::set_nonblocking lives in httplib.cc in the split build, so
test_split did not compile. The test only runs on Linux, where
SOCK_NONBLOCK makes the socket nonblocking at creation.
* Open the connection without socket_mutex_ instead
Rework the change so that send_() and open_stream_() open the
connection without holding socket_mutex_, as requests are already
serialized by request_mutex_. stop() then never waits for a connection
being opened: it always sets stop_requested_, which the connect wait,
the TLS handshake, the CONNECT and the wait for the head of a stream
poll, and the call ends with Error::Canceled before it sends anything.
This replaces the connection state machine and the try_lock loop in
stop().
The flag is cleared once, as a send() or an open_stream() starts, so a
retry inside the call no longer drops a stop() it already got.
Drop set_interrupt() for now: a predicate held per client cannot be
shared, e.g. by an SSEClient that borrows the caller's Client. Also drop
StopEndsATlsReadInsideARecord, which checked existing behavior that
does not hold on Windows, and share the listener setup of the tests.
Keep the deadline_after() fix: a timeout past what the clock reaches
waits forever instead of overflowing into an immediate timeout.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* drop a passed-through Content-Length when the client reframes the body
send_with_content_provider_and_receiver() kept a Content-Length the
caller passed through after the library reframed the body: it compresses
a buffered body under set_compress(true), or streams a provider without a
length as chunked. prepare_default_headers() only adds a Content-Length
when none is present, so the stale value went out describing the original
body. With compression the server reads only that many bytes of the
longer gzip stream and the rest desyncs the next request on a kept-alive
connection; with a chunked provider the client emits both Content-Length
and Transfer-Encoding. Erase it once the library owns the framing so
write_request() emits the matching length, or none for a chunked body.
A plain, uncompressed body keeps the caller's value.
* Document the dropped Content-Length and tidy the new tests
Note in the README that a per-request Content-Length is dropped when the library compresses or chunks the body; a default header is not. Shorten the comments, use HOST like the neighbouring tests, and start the chunked test's Content-Length count at a value the handler has to overwrite.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
gen-certs.sh and test/CMakeLists.txt chose between -x509v1 and -x509 by the
openssl version. When the default openssl changed (macOS runner image
20261112), the root was generated as v3 without basicConstraints, openssl
rejected it as 'invalid CA certificate', and every client certificate test
failed. Add basicConstraints CA:TRUE to test.rootCA.conf, drop the version
branching, and verify the client certs against the root in gen-certs.sh.
close() waited for the peer's Close reply with a read timeout that
applied to each read, so a peer that kept sending other frames, or sent
one frame a byte at a time, held close() for as long as it liked.
Bound the wait as a whole. close() sets the stream's max timeout, which
shortens each wait to what is left, and stops reading frames at the
deadline, which matters when the peer keeps the socket full and no wait
happens at all. WebSocketSSLStream gains set_max_timeout(), the one
stream a WebSocket uses that lacked it.
A read() running on another thread when close() is called still waits
for the reply up to its own read timeout, as documented. The docs now
say how to bound it.
Server::process_and_close_socket() half-closes the socket and drains what
the peer is still sending before the final close (#2534), so that a close
with unread data, or with bytes still in flight, is not answered with a
RST. A RST can make the peer's stack discard a response it has received
but not yet read, as Windows does.
SSLServer::process_and_close_socket() still closed outright. After a
completed handshake, shut down TLS as before (sending close_notify when
the exchange completed) and then drain the same way. A failed handshake
closes outright as before.
Fix#2672
A kept-alive connection was reused while bytes that belong to no
request were waiting on it, and the next request read them as its
response. Such bytes are not a response (RFC 9112 9.2). A server may,
for one, send a 408 before it closes an idle connection.
Close the connection when they show up: after a response, if bytes
past it are already buffered in the stream or the TLS library, and
before reuse, if anything is readable on the socket. As in curl, the
reuse check no longer asks the TLS library whether the peer closed, so
it is the same for every TLS backend. A TLS message on an idle
connection, such as a late session ticket, now costs a new connection.
SSLClient::setup_proxy_connection() left the socket open when the TLS
handshake inside the CONNECT tunnel failed, so the client kept a
connection with no TLS session on it. Only the next request's reuse
check closed it, because tls::is_peer_closed() reports a missing
session as closed. Close it where the handshake fails, as
ensure_socket_connection() does for a direct connection.
* take the request lock in open_stream_ before reusing the socket
* Hold the request lock only while open_stream_ takes the socket
The handle owns the connection once the socket is transferred to it, so
holding request_mutex_ while the stream wrote its request and read the
response head only made requests on other threads wait for the stream.
Also shorten the comments.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* resolve junctions and symlinks in canonicalize_path on Windows
* Shorten the canonicalize_path comment
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
Fuzz Server::handle_file_request through a mount point backed by a temporary directory that holds symlinks escaping the mount root, one of them into a sibling directory sharing its name prefix. Abort if a token stored outside the root appears in a response. The seeds are based on those of #2661.
Move FuzzedStream into fuzzed_stream.h so that both server fuzzers share it, and add FuzzableServer::MarkAsRunning(), without which a server that is not listening writes no file.
Feed the event stream to the parser in chunks and abort if the reported events, ids and retries differ from parsing it in one piece. Also run with a length limit to cover the early-stop path.
Add websocket_fuzzer covering frame reading, UTF-8 and close payload validation, and the upgrade response parser. Add websocket_fuzzer.options to raise max_len above the 64 KiB frame length boundary.
Move the SSE parsing tests from a fixture that started a server and an
SSEClient for every case to unit tests of detail::SSEParser, and turn
NonDigitRetryFieldIgnored, which slept for 500ms, into one of them. Add
tests that feed each stream split at every position and byte by byte,
and that cover the max event length and an event left incomplete at the
end of the stream.
Move the line splitting and event assembly out of
SSEClient::run_event_loop() into a parser that is fed chunks and reports
events, ids and retry values through handlers. The event loop is left
with I/O and reconnection, and the parser can be driven without a
server, which the following test and fuzzer changes rely on.
No behavior change.
ThreadPool::enqueue constructed the dynamic thread without handling
std::system_error. When thread creation failed (e.g. EAGAIN under a
pids limit), the exception escaped the accept loop and destroying the
pool's joinable threads called std::terminate. Treat the failure as
"no spare capacity" and leave the job for an existing worker.
A client using set_address_family(AF_UNIX) took the host of an absolute Location as a socket path, so a redirect to http://localhost/next failed and one to http://other.sock/next went to ./other.sock. Every hop now stays on the client's socket, as with curl's --unix-socket.
The same host is still decided from the Location, as curl does: only localhost (the Host this client sends) counts as this client, so a redirect to any other host goes through the cross-host client and drops Authorization and Cookie. That client reaches the original socket through addr_map_, and a host the caller mapped with set_direct_hostname_addr_map keeps its mapping. An https Location runs TLS over the socket with the Location's host.
The mapped-target lookup shared with make_connect_request_target is factored into detail::addr_map_target.
A request sent with send(Request) and params got those params appended
to the Location of a redirect that had no query, on the same origin or
another one. The Location alone names the next target, as it already
did for Get/Post(path, params).
RFC 9110 14.2 says an origin server MUST ignore a Range header field
with a range unit it does not understand. parse_range_header() accepts
only bytes=, so a request such as "Range: items=1-2" got a 416 instead
of the full response. The unit is now checked first, and only a bytes
range is parsed and answered with a 416 when it is invalid.
* Cap a send() or wolfSSL_read() length at INT_MAX
On 64-bit Windows SocketStream::write() handed size to send(), whose
length is an int, so a single write of 4 GiB or more went out as its low
32 bits: a 4 GiB + 1 byte write sent one byte, and write_data() then
retried the remaining 4 GiB as zero-byte sends forever. The INT_MAX cap
that read() has was limited to 32-bit Windows in df20c27 (#1294), when
the body was still written with one unchecked write(); write_data() now
sends the rest, so the cap applies to every Windows build again.
The wolfSSL tls::read() had the same cast, so a read into a buffer over
INT_MAX failed. Cap it as the OpenSSL backend has done since f80864c.
* Drop the 4 GiB write test and shorten comments
The Windows-only test allocated 4 GiB and took about 5 s, which is too
heavy for CI.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* keep a relative Location on its host in resolve_relative_location
* Shorten the comment on on_this_host in resolve_relative_location
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
stop() shuts the socket down to end a request in flight, but on Windows
that does not wake a read waiting in WSAPoll (#1396). The request then
waited until the server answered, the read timeout passed, or the OS
dropped the connection.
stop() now also sets a flag, and send_() gives the request's stream an
interrupt check that reads it, as SSEClient::stop() does since 801bf2e.
The wait for data then ends within 100 ms on every platform. A request
started after stop() clears the flag. A redirect or a digest retry sent
inside the request keeps it.
select_read_interruptible() now rounds its last wait up to a whole
millisecond. poll() takes milliseconds, and the wait ended up to 1 ms
before the deadline, which MaxTimeoutTest caught once every client
request waited through this function.
ClientImpl and SSLClient let callers set origin credentials with
set_basic_auth()/set_bearer_token_auth(), but WebSocketClient could only
take a fixed Authorization header through its constructor. Add both
setters: each connect() adds the Authorization header to the upgrade
request from the current credentials (Basic over Bearer), unless the
constructor headers already carry one, and never to the proxy's CONNECT
request.
Through a CONNECT proxy it is the proxy that connects to the origin, and the CONNECT target was always the original hostname, so there was no way to send the tunnel to another address. set_tunnel_hostname_addr_map() sets that target on SSLClient, Client and WebSocketClient. The CONNECT request's Host follows it, including the Digest retry, while the Host header, the SNI and the certificate check inside the tunnel keep the original hostname. The map is also copied to a redirect client.
set_hostname_addr_map() is deprecated in favor of set_direct_hostname_addr_map(), which behaves the same and names the connection it applies to. The two maps are independent and correspond to curl's --resolve and --connect-to; neither applies to the proxy's own address.
Mbed TLS and wolfSSL read a TLS EOF without close_notify as a clean end, so a response delimited by the connection close could be cut short and still come back complete. Both reads now end cleanly only on a close_notify, as on OpenSSL.