A read() assembling a fragmented message looped over continuation
frames without looking at closed_, which the loop over frames checks.
A peer that kept sending fragments, empty ones included, never let the
read reach a wait, so a close() from another thread could not end it.
The read now ends with Fail once the connection is closed.
stop() takes the listening socket away, and listen() then waits for every
handler to return. A WebSocket handler waiting in read() on a client that
stays connected never learned of the stop: with no read timeout it waited
for the 300 s backstop, and with one it got Timeout back on an open
connection, so a `while (ws.read(msg) != Fail)` loop went on forever and
listen() with it.
The server now hands each WebSocket a predicate for its own stop, and the
WebSocket's read interrupt answers to it as it does to a pong timeout. A
read cut short while the server stops, between messages or in the middle
of a fragmented one, sends a Close with 1001 Going Away and returns Fail.
That close does not wait for the peer's reply, so a stop is not held up
per connection.
A server WebSocket's reads now wait in the interruptible slices whatever
the heartbeat settings, as keep-alive connections already do for the same
check.
* Let ClientImpl::stop() end a connect or a TLS handshake
stop() took socket_mutex_, which send_() and open_stream() hold while
they open the connection, so a stop() made during a connect, a TLS
handshake or the CONNECT of a proxy tunnel waited for the connection
to open or to time out, 300 seconds by default, for a server that
never answers.
stop() no longer waits for the mutex while a connection opens: it
cancels the connection phase by a compare-exchange from Connecting,
and the end of the phase reads the cancel in the same exchange that
takes the state back to Idle, so a cancel stop() accepts is never
lost. stop() also sets stop_requested_: the wait of the connect and
the waits of the handshake, in every TLS backend, poll it in slices of
100 ms, as do the reads of the CONNECT of a proxy tunnel. A request or
a stream canceled there reports Error::Canceled, before it sends
anything or takes the socket. A request opened after stop() clears the
flag as before, and a stream now does too. A negative connection
timeout still waits without end in the interruptible connect wait.
* Add ClientImpl::set_interrupt(), a stop the caller holds
stop() ends what is in flight, and a request or a stream opened after
it clears the flag, so a stop() made just before a request goes out is
lost.
set_interrupt() gives the client a predicate its caller owns, polled
wherever stop_requested_ is: the connect, the TLS handshake, the
CONNECT of a proxy tunnel and the reads of a request, and the head of
a stream too. The client of a cross-host redirect keeps it. A request
or a stream refuses to go out once it holds, so a stop the caller
holds is never lost, whenever it comes. A call blocked in the system,
a send() of a large body or a TLS read inside a record, still ends
through stop() alone, which shuts the socket down.
A timeout past what the clock reaches now waits without end instead of
expiring at once, so such a wait still ends on the interrupt.
* test: create the backlog sockets with SOCK_NONBLOCK
detail::set_nonblocking lives in httplib.cc in the split build, so
test_split did not compile. The test only runs on Linux, where
SOCK_NONBLOCK makes the socket nonblocking at creation.
* Open the connection without socket_mutex_ instead
Rework the change so that send_() and open_stream_() open the
connection without holding socket_mutex_, as requests are already
serialized by request_mutex_. stop() then never waits for a connection
being opened: it always sets stop_requested_, which the connect wait,
the TLS handshake, the CONNECT and the wait for the head of a stream
poll, and the call ends with Error::Canceled before it sends anything.
This replaces the connection state machine and the try_lock loop in
stop().
The flag is cleared once, as a send() or an open_stream() starts, so a
retry inside the call no longer drops a stop() it already got.
Drop set_interrupt() for now: a predicate held per client cannot be
shared, e.g. by an SSEClient that borrows the caller's Client. Also drop
StopEndsATlsReadInsideARecord, which checked existing behavior that
does not hold on Windows, and share the listener setup of the tests.
Keep the deadline_after() fix: a timeout past what the clock reaches
waits forever instead of overflowing into an immediate timeout.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* drop a passed-through Content-Length when the client reframes the body
send_with_content_provider_and_receiver() kept a Content-Length the
caller passed through after the library reframed the body: it compresses
a buffered body under set_compress(true), or streams a provider without a
length as chunked. prepare_default_headers() only adds a Content-Length
when none is present, so the stale value went out describing the original
body. With compression the server reads only that many bytes of the
longer gzip stream and the rest desyncs the next request on a kept-alive
connection; with a chunked provider the client emits both Content-Length
and Transfer-Encoding. Erase it once the library owns the framing so
write_request() emits the matching length, or none for a chunked body.
A plain, uncompressed body keeps the caller's value.
* Document the dropped Content-Length and tidy the new tests
Note in the README that a per-request Content-Length is dropped when the library compresses or chunks the body; a default header is not. Shorten the comments, use HOST like the neighbouring tests, and start the chunked test's Content-Length count at a value the handler has to overwrite.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
gen-certs.sh and test/CMakeLists.txt chose between -x509v1 and -x509 by the
openssl version. When the default openssl changed (macOS runner image
20261112), the root was generated as v3 without basicConstraints, openssl
rejected it as 'invalid CA certificate', and every client certificate test
failed. Add basicConstraints CA:TRUE to test.rootCA.conf, drop the version
branching, and verify the client certs against the root in gen-certs.sh.
close() waited for the peer's Close reply with a read timeout that
applied to each read, so a peer that kept sending other frames, or sent
one frame a byte at a time, held close() for as long as it liked.
Bound the wait as a whole. close() sets the stream's max timeout, which
shortens each wait to what is left, and stops reading frames at the
deadline, which matters when the peer keeps the socket full and no wait
happens at all. WebSocketSSLStream gains set_max_timeout(), the one
stream a WebSocket uses that lacked it.
A read() running on another thread when close() is called still waits
for the reply up to its own read timeout, as documented. The docs now
say how to bound it.
Server::process_and_close_socket() half-closes the socket and drains what
the peer is still sending before the final close (#2534), so that a close
with unread data, or with bytes still in flight, is not answered with a
RST. A RST can make the peer's stack discard a response it has received
but not yet read, as Windows does.
SSLServer::process_and_close_socket() still closed outright. After a
completed handshake, shut down TLS as before (sending close_notify when
the exchange completed) and then drain the same way. A failed handshake
closes outright as before.
Fix#2672
A kept-alive connection was reused while bytes that belong to no
request were waiting on it, and the next request read them as its
response. Such bytes are not a response (RFC 9112 9.2). A server may,
for one, send a 408 before it closes an idle connection.
Close the connection when they show up: after a response, if bytes
past it are already buffered in the stream or the TLS library, and
before reuse, if anything is readable on the socket. As in curl, the
reuse check no longer asks the TLS library whether the peer closed, so
it is the same for every TLS backend. A TLS message on an idle
connection, such as a late session ticket, now costs a new connection.
SSLClient::setup_proxy_connection() left the socket open when the TLS
handshake inside the CONNECT tunnel failed, so the client kept a
connection with no TLS session on it. Only the next request's reuse
check closed it, because tls::is_peer_closed() reports a missing
session as closed. Close it where the handshake fails, as
ensure_socket_connection() does for a direct connection.
* take the request lock in open_stream_ before reusing the socket
* Hold the request lock only while open_stream_ takes the socket
The handle owns the connection once the socket is transferred to it, so
holding request_mutex_ while the stream wrote its request and read the
response head only made requests on other threads wait for the stream.
Also shorten the comments.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* resolve junctions and symlinks in canonicalize_path on Windows
* Shorten the canonicalize_path comment
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
Fuzz Server::handle_file_request through a mount point backed by a temporary directory that holds symlinks escaping the mount root, one of them into a sibling directory sharing its name prefix. Abort if a token stored outside the root appears in a response. The seeds are based on those of #2661.
Move FuzzedStream into fuzzed_stream.h so that both server fuzzers share it, and add FuzzableServer::MarkAsRunning(), without which a server that is not listening writes no file.
Feed the event stream to the parser in chunks and abort if the reported events, ids and retries differ from parsing it in one piece. Also run with a length limit to cover the early-stop path.
Add websocket_fuzzer covering frame reading, UTF-8 and close payload validation, and the upgrade response parser. Add websocket_fuzzer.options to raise max_len above the 64 KiB frame length boundary.
Move the SSE parsing tests from a fixture that started a server and an
SSEClient for every case to unit tests of detail::SSEParser, and turn
NonDigitRetryFieldIgnored, which slept for 500ms, into one of them. Add
tests that feed each stream split at every position and byte by byte,
and that cover the max event length and an event left incomplete at the
end of the stream.
Move the line splitting and event assembly out of
SSEClient::run_event_loop() into a parser that is fed chunks and reports
events, ids and retry values through handlers. The event loop is left
with I/O and reconnection, and the parser can be driven without a
server, which the following test and fuzzer changes rely on.
No behavior change.
ThreadPool::enqueue constructed the dynamic thread without handling
std::system_error. When thread creation failed (e.g. EAGAIN under a
pids limit), the exception escaped the accept loop and destroying the
pool's joinable threads called std::terminate. Treat the failure as
"no spare capacity" and leave the job for an existing worker.
A client using set_address_family(AF_UNIX) took the host of an absolute Location as a socket path, so a redirect to http://localhost/next failed and one to http://other.sock/next went to ./other.sock. Every hop now stays on the client's socket, as with curl's --unix-socket.
The same host is still decided from the Location, as curl does: only localhost (the Host this client sends) counts as this client, so a redirect to any other host goes through the cross-host client and drops Authorization and Cookie. That client reaches the original socket through addr_map_, and a host the caller mapped with set_direct_hostname_addr_map keeps its mapping. An https Location runs TLS over the socket with the Location's host.
The mapped-target lookup shared with make_connect_request_target is factored into detail::addr_map_target.
A request sent with send(Request) and params got those params appended
to the Location of a redirect that had no query, on the same origin or
another one. The Location alone names the next target, as it already
did for Get/Post(path, params).
RFC 9110 14.2 says an origin server MUST ignore a Range header field
with a range unit it does not understand. parse_range_header() accepts
only bytes=, so a request such as "Range: items=1-2" got a 416 instead
of the full response. The unit is now checked first, and only a bytes
range is parsed and answered with a 416 when it is invalid.
* Cap a send() or wolfSSL_read() length at INT_MAX
On 64-bit Windows SocketStream::write() handed size to send(), whose
length is an int, so a single write of 4 GiB or more went out as its low
32 bits: a 4 GiB + 1 byte write sent one byte, and write_data() then
retried the remaining 4 GiB as zero-byte sends forever. The INT_MAX cap
that read() has was limited to 32-bit Windows in df20c27 (#1294), when
the body was still written with one unchecked write(); write_data() now
sends the rest, so the cap applies to every Windows build again.
The wolfSSL tls::read() had the same cast, so a read into a buffer over
INT_MAX failed. Cap it as the OpenSSL backend has done since f80864c.
* Drop the 4 GiB write test and shorten comments
The Windows-only test allocated 4 GiB and took about 5 s, which is too
heavy for CI.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* keep a relative Location on its host in resolve_relative_location
* Shorten the comment on on_this_host in resolve_relative_location
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
stop() shuts the socket down to end a request in flight, but on Windows
that does not wake a read waiting in WSAPoll (#1396). The request then
waited until the server answered, the read timeout passed, or the OS
dropped the connection.
stop() now also sets a flag, and send_() gives the request's stream an
interrupt check that reads it, as SSEClient::stop() does since 801bf2e.
The wait for data then ends within 100 ms on every platform. A request
started after stop() clears the flag. A redirect or a digest retry sent
inside the request keeps it.
select_read_interruptible() now rounds its last wait up to a whole
millisecond. poll() takes milliseconds, and the wait ended up to 1 ms
before the deadline, which MaxTimeoutTest caught once every client
request waited through this function.
ClientImpl and SSLClient let callers set origin credentials with
set_basic_auth()/set_bearer_token_auth(), but WebSocketClient could only
take a fixed Authorization header through its constructor. Add both
setters: each connect() adds the Authorization header to the upgrade
request from the current credentials (Basic over Bearer), unless the
constructor headers already carry one, and never to the proxy's CONNECT
request.
Through a CONNECT proxy it is the proxy that connects to the origin, and the CONNECT target was always the original hostname, so there was no way to send the tunnel to another address. set_tunnel_hostname_addr_map() sets that target on SSLClient, Client and WebSocketClient. The CONNECT request's Host follows it, including the Digest retry, while the Host header, the SNI and the certificate check inside the tunnel keep the original hostname. The map is also copied to a redirect client.
set_hostname_addr_map() is deprecated in favor of set_direct_hostname_addr_map(), which behaves the same and names the connection it applies to. The two maps are independent and correspond to curl's --resolve and --connect-to; neither applies to the proxy's own address.
Mbed TLS and wolfSSL read a TLS EOF without close_notify as a clean end, so a response delimited by the connection close could be cut short and still come back complete. Both reads now end cleanly only on a close_notify, as on OpenSSL.
On Windows a ':' ends a drive prefix or starts an NTFS stream name, so a filename such as "C:.." or "a.txt:stream" survived sanitize_filename and could replace the upload directory or write to an alternate data stream. Cut at ':' on Windows as well.
resolve_relative_location removed "." and ".." segments only from a relative-path Location. A Location that starts with "/" or an absolute URI was requested as is, so a redirect to /dir/../target went out as /dir/../target and a server that does not normalize the path answered 404.
RFC 3986 section 5.2.2 applies remove_dot_segments to the target path of every reference that has a path, and Go and urllib3 do the same. The loop now lives in detail::remove_dot_segments, and ClientImpl::redirect applies it to an absolute path as well. Relative locations resolve as before.
On NTFS a ':' in a path names a stream of the file, so /dir/test.html::$DATA opened the default stream of dir/test.html and passed a set_pre_routing_handler() guard on /dir/test.html, and /setup.exe:Zone.Identifier served the stream Windows attaches to downloaded files. is_valid_path() now refuses a ':' on Windows.
Elsewhere a ':' is an ordinary file name character, so it is still served: a file such as 2024-01-01T10:00:00.log would otherwise answer 404. The test pins both sides.
last_event_id() returned a reference to a string that run_event_loop() assigns on its own thread under start_async(), so a caller reading it from another thread raced the writes. The member is now guarded by a mutex and the getter returns a copy.
The return type changes from const std::string & to std::string. Code that binds the result to a const reference or copies it still compiles.
write_websocket_frame and the request-body write in open_stream_ called
Stream::write once and only failed on a negative return, so a short
write (one record per mbedtls_ssl_write call, or SO_SNDTIMEO expiring
after part of the data went out) dropped the rest of the frame or body.
Route both through write_data, as the other body writers do.
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
The macOS system CA loader took every certificate that has Trust Settings, in the system, admin or user domain, as a trusted root without reading those settings. A root set to "Never Trust" in Keychain Access, or trusted only for S/MIME or code signing, still verified TLS servers.
The loader now evaluates the settings for the SSL policy. A per-user setting overrides the admin domain, which overrides the system domain, and only TrustRoot and TrustAsRoot are loaded. The OpenSSL path shares the helper that the Mbed TLS and wolfSSL paths already use.
open_stream_() created a decoder whatever set_decompress() said, so a response stream decoded the body when decompression was disabled, and failed with UnsupportedContentEncoding in a build without that decoder. The buffered path already checks the option; the streaming path now does too. The new test covers open_stream() and stream::Get().
Fixes#2639.
OpenSSL 4.1 no longer falls back to the subject Common Name when it
checks a host name (RFC 9525), so SSLClientServerTest.TlsVerifyHostname
began failing in the Windows CI job that installs the newest OpenSSL 4.x.
Make every backend agree instead of following whichever OpenSSL is
installed. OpenSSL passes X509_CHECK_FLAG_NEVER_CHECK_SUBJECT to both the
handshake check and verify_hostname(), and Mbed TLS and wolfSSL drop their
own CN fallback. A certificate that names its host only in the CN is now
rejected on all backends, as it already was on BoringSSL.
is_redirect treated any 3xx with a Location as a redirect, so with set_follow_location(true) a 304 or 305 that carried a Location was followed. Only 301, 302, 303, 307 and 308 are redirects a client follows (RFC 9110 section 15.4), as in Go, requests and urllib3. A Location is still required. The 304 check in process_request that is_redirect now covers is dropped.
Without keep-alive, the client sends "Connection: close", but before
following a redirect it disconnected only when the response repeated
the option. RFC 9112 (section 9.6) lets a server close the connection
without repeating it, and a client that sent "close" must not send
another request on that connection. The redirected request was written
to the closing connection and failed with "Failed to read connection"
whenever the server closed it first. The client now disconnects before
a redirect or an auth retry whenever the request asked for the close.
Co-authored-by: Wleeaf <179125857+wleeaf@users.noreply.github.com>
apt on the runner mirror sometimes crawls for minutes
(actions/runner-images#14823), and the cross compiler is large enough
that its install alone used up the job's 10-minute limit. Loosen the job
limit to 30 minutes and limit the build step, which normally takes a few
seconds, to 2 minutes.
apt on the runner mirror sometimes crawls for minutes
(actions/runner-images#14823), which used up the job's 5-minute limit
before the test finished. Loosen the job limit to 15 minutes and limit
the test step, which normally takes about 80s, to 3 minutes.
In absolute-form the path follows the authority, so a path such as "@evil.example/x" made the proxy connect to another host and forward the origin's credentials. Both handle_request and open_stream_ now refuse such a path before anything is written.
Windows trims a trailing '.' or space from a file name, so a mount
point there served a file under a path other than the one a pre-routing
handler checked. A mount point now serves no path with a segment
ending in '.' or a space, on every platform; "." and ".." are among
them. The README no longer lists such names among the ones the file
system can still alias.
After answering, with a status below 400, a request whose body the
handler left unread, the server read past the body to reuse the
connection up to the payload limit (100MB by default), with only each
read bounded by the read timeout, so a slow sender could hold a worker
for a long time. It now skips at most
CPPHTTPLIB_UNREAD_PAYLOAD_MAX_LENGTH (256KB, as Go's net/http does)
within the read timeout as a whole, and closes the connection
otherwise. An error response already closed the connection.
set_file_extension_and_mimetype_mapping() kept the extension as given,
and a file's extension was looked up as it was, then in lowercase, so a
mapping given as ".pdf" never matched and one given as "PDF" matched
only files ending in ".PDF". The extension is now lowercased and loses
a leading '.' when it is set, as Apache's AddType does, and a file's
extension is looked up in lowercase only.
An exception handler that set no status sent its response with 200, or
with whatever status the throwing handler had set. The status is now
set to 500 before the exception handler runs, as ASP.NET Core does, so
it stays 500 unless the exception handler sets another one.
On a WebSocket upgrade, an exception from the pre-routing handler, the
pre-request handler or the sub-protocol selector skipped the exception
handler and dropped the connection without a response. Everything
before the protocol switches now goes through the same exception
handling as routing, so it is answered over HTTP. The WebSocket handler
itself still runs outside it.
The test was kept off Windows while reading the body of such a request
waited on the socket and could hang there. That wait was made
non-blocking soon after, and such a request no longer has its body
read at all, so nothing in it is specific to the platform any more.
A request with neither Content-Length nor chunked Transfer-Encoding
has no body (RFC 9112 6.3), and builds with TLS support read none.
Builds without it still read whatever followed such a request on a
connection about to close, up to EOF, as its body. Every build now
reads no body. Tests that exercised only that read, or passed whatever
the server did, are removed, and a new one checks in every build that
the bytes after such a request are not taken as its body.
A request whose Transfer-Encoding ended in chunked but listed more
(another coding before chunked, or chunked twice) was read with only
the chunked framing removed, so the handler got a body still in the
other coding. RFC 9112 6.1 lets a server answer 501 to a transfer
coding it does not implement. The server now accepts only a single
Transfer-Encoding field whose value is chunked, as Go and nginx do,
and answers any other list ending in chunked with 501 and closes the
connection. A list whose final coding is not chunked still gets 400.
RFC 9110 14.2 defines range handling only for GET, and a server must
ignore a Range header on any other method. The server parsed it on
every method, so a POST with a Range got a 206 slice of the handler's
response, or a 416 when the Range was malformed. Range is now parsed
only for GET and HEAD; other methods get the whole response, and a
handler can still read the header itself.
set_trusted_proxies() compared addresses as strings, so a proxy listed
as 10.0.0.2 was not trusted when a server listening on both IPv4 and
IPv6 saw it as ::ffff:10.0.0.2, and two spellings of one IPv6 address
did not match either. The listed addresses are now put in one spelling
when they are set, and so are the connection peer and each
X-Forwarded-For hop before they are looked up; an IPv4-mapped IPv6
address becomes its IPv4 address. The README now documents the setter.
The README said a redirect keeps the TLS verification settings. They
pass only from one HTTPS hop to the next; an HTTPS hop after a plain
HTTP one starts over with the defaults.
RejectsChunkOverheadPastItsLimit sent about 78 KB per case, and the
server refused the request a few chunks in and closed the connection
while the client was still writing, so the send failed now and then
instead of returning the 400. Pad each size line with a sixteenth of
the limit, so each request is about 10 KB and goes past the limit only
at its last chunks.
TLS settings take effect as a connection is set up, so after a client
changed one (server certificate or host name verification, a CA store,
a verifier) a later request still went over the connection kept alive
under the old settings, unverified if verification had just been turned
on. Each such setter now bumps a version that the connection records
when it is set up, and a connection set up under an older version is
closed instead of reused. The setters take no lock, so they cannot wait
on a handshake in flight.
A server built with compression support compressed every handler
response the client accepted, with no way to turn it off. Over HTTPS
the length of a compressed response can give away a secret in it when
the same response also carries text an attacker chooses (BREACH).
set_compress(false) sends handler responses, buffered or streamed, as
they are; compression stays on by default, and static file compression
keeps its own setting. The README and the cookbook describe the switch
and when to use it.
tls::create_ca_store() on Mbed TLS parsed the caller's buffer with one
byte more than its length, so it read past the end of the buffer, and
when that byte was not a NUL the PEM was not recognized and the call
returned NULL. Parse a NUL-terminated copy instead, as load_ca_pem()
does. DER input still parses, since its own length ends it.
The server left response header fields with an invalid name or value,
such as a value carrying CR/LF that would split the response, for the
default header writer to skip, so a writer set with set_header_writer()
wrote them. Drop such fields from res.headers right after the handlers
that can fill it directly have run, before the server reads it for
framing and range decisions and before any header writer sees it. The
client already rejects invalid request headers before writing them.
The payload limit counts only chunk data, and a chunk-size line is only
bounded by the line length limit, so a peer could send a few bytes of
data with any amount of padding, leading zeros and chunk extensions
around them. Count those bytes as Go does: each chunk may spend 16
bytes plus twice its data on its size line and the CRLF after it, which
still lets a stream send a byte at a time, and the excess may build up
to CPPHTTPLIB_HEADER_MAX_LENGTH before the read fails. This covers
request bodies on the server and response bodies on the client.
The OpenSSL client finished the handshake with SSL_VERIFY_NONE and
checked the server certificate only afterwards, so it presented its
client certificate to a server that then failed verification. With a
server certificate verifier that accepted, wolfSSL did the same. Verify
the chain, and the host name, during the handshake on every backend,
keeping the built-in verdict when a verifier runs alongside it. On
OpenSSL a session verifier, which may accept a certificate the built-in
check rejects, still decides after the handshake.
A certificate that fails the built-in verification now reports
CertVerifyFailed, or HostnameMismatch for its name, in ssl_error() on
every backend, where OpenSSL left it 0.
tls::create_ca_store() parses certificates, which on Mbed TLS 4.x needs
PSA Crypto. A context readies it when it is created, but a store can be
made before any context exists, and then a valid PEM failed to parse
and the call returned NULL. Ready PSA Crypto first, as the contexts do.
Under the default system CA policy, a custom CA given to a client keeps
the system CAs out. SSLClient and WebSocketClient only noted the custom
CA when set_ca_cert_store() got a store or load_ca_cert_store() got some
bytes, so a NULL store, such as the one tls::create_ca_store() returns
for a PEM it cannot parse, or an empty PEM left the client trusting
every system CA instead. Note the custom CA on every call, so that such
a client fails verification. A redirect still passes a CA on only when
the original client had one.
A mount point left ".", ".." and empty path segments for the file
system to resolve, while the pre-routing handler, where the README has
applications guard static files, saw the path as it was sent. A file
could then be reached under a spelling that a check on req.path did not
match. Serve a path from a mount point only when it has none of those
segments other than a trailing '/', so that the file served is the one
req.path names; any other path now gets 404. The README also notes the
other names a file system can give a file.
SSLServer::update_certs_pem() loaded the client CA it was given but
left client certificate verification as it was, so on a server built
without a client CA it returned true while clients without a
certificate still got in. Turn verification on, as the constructors do
when they are given a client CA. A client CA passed to
update_certs_pem() now always makes client certificates required.
The Mbed TLS server keeps the SNI from the ClientHello in a
thread-local until the handshake succeeds, and its SNI callback only
runs for a ClientHello that carries one. A handshake that failed left
its SNI behind, so the next connection served by the same thread
without SNI reported the earlier client's name in req.sni(). Start
each handshake with the thread-local cleared.
SSLServer::PemMemory takes a length of 0 to mean a NUL-terminated
string for cert_pem and key_pem, but skipped client_ca_pem when its
length was 0, so a server given its client CA that way started without
requiring client certificates. Read the client CA under the same rule
as the other fields. An empty client CA now fails to load instead of
turning the check off, and the README documents the length-0 rule.
ClientImpl::open_stream built the body reader from Content-Length and Transfer-Encoding whatever the method or status, so a stream on a response that RFC 9112 6.3 ends at its header section was read on: with a keep-alive server read() blocked until the read timeout, and when the server closed it reported a read error for a complete response. stream::Head() against cpp-httplib's own server, which puts the GET body's Content-Length on a HEAD response, hit it every time.
open_stream_() now returns with the body reader at its end for a response to HEAD and for a 204 or 304, as the buffered path skips the body for them. Content-Encoding is skipped too, so a coding the build lacks no longer fails such a stream.
When pre_routing_handler or pre_request_handler answers a WebSocket upgrade request, Server::process_request returns before the drain of an unread body and closes the connection only for a 4xx/5xx. A 2xx/3xx answer to an upgrade request with a framed body left that body to be parsed as the next request on the keep-alive connection. Those two early returns now close the connection when the request has a framed body, as the Expect rejection already does.
2026-10-20 17:08:21 -04:00
41 changed files with 5002 additions and 1264 deletions
The server accepts a handshake from any `Origin`. A browser sends its cookies with a handshake another site starts, so if cookies authenticate the connection, also check `req.get_header_value("Origin")` against your own origins in one of these handlers (cross-site WebSocket hijacking).
The credentials are read on every `connect()`, so a new token takes effect on
the next reconnect. An `Authorization` header passed to the constructor wins
over them. Through a proxy they are sent only inside the tunnel, never on
`CONNECT`.
### Custom Headers and Timeouts
```cpp
@@ -547,7 +568,7 @@ Choose sizes that account for both your expected HTTP load and the maximum numbe
A single `WebSocket` (server-side) or `WebSocketClient` handle is shared by three potential callers: the thread running your handler (or holding the client), the heartbeat thread, and, if your code does its own thing, a separate thread calling `send()`/`close()` while another thread is blocked in `read()`.
**Supported**: calling `read()` from one thread while calling `send()`/`close()` from another. This is the common pattern for a client that reads incoming messages in a loop on one thread and sends from elsewhere (e.g. a UI thread). A message that is in flight when `close()` is called still arrives intact; `close()` sends the Close frame and returns, leaving the connection's read side to the thread that owns it, so it does not block waiting for the peer's Close reply in that case. The heartbeat thread's automatic pings use the same `send()` path internally, so they are safe to run concurrently with your `read()` loop too — for `wss://` this requires every TLS call on a connection to be serialized internally, which cpp-httplib does for you.
**Supported**: calling `read()` from one thread while calling `send()`/`close()` from another. This is the common pattern for a client that reads incoming messages in a loop on one thread and sends from elsewhere (e.g. a UI thread). A message that is in flight when `close()` is called still arrives intact; `close()` sends the Close frame and returns, leaving the connection's read side to the thread that owns it, so it does not block waiting for the peer's Close reply in that case. That thread's `read()` keeps waiting for the reply until its own read timeout (by default none on a client, 300 seconds on a server); to bound it, set one with `set_read_timeout()` before the read starts. The heartbeat thread's automatic pings use the same `send()` path internally, so they are safe to run concurrently with your `read()` loop too — for `wss://` this requires every TLS call on a connection to be serialized internally, which cpp-httplib does for you.
**Not supported**: calling `read()` from two threads at the same time on the same handle. The calls are serialized rather than left to corrupt each other, but which thread receives which message is unspecified, so there is nothing useful to build on it.
@@ -76,7 +76,10 @@ cpp-httplib supports multiple TLS backends through an abstraction layer:
> A wolfSSL server that verifies client certificates does not resume TLS sessions: wolfSSL shares its session cache between contexts, so a resumed session could carry a certificate checked against another server's client CA.
> [!NOTE]
> **BoringSSL (best-effort):** BoringSSL builds under `CPPHTTPLIB_OPENSSL_SUPPORT` and is exercised by CI against current upstream. Because BoringSSL does not guarantee API stability, support is best-effort — breakage may occasionally land. Two known behavioral differences vs OpenSSL: (1) BoringSSL's public headers require C++14 or later, so consumers must compile accordingly; (2) hostname verification is SAN-only per RFC 6125 §6.4.4 (no CN fallback).
> **BoringSSL (best-effort):** BoringSSL builds under `CPPHTTPLIB_OPENSSL_SUPPORT` and is exercised by CI against current upstream. Because BoringSSL does not guarantee API stability, support is best-effort — breakage may occasionally land. One known difference vs OpenSSL: BoringSSL's public headers require C++14 or later, so consumers must compile accordingly.
> [!NOTE]
> **Hostname verification is SAN-only on every backend** (RFC 9525). A host is matched against the certificate's subjectAltName entries, never its Common Name, so a certificate that names its host only in the CN is rejected. Reissue it with a subjectAltName.
> The CA certificates are loaded when the client makes its first connection. Changing `set_ca_cert_path()` or `enable_system_ca()` after that has no effect, so make the settings before the first request, or use a new client. `set_ca_cert_store()` and `load_ca_cert_store()` take effect at once. `WebSocketClient` works the same way.
> The CA certificates are loaded when the client makes its first connection. Changing `set_ca_cert_path()` or `enable_system_ca()` after that has no effect, so make the settings before the first request, or use a new client. `set_ca_cert_store()` and `load_ca_cert_store()` take effect at once. `WebSocketClient` works the same way. On `Client` and `SSLClient`, a setter that changes a TLS setting also closes a connection kept alive under the old one, so the next request connects again.
### SSL Error Handling
When SSL operations fail, cpp-httplib provides detailed error information through `ssl_error()` and `ssl_backend_error()`:
- `ssl_error()` - Returns the TLS-level error as a backend-independent `httplib::tls::ErrorCode` value (e.g., `Fatal`, `CertVerifyFailed`), cast to `int`. `CertVerifyFailed` comes from Mbed TLS and wolfSSL. OpenSSL checks the chain and the hostname after the handshake, so a failure there leaves this `0`
- `ssl_error()` - Returns the TLS-level error as a backend-independent `httplib::tls::ErrorCode` value (e.g., `Fatal`, `CertVerifyFailed`), cast to `int`. A server certificate that the built-in verification rejects reports `CertVerifyFailed`, or `HostnameMismatch` when only its name does not match
- `ssl_backend_error()` - Returns the backend-specific error code (e.g., `ERR_get_error()` for OpenSSL/wolfSSL, return value for Mbed TLS). With OpenSSL, a certificate verification failure reports the verify result (`X509_V_ERR_*`) here instead. On Windows, when CryptoAPI verifies the chain, a failure there reports the CryptoAPI error code here
A request that fails after the handshake with `Error::Read` or `Error::Write` also carries the TLS error behind it in both values, such as a TLS 1.3 "certificate required" alert from the server. Both values are reset for each request.
A length of 0 means the field is a NUL-terminated string.
`httplib::ws::WebSocketClient` has the same `PemMemory` constructor for `wss://` connections. See [README-websocket.md](README-websocket.md) for details.
### Peer Certificate Inspection
@@ -238,7 +243,7 @@ cpp-httplib automatically integrates with the OS certificate store on macOS and
| Platform | Behavior | Disable (compile time) |
| :------- | :------- | :--------------------- |
| macOS | Loads system certs from Keychain (link `CoreFoundation` and `Security` with `-framework`). Requires Apple Clang; GCC is not supported for this feature. | `CPPHTTPLIB_DISABLE_MACOSX_AUTOMATIC_ROOT_CERTIFICATES` |
| macOS | Loads system certs from Keychain, honouring their Trust Settings: a certificate set to "Never Trust", or trusted only for another policy such as S/MIME, is not loaded (link `CoreFoundation` and `Security` with `-framework`). Requires Apple Clang; GCC is not supported for this feature. | `CPPHTTPLIB_DISABLE_MACOSX_AUTOMATIC_ROOT_CERTIFICATES` |
| Windows | Verifies the certificate chain with CryptoAPI (`CertGetCertificateChain` / `CertVerifyCertificateChainPolicy`) instead of the TLS backend. Revocation checking is best-effort: a revoked server certificate is rejected, while one whose revocation status cannot be determined is accepted. Windows fetches missing roots and intermediates on demand. With a custom CA, the TLS backend verifies the chain instead; with `set_server_certificate_verifier()`, both do. | `CPPHTTPLIB_DISABLE_WINDOWS_AUTOMATIC_ROOT_CERTIFICATES_UPDATE` |
On Windows, the CryptoAPI check can also be turned off at runtime. The TLS backend then verifies the chain instead, as on other platforms:
@@ -404,6 +409,8 @@ ret = svr.remove_mount_point("/");
ret = svr.remove_mount_point("/public");
```
A mount point serves a path only when it has no `.`, `..` or empty segment other than a trailing `/`, no segment ending in `.` or a space and, on Windows, no `:` (it names an NTFS stream), so a check on `req.path` in `set_pre_routing_handler()` sees the path of the file that is served. A path ending in `/`, and the mount point itself such as `/public`, serve the `index.html` in that directory. The file system can still give a file other names: another case on a case-insensitive file system, such as the default ones on macOS and Windows, and on Windows an 8.3 short name such as `PRIVAT~1`. Compare paths without regard to case on such a file system, and on Windows keep files that need a guard out of mounted directories.
```cpp
// User defined file extension and MIME type mappings
Extensions match regardless of case, so `photo.JPG` is served as `image/jpeg`. A user mapping is looked up by the exact extension first, then by the extension in lowercase.
Extensions match regardless of case, so `photo.JPG` is served as `image/jpeg`. The extension of a user mapping can be given in any case, with or without a leading `.`.
The exception handler gets called if a user routing handler throws an error. This covers the pre-routing handler, the file request handler, the pre-request handler, route handlers and content readers. An exception from the post-routing handler, the error handler, the `Expect: 100-continue` handler, a content provider, a logger or a WebSocket handler does not reach it: the connection is dropped, and the error logger receives `Error::UserCallbackException`.
The exception handler gets called if a user routing handler throws an error. This covers the pre-routing handler, the file request handler, the pre-request handler, route handlers, content readers and, on a WebSocket upgrade, the sub-protocol selector. The response is a 500 unless the exception handler sets another status. An exception from the post-routing handler, the error handler, the `Expect: 100-continue` handler, a content provider, a logger or a WebSocket handler does not reach it: the connection is dropped, and the error logger receives `Error::UserCallbackException`.
auto safe = httplib::sanitize_filename(file.filename);
```
This function keeps only the part after the last `/` or `\`, removes null bytes, and trims leading and trailing spaces and tabs. It returns an empty string if what is left is empty, `.` or `..`.
This function keeps only the part after the last `/` or `\` (or, on Windows, `:`, which starts a drive prefix such as `C:` or an NTFS stream name), removes null bytes, and trims leading and trailing spaces and tabs. It returns an empty string if what is left is empty, `.` or `..`.
> [!NOTE]
> `name` and `filename` are read as quoted-strings (RFC 9110 §5.6.4), where `\"` and `\\` stand for `"` and `\`. Browsers send a backslash as it is, so from a browser, a value with `\\` in it arrives with a single `\`, and one that ends in `\` is not read correctly.
@@ -912,6 +919,14 @@ svr.set_keep_alive_max_count(2); // Default is 100, 0 for no limit
svr.set_keep_alive_timeout(10);// Default is 5
```
### Client address behind a reverse proxy
```cpp
svr.set_trusted_proxies({"10.0.0.2"});
```
A request that comes straight from one of these addresses gets `req.remote_addr` from `X-Forwarded-For`: the rightmost address in it that is not a trusted proxy. A request from anywhere else keeps the address of its connection, whatever `X-Forwarded-For` says. Addresses match as IP addresses, so an IPv4 peer that a server listening on both IPv4 and IPv6 sees as `::ffff:10.0.0.2` matches `10.0.0.2`. Host names and CIDR ranges are not supported.
The read timeout bounds the request line and headers as a whole, and each read of a request body.
The read timeout bounds the request line and headers as a whole, and each read of a request body. An error response closes the connection. Any other response to a request whose body was left unread, such as a redirect from a pre-routing handler, keeps the connection only when the rest of the body is at most `CPPHTTPLIB_UNREAD_PAYLOAD_MAX_LENGTH` (256KB, `0` for no limit) and arrives within the read timeout, since the server has to read past it first.
### Set maximum payload length for reading a request body
@@ -1157,7 +1172,7 @@ cli.set_default_headers({
auto res = cli.Get("/hi");
```
A `Content-Length` among these headers is sent in place of the one the library would send, so it has to match the body as written. A `Transfer-Encoding` is sent in addition to the library's own framing, so leave it out. Leave both out when passing on the headers of another message, as a proxy does.
A `Content-Length` among these headers is sent in place of the one the library would send, so it has to match the body as written. One passed with the request is dropped when the library compresses the body or sends it chunked. A `Transfer-Encoding` is sent in addition to the library's own framing, so leave it out. Leave both out when passing on the headers of another message, as a proxy does.
### POST
@@ -1445,7 +1460,7 @@ res->status; // 200
A URL with userinfo (`http://user:pass@host/`) is not supported, neither for a `Client` nor in a `Location`; use `set_basic_auth()` for the credentials. A `Client` constructed from a URL it cannot use, such as one with userinfo, a backslash in the authority or an invalid port, is not valid: `is_valid()` returns `false`, and every request fails with `Error::Connection`.
On a redirect to another origin (a different scheme, host or port), `Authorization`, `Cookie` and `Proxy-Authorization` are dropped along with the credentials set with `set_basic_auth()`, `set_bearer_token_auth()` and `set_digest_auth()`, and the client certificate is not presented. Other headers, the default ones included, go along (RFC 9110 §15.4), and so do the timeouts, the payload limit and the TLS verification settings. A `303` that turns a request into a `GET` drops its headers and body; only on the same origin are the default headers sent again.
On a redirect to another origin (a different scheme, host or port), `Authorization`, `Cookie` and `Proxy-Authorization` are dropped along with the credentials set with `set_basic_auth()`, `set_bearer_token_auth()` and `set_digest_auth()`, and the client certificate is not presented. Other headers, the default ones included, go along (RFC 9110 §15.4), and so do the timeouts, the payload limit and the TLS verification settings. The TLS settings pass only from one HTTPS hop to the next: an HTTPS hop that follows a plain HTTP one starts over with the defaults, verifying against the system CAs with no verifier set. A `303` that turns a request into a `GET` drops its headers and body; only on the same origin are the default headers sent again.
Up to `CPPHTTPLIB_REDIRECT_MAX_COUNT` (20 by default) redirects are followed for a request; one more fails it with `Error::ExceedRedirectCount`. Interim `1xx` responses before the final response are skipped, up to `CPPHTTPLIB_INTERIM_RESPONSE_MAX_COUNT` (100 by default); one more fails the request with `Error::Read`.
@@ -1462,17 +1477,17 @@ The same method is available on `httplib::ws::WebSocketClient`.
### Override the connection target for a hostname
`set_hostname_addr_map` redirects where the socket connects, without changing
the identity of the request. The hostname the client was constructed with keeps
supplying the `Host` header, the SNI, and the name that the server certificate
is verified against, so this is a connection-level override only, not a way to
talk to a different origin.
`set_direct_hostname_addr_map` redirects where the socket connects, without
changing the identity of the request. The hostname the client was constructed
with keeps supplying the `Host` header, the SNI, and the name that the server
certificate is verified against, so this is a connection-level override only,
not a way to talk to a different origin.
```cpp
httplib::Client cli("https://example.com");
// Connect to this IP address instead of resolving "example.com"
The two maps are independent; to use the same target on both paths, set both.
They correspond to curl's `--resolve` and `--connect-to`.
Neither map applies to the proxy's own address ("direct" means not through a
proxy, as in a PAC file's `DIRECT`). To connect to the proxy at a fixed
address, pass an IP literal to `set_proxy()`.
`set_hostname_addr_map` is deprecated and works the same as
`set_direct_hostname_addr_map`.
These methods are also available on `httplib::ws::WebSocketClient`.
### Automatic Path Encoding
@@ -1586,6 +1625,8 @@ This holds for every kind of response, including the file-backed ones below.
`Vary: Accept-Encoding` is added only to responses the server encoded itself. A handler that chooses between an encoded and an identity representation by reading `Accept-Encoding` should set the field itself, so that shared caches keep the two apart.
Turn this compression off with `svr.set_compress(false)`. Over HTTPS, the length of a compressed response can give away a secret in it, such as a CSRF token, when the same response also carries text an attacker can choose (the BREACH attack). A server whose responses mix the two should turn compression off, or keep such secrets out of those responses. Static file compression, below, is set separately.
### Static file compression
Responses served from a file, whether through `set_mount_point()` or `Response::set_file_content()`, are sent as is by default. Turn compression on for them with:
This library automatically sets the Host header to "localhost" for Unix socket connections, similar to curl's behavior.
With `set_follow_location(true)`, a redirect is followed over the same socket, whatever host the `Location` names, as curl's `--unix-socket` does. A host mapped with `set_direct_hostname_addr_map` goes to its mapped socket instead. Only a `Location` naming `localhost` counts as the same host; any other host is treated as cross-host, so `Authorization` and `Cookie` are not sent to it.
Handler responses, buffered or streamed, are then sent as they are. Over HTTPS, the length of a compressed response can give away a secret in it, such as a CSRF token, when the same response also carries text an attacker can choose (the BREACH attack). Static file compression is set separately.
## Static files need to be opted in
Files served as they are, through `set_mount_point()` or `Response::set_file_content()`, are not compressed by default. Turn it on with:
@@ -56,4 +56,4 @@ When you call `stop()`, new connections are refused, but handlers that are alrea
> **Warning:** There's a wait between calling `stop()` and `listen()` returning — it's the time in-flight requests take to finish. To enforce a timeout, you'll need to add your own shutdown timer in application code.
> **Note:** `stop()` doesn't interrupt WebSocket handlers. A handler looping on `read()` keeps running as long as its peer stays connected, and `listen()` doesn't return until it ends. Keep track of open WebSocket connections and `close()` them on shutdown, e.g. with `CloseStatus::GoingAway`.
> **Note:** `stop()` doesn't interrupt WebSocket handlers. A handler looping on `read()` keeps running as long as its peer stays connected, and `listen()` doesn't return until it ends. Keep track of open WebSocket connections and `close()` them on shutdown, e.g. with `CloseStatus::GoingAway`. The handler's `read()` then waits for the peer's Close reply up to its read timeout (300 seconds by default). To shorten that, call `set_read_timeout()` in the handler before the loop; `read()` then also returns `Timeout` whenever the peer stays idle that long, so the loop has to handle it.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.