RFC 9110 14.2 defines range handling only for GET, and a server must
ignore a Range header on any other method. The server parsed it on
every method, so a POST with a Range got a 206 slice of the handler's
response, or a 416 when the Range was malformed. Range is now parsed
only for GET and HEAD; other methods get the whole response, and a
handler can still read the header itself.
set_trusted_proxies() compared addresses as strings, so a proxy listed
as 10.0.0.2 was not trusted when a server listening on both IPv4 and
IPv6 saw it as ::ffff:10.0.0.2, and two spellings of one IPv6 address
did not match either. The listed addresses are now put in one spelling
when they are set, and so are the connection peer and each
X-Forwarded-For hop before they are looked up; an IPv4-mapped IPv6
address becomes its IPv4 address. The README now documents the setter.
RejectsChunkOverheadPastItsLimit sent about 78 KB per case, and the
server refused the request a few chunks in and closed the connection
while the client was still writing, so the send failed now and then
instead of returning the 400. Pad each size line with a sixteenth of
the limit, so each request is about 10 KB and goes past the limit only
at its last chunks.
TLS settings take effect as a connection is set up, so after a client
changed one (server certificate or host name verification, a CA store,
a verifier) a later request still went over the connection kept alive
under the old settings, unverified if verification had just been turned
on. Each such setter now bumps a version that the connection records
when it is set up, and a connection set up under an older version is
closed instead of reused. The setters take no lock, so they cannot wait
on a handshake in flight.
A server built with compression support compressed every handler
response the client accepted, with no way to turn it off. Over HTTPS
the length of a compressed response can give away a secret in it when
the same response also carries text an attacker chooses (BREACH).
set_compress(false) sends handler responses, buffered or streamed, as
they are; compression stays on by default, and static file compression
keeps its own setting. The README and the cookbook describe the switch
and when to use it.
tls::create_ca_store() on Mbed TLS parsed the caller's buffer with one
byte more than its length, so it read past the end of the buffer, and
when that byte was not a NUL the PEM was not recognized and the call
returned NULL. Parse a NUL-terminated copy instead, as load_ca_pem()
does. DER input still parses, since its own length ends it.
The server left response header fields with an invalid name or value,
such as a value carrying CR/LF that would split the response, for the
default header writer to skip, so a writer set with set_header_writer()
wrote them. Drop such fields from res.headers right after the handlers
that can fill it directly have run, before the server reads it for
framing and range decisions and before any header writer sees it. The
client already rejects invalid request headers before writing them.
The payload limit counts only chunk data, and a chunk-size line is only
bounded by the line length limit, so a peer could send a few bytes of
data with any amount of padding, leading zeros and chunk extensions
around them. Count those bytes as Go does: each chunk may spend 16
bytes plus twice its data on its size line and the CRLF after it, which
still lets a stream send a byte at a time, and the excess may build up
to CPPHTTPLIB_HEADER_MAX_LENGTH before the read fails. This covers
request bodies on the server and response bodies on the client.
The OpenSSL client finished the handshake with SSL_VERIFY_NONE and
checked the server certificate only afterwards, so it presented its
client certificate to a server that then failed verification. With a
server certificate verifier that accepted, wolfSSL did the same. Verify
the chain, and the host name, during the handshake on every backend,
keeping the built-in verdict when a verifier runs alongside it. On
OpenSSL a session verifier, which may accept a certificate the built-in
check rejects, still decides after the handshake.
A certificate that fails the built-in verification now reports
CertVerifyFailed, or HostnameMismatch for its name, in ssl_error() on
every backend, where OpenSSL left it 0.
Under the default system CA policy, a custom CA given to a client keeps
the system CAs out. SSLClient and WebSocketClient only noted the custom
CA when set_ca_cert_store() got a store or load_ca_cert_store() got some
bytes, so a NULL store, such as the one tls::create_ca_store() returns
for a PEM it cannot parse, or an empty PEM left the client trusting
every system CA instead. Note the custom CA on every call, so that such
a client fails verification. A redirect still passes a CA on only when
the original client had one.
A mount point left ".", ".." and empty path segments for the file
system to resolve, while the pre-routing handler, where the README has
applications guard static files, saw the path as it was sent. A file
could then be reached under a spelling that a check on req.path did not
match. Serve a path from a mount point only when it has none of those
segments other than a trailing '/', so that the file served is the one
req.path names; any other path now gets 404. The README also notes the
other names a file system can give a file.
SSLServer::update_certs_pem() loaded the client CA it was given but
left client certificate verification as it was, so on a server built
without a client CA it returned true while clients without a
certificate still got in. Turn verification on, as the constructors do
when they are given a client CA. A client CA passed to
update_certs_pem() now always makes client certificates required.
The Mbed TLS server keeps the SNI from the ClientHello in a
thread-local until the handshake succeeds, and its SNI callback only
runs for a ClientHello that carries one. A handshake that failed left
its SNI behind, so the next connection served by the same thread
without SNI reported the earlier client's name in req.sni(). Start
each handshake with the thread-local cleared.
SSLServer::PemMemory takes a length of 0 to mean a NUL-terminated
string for cert_pem and key_pem, but skipped client_ca_pem when its
length was 0, so a server given its client CA that way started without
requiring client certificates. Read the client CA under the same rule
as the other fields. An empty client CA now fails to load instead of
turning the check off, and the README documents the length-0 rule.
ClientImpl::open_stream built the body reader from Content-Length and Transfer-Encoding whatever the method or status, so a stream on a response that RFC 9112 6.3 ends at its header section was read on: with a keep-alive server read() blocked until the read timeout, and when the server closed it reported a read error for a complete response. stream::Head() against cpp-httplib's own server, which puts the GET body's Content-Length on a HEAD response, hit it every time.
open_stream_() now returns with the body reader at its end for a response to HEAD and for a 204 or 304, as the buffered path skips the body for them. Content-Encoding is skipped too, so a coding the build lacks no longer fails such a stream.
When pre_routing_handler or pre_request_handler answers a WebSocket upgrade request, Server::process_request returns before the drain of an unread body and closes the connection only for a 4xx/5xx. A 2xx/3xx answer to an upgrade request with a framed body left that body to be parsed as the next request on the keep-alive connection. Those two early returns now close the connection when the request has a framed body, as the Expect rejection already does.
open_stream(), and with it stream::Get() and SSEClient, wrote its own
request head and left out what a buffered request gets: the
Authorization from set_basic_auth() or set_bearer_token_auth(), and
through a plain HTTP proxy the absolute-form target (RFC 9112 3.2.2)
and Proxy-Authorization. A proxy refused such a request, and an origin
that needs credentials answered 401.
Move the two decisions out of write_request() and handle_request() into
add_credential_headers() and proxy_target_prefix(), and use them in
open_stream() too, after connecting as a buffered request does. A
Digest challenge or a redirect is still handed to the caller.
A failed request could reach the client's error logger twice: once from
the step that failed, without a request when that was the TLS setup or
the proxy CONNECT, and again from send_() with the request. A content
receiver or progress callback that canceled was reported twice too, a
CONNECT answered with 407 only without a request, and a stream that
failed to open not at all unless TLS failed.
Report a failure in one place instead. The outermost send() reports the
request's error once, with the request the caller made, also when the
failure happened on a redirect (a redirect client no longer gets the
error logger) and when an answer carries an error, such as a proxy's
407. open_stream() reports a stream that fails to open the same way.
Only an upload body compressed before send() still reports where it
fails.
A buffered request whose response body went over set_payload_max_length(),
also once decompressed, failed with Error::Read, while the Stream API
reports the same limit as Error::ExceedMaxPayloadSize. read_content()
already marks it with a 413 status, so map that status to
ExceedMaxPayloadSize next to the 415 and 500 the client maps already.
696b94b built and ran test_websocket_heartbeat in the Windows jobs to
cover the pong timeout wake, which added more than a minute to each of
them. Only its two pong timeout tests exercise what differs by
platform, and both work with the ping interval set at runtime. Move
them into test.cc, where every job runs them in its shards, and drop
the Windows heartbeat build and step.
The heartbeat file keeps CompileTimeClientReadTimeoutIsFail, whose
fixture is renamed WebSocketSilentPeerTest after the peer it sets up.
When max_missed_pongs ran out, the heartbeat thread shut down the read
side of the socket to end a read() blocked on the silent peer. On
Windows that does not wake the wait, so read() stayed blocked until its
read timeout and then returned Timeout. The heartbeat test never ran on
Windows, so CI did not catch it.
The heartbeat now sets a flag that the stream's read interrupt checks,
the mechanism SSEClient::stop() uses, and WebSocketSSLStream takes the
interrupt too. It is set only when a pong timeout can happen, so other
connections wait as before. read() reports a timeout on a connection
that has closed as Fail, since Timeout means it is still usable.
Build and run the heartbeat test in the Windows jobs, and add a TLS
pong timeout test.
The event loop reads through open_stream(), which takes the socket out
of the Client, so the client_.stop() in stop() could not reach it.
stop(), and the destructor with it, waited until the server sent
something or the read timeout passed, 300 seconds by default.
Shutting the socket down from stop() does not wake a read waiting in
WSAPoll on Windows, so 3744227 was reverted. A stream now takes an
interrupt check instead: while one is set, it waits for data in 100 ms
slices and gives up once the check returns true, as the server's
keep-alive wait does with its flag. SSEClient sets one that turns true
when stop() is called. Streams without one wait as before.
SSEIntegrationTest.ManualStop now ends its stream with the fixture's
flag. Its server outlives the test body, and the faster stop() let the
stream read a local of the body that had returned.
This reverts commit 3744227de2.
Shutting down the socket does not wake a read waiting in WSAPoll on
Windows while the peer stays silent, so stop() still waited out the read
timeout there and the new tests failed. Faster stop() on the other
platforms also let SSEIntegrationTest.ManualStop return while its server
handler still read a local of the test body (stack-use-after-return).
The fix will come back built on a wait that works on every platform.
An upgrade request goes through the pre-routing handler before the
WebSocket routes are matched. When none matched, it was routed as a
plain HTTP request, and routing() ran the handler a second time. Skip
it there for an upgrade request.
An `event:` line without a value left the type empty, so the event
never reached on_event("message"), and on_message saw an empty type.
The HTML standard dispatches an event with an empty type as
"message".
StreamHandle::read() parsed the trailers only on the path without a
decompressor, so response->trailers stayed empty for a compressed
chunked response. Parse them in one place for both paths, and only
after the last chunk: after a failed read it waited another read
timeout for a trailer line, and parse_trailers_if_needed() called
before the end read body bytes as trailers.
A response with neither Content-Length nor chunked encoding ends when
the connection closes (RFC 9112 6.3), but the stream API set
read_error() to ConnectionClosed on reaching that end, and SSEClient
passed it on to on_error. Only a body shorter than its Content-Length
is cut short there.
With follow_location on, every 3xx was taken for a redirect, so a 304
Not Modified, or any 3xx without a Location, failed with Error::Unknown.
Follow a 3xx only when it has a Location, as curl does, and return the
others as the response, as curl, Go, requests and urllib3 do. Response
handlers and content receivers now get such a response too.
With built-in verification on, a server certificate verifier returning
true made Mbed TLS accept a certificate its own check had rejected,
such as a self-signed one, while OpenSSL and wolfSSL still failed with
SSLServerVerification. On a client, keep the flags of Mbed TLS's own
verification, so the verifier is an additional check as on the other
backends. With verification off, the verifier is still the only check.
A server still clears them: there all three backends let an accepting
callback admit a client certificate.
The event loop reads through open_stream(), which takes the socket out
of the Client, so the client_.stop() in stop() could not reach it.
stop(), and the destructor with it, waited until the server sent
something or the read timeout passed, 300 seconds by default.
The loop now hands the stream's socket to stop() while it reads, and
withdraws it under the same lock before the socket is closed. stop()
shuts down the read side only: a TLS backend answers the EOF with an
alert, and writing it to a socket closed for writing raises SIGPIPE.
SSLServer, SSLClient and WebSocketClient take a PemMemory with
cert_pem_len and key_pem_len, but the lengths were never read: every
backend read the PEM up to a NUL, so a buffer holding exactly the PEM,
such as a std::vector<char> read from a file, was read past its end.
Hand on a NUL-terminated copy of the given number of bytes instead.
A length of 0 still means a NUL-terminated string, which is what
callers got before. The client CA already honored its length.
WebSocketClient gets the proxy setters Client has: set_proxy(),
set_proxy_basic_auth(), set_proxy_bearer_token_auth(), set_no_proxy()
and, in SSL builds, set_proxy_digest_auth(). connect() opens a CONNECT
tunnel for ws:// as well as wss:// (RFC 6455 4.1) and sends the proxy
credentials on the CONNECT request only. A proxy that does not answer
200 is reported as Error::ProxyConnection, with its status and headers
in Result.
CURLOPT_READDATA takes a pointer, but ServerClosesConnection passed
the read_data struct itself through curl_easy_setopt()'s varargs, so
the read callback never got its state and never produced a body.
time_t is 32 bits wide on 32-bit builds, so the millisecond counts in
set_max_timeout() and the streams' duration() narrowed into it
implicitly. Cast them as duration_to_sec_and_usec() already does. In
the tests, six content receivers took a uint64_t length where
ContentReceiver passes size_t, the Expect: 100-continue test kept
10 GiB in a size_t, and the Range checks compared ssize_t fields with
unsigned literals, so they now compare whole Ranges.
DoesNotReserveTheClaimedLength shifted a size_t by 41. It now claims
std::string().max_size(), the most the client accepts, and still
fails if the reserve cap is removed.
BoringSSL counts stacks in size_t, and takes the protocol version as
uint16_t and the error code as uint32_t. Walk the certificate stacks
with the decltype(count) loop used elsewhere, pass the version as
uint16_t, and cast the error code to what ERR_get_error() returns.
MSVC takes the send() length as int, so the raw send() calls left in
the tests go through detail::send_socket().
ThreadPool::enqueue only spawned a dynamic thread when no thread was
idle. Threads already notified for an earlier job still counted as
idle until they took it, so a burst of connections could leave jobs
queued with no thread while the pool was below max_threads. With
keep-alive, such a connection waited until another connection closed.
Fix#2633
from_chars() mixed its int digit and base into the arithmetic of an
unsigned T, a sign conversion since Content-Length is parsed as size_t.
Convert both to T first.
Comparing a uint64_t with SIZE_MAX is always false on 64-bit builds,
so the WebSocket frame length and mmap size checks tripped
-Wtautological-type-limit-compare. fits_in_size_t() checks with a
round trip instead, and the Windows branch of mmap::open() uses it too.
The raw ::send() calls in the WebSocket tests passed an int length,
which POSIX takes as size_t. Use detail::send_socket(), which handles
both.
This reverts commit 28f3922. Keep the path encoding settled in #2082
(#2170) and #2608 until a concrete report needs more: user input
belongs in encode_path_component(), and an embedded NUL is invalid
input.
encode_path() sent '"', '<', '>', '\', '^', '`', '{', '|', '}', '['
and ']' raw, although RFC 3986 §3.3 doesn't allow them in a path.
Tomcat, Jetty and Undertow answer each of them with 400 by default, but
accept the percent-encoded form. It also stopped at an embedded NUL, so
"/a\0b" went out as "/a". Encode every byte outside pchar, keeping '/'
and '%' so that escapes pass through, and still encode '+', ',', ';'
and '\'' as before.
encode_request_target() now passes only the path to encode_path(). The
"http://host" of a target sent to a proxy and a CONNECT "host:port" are
sent as they are, so the brackets of an IPv6 host stay.
PemMemoryClientCertPresent now checks that the server saw the client's
certificate, as ClientCertPresent does. Before, it also passed when the
server built from PemMemory did not ask for a certificate at all.
The ClientCertPresent and ClientCertMissing helpers no longer pass
CLIENT_CA_CERT_DIR, which is "." and makes Mbed TLS parse every file in
the test directory, test binaries included, on each server they build.
The CA file is all they need. That makes TrustDirOptional, which built
the same server and client, a duplicate, so it goes. The tests that
still pass the directory keep the constructor taking one covered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
Make ClientCertStillRequired take the setup to apply, so the
update_certs_pem test runs the same pair as the verify callback tests:
a client without a certificate is rejected and one with a certificate
is still accepted. Rejection alone would also pass if the update broke
the server for every client.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
Split the rejection scenario out of ClientCertMissing so it can run
against any server, then use it to check that a server built from
PemMemory with a client CA, and a server whose certificates were
replaced with update_certs_pem, both reject a client without a
certificate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
On BoringSSL, SSL_get1_session() after a TLS 1.3 handshake gives a
session that cannot be resumed; the tickets come only through the
new-session callback, which its documentation recommends. Store the
session from that callback, so the resumption tests resume there too.
ThrowingVerifyCallbackRejectsTheCertificate and
ErrorAndExceptionHandlersReplaceProvider throw from a callback, which
does not compile with CPPHTTPLIB_NO_EXCEPTIONS and -fno-exceptions.
Both sides of the merge tested the same behavior. Keep one test of
each: ClientCertStillRequired* for a server keeping client certificates
required, which covers a cleared callback too, VerifierStaysWithItsClient
for per-client verifiers across threads, and the Keep* redirect tests,
which share one helper. KeepSessionVerifier now also checks the error,
as the dropped SessionVerifier did.
Both sides keep TLS verify callbacks per context. Keep this side's
storage, which lets a handshake keep the callback it is running and
takes an exception from it as a rejection, and take the other side's
apply_verify_mode(), which works out the verify mode in one place on
every backend, with Mbed TLS and wolfSSL client contexts verifying by
default. Mbed TLS installs its verify callback on every session, and
each session runs its own copy of the user's callback. The redirect
client setup that both sides added is kept once.
A resumed handshake does not show the client certificate again, so since
OpenSSL mTLS servers resume sessions, a verify callback, such as one
checking a deny list, was skipped for as long as the ticket was good
(7200 s by default). Turn off session tickets, TLS 1.3 ones included,
and the session cache on a context with a callback, as on wolfSSL.
A server sending "102 Processing" without end kept the client reading
them unless max_timeout was set. Skip at most
CPPHTTPLIB_INTERIM_RESPONSE_MAX_COUNT (100) and fail with Error::Read
past it.
Client took any string with "//" in it that it could not use as a URL
for a bad URL, so a Unix socket path such as /run/app//api.sock gave an
invalid client. Only a string that starts with a scheme and "://", or
with "//", is taken for a URL now.
set_content_provider() and file content added their Content-Type next to
the one an earlier set_content() had set, so the response went out with
two, and clients took the first. Each content setter now replaces it. A
Content-Type set after set_file_content() is used for the file.
A content provider without a length or chunked coding sends a body that
only the end of the connection delimits (RFC 9112 6.3), yet the response
offered keep-alive, so a keep-alive client waited for the server's idle
timeout. Send Connection: close and close after it, as Go's net/http
does. A Content-Length set by the handler is still dropped; the
overload that takes a length frames such a body.
yhirose