open_stream(), and with it stream::Get() and SSEClient, wrote its own
request head and left out what a buffered request gets: the
Authorization from set_basic_auth() or set_bearer_token_auth(), and
through a plain HTTP proxy the absolute-form target (RFC 9112 3.2.2)
and Proxy-Authorization. A proxy refused such a request, and an origin
that needs credentials answered 401.
Move the two decisions out of write_request() and handle_request() into
add_credential_headers() and proxy_target_prefix(), and use them in
open_stream() too, after connecting as a buffered request does. A
Digest challenge or a redirect is still handed to the caller.
A failed request could reach the client's error logger twice: once from
the step that failed, without a request when that was the TLS setup or
the proxy CONNECT, and again from send_() with the request. A content
receiver or progress callback that canceled was reported twice too, a
CONNECT answered with 407 only without a request, and a stream that
failed to open not at all unless TLS failed.
Report a failure in one place instead. The outermost send() reports the
request's error once, with the request the caller made, also when the
failure happened on a redirect (a redirect client no longer gets the
error logger) and when an answer carries an error, such as a proxy's
407. open_stream() reports a stream that fails to open the same way.
Only an upload body compressed before send() still reports where it
fails.
A buffered request whose response body went over set_payload_max_length(),
also once decompressed, failed with Error::Read, while the Stream API
reports the same limit as Error::ExceedMaxPayloadSize. read_content()
already marks it with a 413 status, so map that status to
ExceedMaxPayloadSize next to the 415 and 500 the client maps already.
696b94b built and ran test_websocket_heartbeat in the Windows jobs to
cover the pong timeout wake, which added more than a minute to each of
them. Only its two pong timeout tests exercise what differs by
platform, and both work with the ping interval set at runtime. Move
them into test.cc, where every job runs them in its shards, and drop
the Windows heartbeat build and step.
The heartbeat file keeps CompileTimeClientReadTimeoutIsFail, whose
fixture is renamed WebSocketSilentPeerTest after the peer it sets up.
When max_missed_pongs ran out, the heartbeat thread shut down the read
side of the socket to end a read() blocked on the silent peer. On
Windows that does not wake the wait, so read() stayed blocked until its
read timeout and then returned Timeout. The heartbeat test never ran on
Windows, so CI did not catch it.
The heartbeat now sets a flag that the stream's read interrupt checks,
the mechanism SSEClient::stop() uses, and WebSocketSSLStream takes the
interrupt too. It is set only when a pong timeout can happen, so other
connections wait as before. read() reports a timeout on a connection
that has closed as Fail, since Timeout means it is still usable.
Build and run the heartbeat test in the Windows jobs, and add a TLS
pong timeout test.
The event loop reads through open_stream(), which takes the socket out
of the Client, so the client_.stop() in stop() could not reach it.
stop(), and the destructor with it, waited until the server sent
something or the read timeout passed, 300 seconds by default.
Shutting the socket down from stop() does not wake a read waiting in
WSAPoll on Windows, so 3744227 was reverted. A stream now takes an
interrupt check instead: while one is set, it waits for data in 100 ms
slices and gives up once the check returns true, as the server's
keep-alive wait does with its flag. SSEClient sets one that turns true
when stop() is called. Streams without one wait as before.
SSEIntegrationTest.ManualStop now ends its stream with the fixture's
flag. Its server outlives the test body, and the faster stop() let the
stream read a local of the body that had returned.
This reverts commit 3744227de2.
Shutting down the socket does not wake a read waiting in WSAPoll on
Windows while the peer stays silent, so stop() still waited out the read
timeout there and the new tests failed. Faster stop() on the other
platforms also let SSEIntegrationTest.ManualStop return while its server
handler still read a local of the test body (stack-use-after-return).
The fix will come back built on a wait that works on every platform.
An upgrade request goes through the pre-routing handler before the
WebSocket routes are matched. When none matched, it was routed as a
plain HTTP request, and routing() ran the handler a second time. Skip
it there for an upgrade request.
An `event:` line without a value left the type empty, so the event
never reached on_event("message"), and on_message saw an empty type.
The HTML standard dispatches an event with an empty type as
"message".
StreamHandle::read() parsed the trailers only on the path without a
decompressor, so response->trailers stayed empty for a compressed
chunked response. Parse them in one place for both paths, and only
after the last chunk: after a failed read it waited another read
timeout for a trailer line, and parse_trailers_if_needed() called
before the end read body bytes as trailers.
A response with neither Content-Length nor chunked encoding ends when
the connection closes (RFC 9112 6.3), but the stream API set
read_error() to ConnectionClosed on reaching that end, and SSEClient
passed it on to on_error. Only a body shorter than its Content-Length
is cut short there.
With follow_location on, every 3xx was taken for a redirect, so a 304
Not Modified, or any 3xx without a Location, failed with Error::Unknown.
Follow a 3xx only when it has a Location, as curl does, and return the
others as the response, as curl, Go, requests and urllib3 do. Response
handlers and content receivers now get such a response too.
With built-in verification on, a server certificate verifier returning
true made Mbed TLS accept a certificate its own check had rejected,
such as a self-signed one, while OpenSSL and wolfSSL still failed with
SSLServerVerification. On a client, keep the flags of Mbed TLS's own
verification, so the verifier is an additional check as on the other
backends. With verification off, the verifier is still the only check.
A server still clears them: there all three backends let an accepting
callback admit a client certificate.
The event loop reads through open_stream(), which takes the socket out
of the Client, so the client_.stop() in stop() could not reach it.
stop(), and the destructor with it, waited until the server sent
something or the read timeout passed, 300 seconds by default.
The loop now hands the stream's socket to stop() while it reads, and
withdraws it under the same lock before the socket is closed. stop()
shuts down the read side only: a TLS backend answers the EOF with an
alert, and writing it to a socket closed for writing raises SIGPIPE.
SSLServer, SSLClient and WebSocketClient take a PemMemory with
cert_pem_len and key_pem_len, but the lengths were never read: every
backend read the PEM up to a NUL, so a buffer holding exactly the PEM,
such as a std::vector<char> read from a file, was read past its end.
Hand on a NUL-terminated copy of the given number of bytes instead.
A length of 0 still means a NUL-terminated string, which is what
callers got before. The client CA already honored its length.
Bring the READMEs and the docs pages (en and ja) in line with the
current header:
- Fix the client sample that does not compile, the Transfer-Encoding
statement for default headers, the verifier callback comment, the
thread pool spawn rule, the sanitize_filename description, and the
inverted Accept-Encoding sentence.
- Describe recent behaviour: TLS errors behind Read/Write failures,
Host validation, Connection: close for a body without length,
content_reader rules, refused CONNECT, WebSocketClient proxies,
CPPHTTPLIB_INTERIM_RESPONSE_MAX_COUNT, the SSE event size limit, and
the payload limit of the Stream API.
- Complete the API references of WebSocket, WebSocketClient and the
stream:: helpers, and the SSE reconnect rules.
- Fix cookbook pages: keychain loading on every backend, received Close
status, a multipart sample that trusted the client's file name, an
echo sample that sent binary as text, SNI, pre-request handler scope,
stop() with WebSocket handlers, and the thread pool notes.
- Restore the mutex in the LLM app's Chapter 5 code.
WebSocketClient gets the proxy setters Client has: set_proxy(),
set_proxy_basic_auth(), set_proxy_bearer_token_auth(), set_no_proxy()
and, in SSL builds, set_proxy_digest_auth(). connect() opens a CONNECT
tunnel for ws:// as well as wss:// (RFC 6455 4.1) and sends the proxy
credentials on the CONNECT request only. A proxy that does not answer
200 is reported as Error::ProxyConnection, with its status and headers
in Result.
CURLOPT_READDATA takes a pointer, but ServerClosesConnection passed
the read_data struct itself through curl_easy_setopt()'s varargs, so
the read callback never got its state and never produced a body.
time_t is 32 bits wide on 32-bit builds, so the millisecond counts in
set_max_timeout() and the streams' duration() narrowed into it
implicitly. Cast them as duration_to_sec_and_usec() already does. In
the tests, six content receivers took a uint64_t length where
ContentReceiver passes size_t, the Expect: 100-continue test kept
10 GiB in a size_t, and the Range checks compared ssize_t fields with
unsigned literals, so they now compare whole Ranges.
DoesNotReserveTheClaimedLength shifted a size_t by 41. It now claims
std::string().max_size(), the most the client accepts, and still
fails if the reserve cap is removed.
BoringSSL counts stacks in size_t, and takes the protocol version as
uint16_t and the error code as uint32_t. Walk the certificate stacks
with the decltype(count) loop used elsewhere, pass the version as
uint16_t, and cast the error code to what ERR_get_error() returns.
MSVC takes the send() length as int, so the raw send() calls left in
the tests go through detail::send_socket().
ThreadPool::enqueue only spawned a dynamic thread when no thread was
idle. Threads already notified for an earlier job still counted as
idle until they took it, so a burst of connections could leave jobs
queued with no thread while the pool was below max_threads. With
keep-alive, such a connection waited until another connection closed.
Fix#2633
from_chars() mixed its int digit and base into the arithmetic of an
unsigned T, a sign conversion since Content-Length is parsed as size_t.
Convert both to T first.
Comparing a uint64_t with SIZE_MAX is always false on 64-bit builds,
so the WebSocket frame length and mmap size checks tripped
-Wtautological-type-limit-compare. fits_in_size_t() checks with a
round trip instead, and the Windows branch of mmap::open() uses it too.
The raw ::send() calls in the WebSocket tests passed an int length,
which POSIX takes as size_t. Use detail::send_socket(), which handles
both.
This reverts commit 28f3922. Keep the path encoding settled in #2082
(#2170) and #2608 until a concrete report needs more: user input
belongs in encode_path_component(), and an embedded NUL is invalid
input.
encode_path() sent '"', '<', '>', '\', '^', '`', '{', '|', '}', '['
and ']' raw, although RFC 3986 §3.3 doesn't allow them in a path.
Tomcat, Jetty and Undertow answer each of them with 400 by default, but
accept the percent-encoded form. It also stopped at an embedded NUL, so
"/a\0b" went out as "/a". Encode every byte outside pchar, keeping '/'
and '%' so that escapes pass through, and still encode '+', ',', ';'
and '\'' as before.
encode_request_target() now passes only the path to encode_path(). The
"http://host" of a target sent to a proxy and a CONNECT "host:port" are
sent as they are, so the brackets of an IPv6 host stay.
PemMemoryClientCertPresent now checks that the server saw the client's
certificate, as ClientCertPresent does. Before, it also passed when the
server built from PemMemory did not ask for a certificate at all.
The ClientCertPresent and ClientCertMissing helpers no longer pass
CLIENT_CA_CERT_DIR, which is "." and makes Mbed TLS parse every file in
the test directory, test binaries included, on each server they build.
The CA file is all they need. That makes TrustDirOptional, which built
the same server and client, a duplicate, so it goes. The tests that
still pass the directory keep the constructor taking one covered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
Make ClientCertStillRequired take the setup to apply, so the
update_certs_pem test runs the same pair as the verify callback tests:
a client without a certificate is rejected and one with a certificate
is still accepted. Rejection alone would also pass if the update broke
the server for every client.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
Split the rejection scenario out of ClientCertMissing so it can run
against any server, then use it to check that a server built from
PemMemory with a client CA, and a server whose certificates were
replaced with update_certs_pem, both reject a client without a
certificate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
On BoringSSL, SSL_get1_session() after a TLS 1.3 handshake gives a
session that cannot be resumed; the tickets come only through the
new-session callback, which its documentation recommends. Store the
session from that callback, so the resumption tests resume there too.
ThrowingVerifyCallbackRejectsTheCertificate and
ErrorAndExceptionHandlersReplaceProvider throw from a callback, which
does not compile with CPPHTTPLIB_NO_EXCEPTIONS and -fno-exceptions.
windows.h defines max() as a macro, which broke the Windows builds at
the two time_point::max() calls the handshake deadline added. Wrap them
in parentheses, as the other max() calls are.
Both sides of the merge tested the same behavior. Keep one test of
each: ClientCertStillRequired* for a server keeping client certificates
required, which covers a cleared callback too, VerifierStaysWithItsClient
for per-client verifiers across threads, and the Keep* redirect tests,
which share one helper. KeepSessionVerifier now also checks the error,
as the dropped SessionVerifier did.
Both sides keep TLS verify callbacks per context. Keep this side's
storage, which lets a handshake keep the callback it is running and
takes an exception from it as a rejection, and take the other side's
apply_verify_mode(), which works out the verify mode in one place on
every backend, with Mbed TLS and wolfSSL client contexts verifying by
default. Mbed TLS installs its verify callback on every session, and
each session runs its own copy of the user's callback. The redirect
client setup that both sides added is kept once.
name and filename are read as RFC 9110 quoted-strings, while browsers
send a backslash as it is, so from a browser a value with "\\" in it
arrives with a single backslash, and one ending in "\" is misread.
Through a plain-HTTP proxy the request line carries the absolute form,
while the Digest uri stays in origin form, as curl sends it. The comment
cited RFC 7616 3.4, which asks for the effective request URI.
A resumed handshake does not show the client certificate again, so since
OpenSSL mTLS servers resume sessions, a verify callback, such as one
checking a deny list, was skipped for as long as the ticket was good
(7200 s by default). Turn off session tickets, TLS 1.3 ones included,
and the session cache on a context with a callback, as on wolfSSL.
A server sending "102 Processing" without end kept the client reading
them unless max_timeout was set. Skip at most
CPPHTTPLIB_INTERIM_RESPONSE_MAX_COUNT (100) and fail with Error::Read
past it.
Client took any string with "//" in it that it could not use as a URL
for a bad URL, so a Unix socket path such as /run/app//api.sock gave an
invalid client. Only a string that starts with a scheme and "://", or
with "//", is taken for a URL now.
set_content_provider() and file content added their Content-Type next to
the one an earlier set_content() had set, so the response went out with
two, and clients took the first. Each content setter now replaces it. A
Content-Type set after set_file_content() is used for the file.
A content provider without a length or chunked coding sends a body that
only the end of the connection delimits (RFC 9112 6.3), yet the response
offered keep-alive, so a keep-alive client waited for the server's idle
timeout. Send Connection: close and close after it, as Go's net/http
does. A Content-Length set by the handler is still dropped; the
overload that takes a length frames such a body.
An exception from a verify callback unwound through the TLS library,
which is C: OpenSSL leaked the objects of the handshake, and a server
worker thread ended the process. It also left the handshake count that
update_certs_pem() waits on raised, so with a task queue that caught the
exception, every later update waited forever. Take the exception as a
rejection, and lower the count on every way out of the handshake.
Replacing a verify callback, as set_server_certificate_verifier() does,
deleted the one that a handshake on another thread could be running, on
OpenSSL and wolfSSL. Handshakes now share the callback and take their
reference under a lock, and SSLClient replaces it under the lock that
guards creating a session from its context.
tls::set_verify_callback() set the verify mode anew, so on a server that
required client certificates it made them optional: OpenSSL and wolfSSL
dropped SSL_VERIFY_FAIL_IF_NO_PEER_CERT, and Mbed TLS went from REQUIRED
to OPTIONAL. A client without a certificate got through. Keep what the
server requires.
update_server_client_ca() rotated the session ticket keys with 80 bytes,
which only OpenSSL takes. BoringSSL and LibreSSL take 48, refused them,
and the result was not checked, so a client went on resuming with a
ticket from before the client CA changed. Use the size each library
takes, and fail the update when the keys cannot be set.
Reset last_ssl_error_ and last_backend_error_ at the start of each
attempt in send_() rather than once in send(), so a request that
succeeds on the SSLPeerCouldBeClosed_ retry does not report the TLS
error of the attempt before it.
Now that every request writes those members, hold request_mutex_ until
the Result has read them, so another thread's request on the same
client cannot reset or replace them in between.
Also pick the TLS error up in a lambda in SSLClient::process_socket()
instead of through an out-parameter on process_client_socket_ssl(),
which has no other caller.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
A request that fails after the TLS handshake reports Error::Read or
Error::Write, and SSLSocketStream dropped the TLS error behind it. On
TLS 1.3, where a server rejects a client without a certificate only
after the handshake, ssl_error() and ssl_backend_error() were 0 instead
of showing, for example, OpenSSL's "tlsv13 alert certificate required".
SSLSocketStream now keeps the error from its last failed TLS read or
write, and SSLClient copies it into the values Result reports. Error
itself stays Read or Write.
Those values are also reset at the start of each request, so a later
successful request no longer reports the TLS error of an earlier one. A
client whose TLS context failed to set up keeps reporting that error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
wolfSSL shares its session cache between contexts, so a session resumed
on a server that checks client certificates could carry a certificate
checked against another server's CA, or against a client CA replaced
since. Such a server no longer resumes sessions.
Without a session id context, OpenSSL answers a client resuming its
session with a fatal alert once client certificates are verified, which
curl and browsers run into on their second connection. Set one on every
server context. A changed client CA still stops resumption, as fcf7568
flushes the cache and rotates the ticket keys.
With set_ca_cert_store() after load_ca_cert_store(), the redirect client
trusted the earlier PEM too, though the store had replaced it, and of
several load_ca_cert_store() calls it kept only the last. Take the CAs
from the context when a store was set, and from every PEM loaded
otherwise.
Hostname verification on these backends fell back to the subject CN
even when the certificate had a DNS SAN, unlike X509_check_host and
RFC 6125 6.4.4. It does so only without one now. Mbed TLS also reads
the CN by its OID instead of searching the formatted DN for "CN=", and
no longer stops at an empty SAN.
The Digest uri and A2 took the raw path, while the request line carried
it encoded, with the params and without the fragment, so a server
comparing the two rejected the credentials. Build the target in one
place for both.
set_keep_alive_max_count(0) tripped an assert, a negative WebSocket ping
interval sent pings in a tight loop, and the form length, multipart file
count and range count macros rejected every request at 0. Make 0 mean
no limit for those, and a ping interval of 0 or less turn pings off. The
URI, header and line bounds cannot be lifted, so a static_assert keeps
them positive.
yhirose