RFC 9112 6.1 has a server close the connection after answering a
request with both Transfer-Encoding and Content-Length, or with
Transfer-Encoding in HTTP/1.0. Close it, and send Connection: close
whenever the connection closes after a response.
split_unquoted ended a quoted-string at an escaped quote, so
name="a\"; filename=evil.php" came apart into a name and a filename that
a parser in front following RFC 9110 would not see. Skip quoted-pairs
when splitting, and unescape \" and \\ in Content-Disposition values,
leaving other backslashes, such as those of a Windows path, as they are.
A part header arriving a few bytes at a time was searched for its CRLF
from the start on every read, which made parsing it quadratic. Remember
how far the search got.
The SSE client buffered a line, or the data of an event, with no limit
of its own, and with payload_max_length(0) with none at all. Add
CPPHTTPLIB_SSE_MAX_EVENT_LENGTH (16MB) and end the connection with
ExceedMaxPayloadSize past it.
Every continuation frame could be as long as the whole message limit,
and the total was checked only after appending it, so a fragmented
message could take about twice the limit. Read each fragment against
what is left, with room for a control frame between them.
The client reserved what Content-Length claimed, up to
payload_max_length, before any of the body arrived, and without a limit
nothing bounded it. Reserve at most 1 MiB up front and let a longer body
grow as it is read.
An unsupported scheme without exceptions, or a port out of range, left
Client with no implementation, so the first request after it went
through a null pointer. Put a client in its place that reports itself
invalid and fails every request with Error::Connection.
CPPHTTPLIB_WEBSOCKET_MAX_PAYLOAD_LENGTH = 0 rejected every frame with a
payload, while 0 means no limit for payload_max_length. Make it mean the
same here, and refuse a length that size_t cannot hold.
set_payload_max_length(0) means no limit, but StreamHandle compared the
decompressed size against 0, so every compressed response read through
the streaming API failed.
case_ignore::hash recursed once per character of the key, and trailer
field names from the wire reach it, so a long one could overflow the
stack of a build that keeps the recursion (-O0). Hash in a loop instead,
which gives the same values.
find_content_type(), can_compress_content_type() and
is_prohibited_header_name() switched on a 26-bit hash of the name and
never compared the name itself, so any name that hashed alike was taken
for a known one. An uploaded file with an unknown extension could be
served as text/html, for one.
Compare the names themselves, and drop the hash and its literal
operator, which nothing else uses.
* Carry the server certificate verifier over to the redirect client
create_redirect_client() now copies the CA store and the session verifier
to the SSLClient it builds for a cross-host https redirect (#2632), but a
verifier given to set_server_certificate_verifier() was still dropped: the
redirect client's ctx never had the verify callback wired in, so a
pinning verifier never ran on the host the server redirects to.
SSLClient now keeps its own copy of the verifier, since ctx_ cannot report
it back and the backends hold it in a slot shared by every client, and
installs it on the redirect client. The copy also replaces the
Windows-only server_certificate_verifier_set_ flag.
SSLClientRedirectTest.ServerCertificateVerifier fails before this change
and passes after (OpenSSL 3, Mbed TLS 2.28, wolfSSL 5.6).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
* Clarify the server_certificate_verifier_ comment
The backends keep the verify callback in a per-thread slot, not one
shared by every client.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGAYpQgXt95LdDQcJYgjB8
---------
Co-authored-by: Claude <noreply@anthropic.com>
tls::set_verify_callback() kept the callback in a thread_local slot, so two
SSLClients configured on one thread overwrote each other's verifier, and a
client used from another thread ran none. Keep it with the context instead
(OpenSSL SSL_CTX ex_data, MbedTlsContext/WolfSSLContext member) and have each
backend trampoline look it up from the session's context.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wxevjmRqzpy8qDAD3vdwr
update_certs_pem() replaced the certificate, key and client CAs inside
the TLS context while handshakes on other threads were still reading
them, and only the creation of a session was serialized with it. A
server rotating its certificates under load could crash.
Count the handshakes in progress, make an update wait until there are
none, and hold off new handshakes while an update is pending so that a
steady stream of them cannot keep it waiting. The wait is bounded by
the handshake timeout.
The handshake loops gave every wait for the socket the full timeout
again, so a peer that sent a byte now and then kept a handshake going,
and the worker serving it, for as long as it liked. Work out the
deadline once and wait only for what is left of it, on both sides and in
every backend.
A cross-host redirect is followed by a new client, which did not get the
payload limit, the max timeout, the connection timeout's microseconds or
the hostname address map. Over HTTPS it also lost the session verifier,
the server certificate verifier and a store given to set_ca_cert_store(),
and fell back to the default CAs. Copy them over. The store belongs to
the original TLS context, so pass on the CA certificates it holds as PEM.
After the redirect, the original client also answered a 401 or 407 from
the new host with its own digest credentials and sent that to its own
host, with the new host's path. Return the redirected response as it is,
since the client that followed it has already answered what it could.
read_websocket_frame() resized the payload to the length a frame header
declared before reading any of it, so a 14-byte header made the peer
allocate and zero-fill up to CPPHTTPLIB_WEBSOCKET_MAX_PAYLOAD_LENGTH and
hold it until the read timed out. 70 such headers took a server to over
1 GB, and a client reading from a hostile server was exposed the same way.
Read the payload in CPPHTTPLIB_RECV_BUFSIZ chunks and grow it by each one,
so memory follows the bytes that actually came in.
std::regex_match recursed once per character of a status line's reason
phrase, so a server that sent one as long as a whole line overflowed the
client's stack. file_extension() and append_query_params() ran the same
kind of regex over a request path and a caller's path.
Parse all three by hand, with the same results, except that a path that
starts with '?' now gets its parameters joined with '&'. Serving a static
file no longer defers the content type lookup to keep a regex off the 304
path, so it looks the type up once.
FileContentProviderStaysWithinFile, added in e374966, calls
detail::set_file_content_provider, which split.py moves into httplib.cc,
so test_split did not compile. Declare it in the test, as the other tests
that call into detail:: already do.
When a content receiver returned false, or threw, partway through a body,
the drain after the response read a whole Content-Length again from where
it stopped. It swallowed the start of the next request, so the rest was
parsed as a request of its own once the handler answered below 400. A
handler that called the content reader a second time read into the next
request the same way.
Mark the body once reading starts, and if it does not finish, answer with
Connection: close and never drain it from the middle. Let each request's
body be read only once.
The wolfSSL verify callback set VerifyContext::session to the raw WOLFSSL
pointer, while every other tls:: function takes the WolfSSLSession that
wraps it, so passing ctx.session on from a verifier read the wrong type.
Point each WOLFSSL back to its WolfSSLSession and hand that out instead,
as the OpenSSL and Mbed TLS backends already do with their own handles.
tls::set_verify_callback(), which set_server_certificate_verifier() uses,
stored the callback in a single thread_local slot shared by every context.
A verifier was skipped when the handshake ran on another thread, ran for
other clients on the thread that set it, and could run after its client
was destroyed.
Store it with the context instead: in SSL_CTX ex_data for OpenSSL, and in
MbedTlsContext and WolfSSLContext for Mbed TLS and wolfSSL.
set_content() did not drop a content provider installed earlier, and
apply_ranges() then took the body's size as the Content-Length while the
provider was still the one written. The provider was asked for more bytes
than it had: a file under a mount point went out under the error status a
file request handler set to hide it, its mapping was read past the end,
and a user provider read past its own data.
Make every content setter clear the other sources so the last one wins,
let a body assigned directly after a provider or a file win as well, and
never read past the mapping in the file content provider.
* Redirect a directory request with a relative Location
The trailing-slash redirect used the request path as-is, so a path
starting with "//" produced a Location that clients read as a
network-path reference to another host. Name only the last segment,
percent-encoded and prefixed with "./", so the target always resolves
against the request URL, whatever the mount point or any prefix a
proxy put in front of it.
* Shorten the relative redirect comment
A directory request without a trailing slash was redirected to the path
relative to the mount point, so /mount/dir under a mount at /mount sent
Location: //dir/ instead of /mount/dir/. Redirect to the request path
with the slash appended.
SSEIntegrationTest.DigitRetryFieldApplied slept for 500ms and then
expected a second connection. The first connection, the 100ms wait and
the reconnect do not always fit in that window: the macOS Mbed TLS job
saw only one connection.
Wait until the second connection arrives, and check that it came at
least 100ms after the first one rather than counting connections. The
test no longer depends on how fast the runner is, and takes about 150ms
instead of 500ms.
Proxy-Authorization was added only when both the proxy username and the
proxy password were set, and a cross-host redirect carried the proxy
credentials over only when the username was set. Authorization has been
sent when either one is set since #958; do the same for the proxy.
A client that has both a proxy Basic credential with one empty field
and a proxy bearer token now sends Basic rather than Bearer, the order
Authorization already follows.
get_header_value(), get_header_value_u64(), get_multimap_value() and
MultipartFormData::get_field() advanced an equal_range() iterator by
the caller's index after casting it to ssize_t. An index that is
negative as a signed value stepped back onto an entry of another key,
and a huge positive one kept looping after the last match.
Advance with bounded_next(), which stops at the end of the range.
parse_accept_header() split on every ',' and then on every ';', with no
idea that a parameter value can be a quoted-string (RFC 9110 §5.6.6). A
value such as text/html;profile="a,b" was cut in the middle, and the
request was answered with 400.
The parser never looks at a parameter value, so remove each
quoted-string (§5.6.4) before parsing rather than teach every split
about quotes. A header without a '"' takes the same path as before.
A quoted-string is removed wherever it appears, so a malformed value
can now be read without it instead of being rejected, and an
unterminated quote hides the rest of the value.
RFC 9110 §14.1 makes range unit names case-insensitive, but
parse_range_header() matched "bytes=" exactly, so a request with
Range: Bytes=1-3 was answered with 416.
A request with more than one Host field line, or with a Host value
holding a byte that cannot appear in uri-host [ ":" port ], is now
answered with 400 (RFC 9112 §3.2). The handler sees only the first
Host line, so such a request could reach it with an authority that
differs from the one an intermediary acted on.
Only the byte set is checked, not the full grammar. An empty value
stays valid, and a request without a Host line is still accepted.
Close#2623
The SSE spec ends a line with CRLF, LF or CR, but SSEClient split only
on LF, so a stream that uses CR alone never produced an event. The LF
of a CRLF is skipped even when it arrives in a later chunk than its CR.
Search for a line ending only in the newly received data. What is left
in the buffer has none, and scanning it again for every chunk made a
long line quadratic.
SSEClient stored any id it received, but the client rejects a request
whose header value is invalid. An id with a NUL, a control character or
surrounding whitespace therefore made every reconnect fail with
Error::InvalidHeaders before reaching the server, and the id could
never be replaced.
Accept an id only if it is a valid header value, so that the previous
id stays in effect. The SSE spec asks for this for NUL only; the rest
follows from what this client can send.
The SSE spec strips one BOM at the start of the stream, but
SSEClient kept it as part of the first field name, so the
first event was silently dropped.
set_payload_max_length(0) is documented to disable the limit, but the
readers for a chunked body and for a body delimited by the connection
closing compared the size against the limit without checking for 0.
A server configured that way answered a chunked request with 413, and
a client failed to read a chunked or unframed response. A body with
Content-Length was already unlimited.
Give both readers the same `payload_max_length > 0` guard the other
payload checks use.
* match a wildcard only in the leftmost label in match_hostname
* Shorten the wildcard comment in match_hostname
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
verify_cert_with_windows_schannel() rejected a chain whenever
TrustStatus.dwErrorStatus was non-zero, before
CertVerifyCertificateChainPolicy() ran. The
CERT_CHAIN_POLICY_IGNORE_ALL_REV_UNKNOWN_FLAGS flag passed to that
policy check was therefore dead code: a certificate without revocation
information, or one whose CRL could not be fetched, failed with
CERT_TRUST_REVOCATION_STATUS_UNKNOWN.
Drop the pre-check so the SSL chain policy is the only judge.
Revocation checking becomes best-effort: a revoked certificate and
every other chain error are still rejected, while an undetermined
revocation status is accepted.
On a rejected chain, ssl_backend_error() now holds the policy status,
such as CERT_E_UNTRUSTEDROOT, instead of the trust status bit mask.
RFC 9110 14.1.2: if the representation is shorter than the suffix-length,
the entire representation is used. range_error computed a negative first
byte position for bytes=-8 on a 7-byte body and answered 416. Clamp it
at 0, as #711 did before the range handling was reworked.
Co-authored-by: youdie006 <youdie006@users.noreply.github.com>
* use the SAN type tag in Mbed TLS verify_hostname and get_cert_sans
Mbed TLS keeps a subjectAltName entry's GeneralName tag in buf.tag and the bare value in buf.p / buf.len. verify_hostname ignored the tag, so a dNSName whose bytes equal an address authenticated that IP host, and an iPAddress or rfc822Name was matched as a DNS pattern. get_cert_sans looked for the tag inside the value, so it reported no entries for an ordinary certificate, or part of a dNSName as an entry of its own.
* Shorten the SAN type comments
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
A SubProtocolSelector could return a value outside the client's
Sec-WebSocket-Protocol list, and the server sent it back as is. The
server now treats such a value as no selection.
* reject an unoffered subprotocol in the ws client handshake
* test the subprotocol check through WebSocketClient so the split build compiles
* Trim comments in the subprotocol check
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
A 303 response turns the follow-up request into a GET, but only the buffered body and headers were cleared. A content provider (sized or chunked) stayed on the request, so the original payload was sent again, and for a chunked provider the unframed chunks also broke the keep-alive connection.
The redirect also percent-decoded the Location path before sending it. That turned %23, %3F and %25 into a fragment, a query delimiter and a different octet, so the client requested a different resource than the one named, and made set_path_encode(false) fail on any Location containing %20. The path is now sent as given.
An event with an empty id field must reset the last event ID, so that
no Last-Event-ID header is sent on reconnect. run_event_loop only
updated last_event_id_ when the id was non-empty, so it could not tell
an empty id field from an event with no id field, and kept sending the
stale ID.
Track whether an id field was seen with a has_id flag, as has_data
does for the data field, and add a regression test.
A client may pipeline its requests (RFC 9112 9.3.2). The server read the
following request(s) into a per-request SocketStream buffer, discarded them
with the stream after the first response, and then waited in keep_alive()
for socket data that never came, closing the connection after the
keep-alive timeout. Over TLS the bytes stayed decrypted in the TLS library,
where keep_alive() could not see them either.
Create one stream per connection and serve a request that is already
buffered (Stream::is_readable()) without waiting in keep_alive().
Keeping the buffer means an extra CRLF that some clients send after a
request body is now parsed as the next request line, which answered 400
and closed the connection. Ignore one empty line before the request-line,
as RFC 9112 2.2 recommends.
Fixes#2599
A field line without a colon kept the \r of a CRLF line ending in its
name, so "data\r\n" was not recognized. Strip the \r once per line
before parsing instead of from each value.
An event without a data field was neither dispatched nor cleared, so
its event type leaked into the next event and its id only reached
last_event_id after a later event was dispatched. Reset the message on
every blank line and record the id even when nothing is dispatched.
The parsing tests exercised a copy of parse_sse_line that had drifted
from the real one. Run them through SSEClient against a local server
instead, and add regression tests for the cases above.
SSE events may contain an empty data field, and a data field without a colon also has an empty value. Track whether a data field was seen separately from the accumulated payload so empty events are dispatched and leading empty lines are preserved. Add an integration regression test for both forms.
yhirose