Commit Graph
2356 Commits
Author SHA1 Message Date
yhirose 36065c657c Bound the wait of WebSocket::close() as a whole
close() waited for the peer's Close reply with a read timeout that
applied to each read, so a peer that kept sending other frames, or sent
one frame a byte at a time, held close() for as long as it liked.

Bound the wait as a whole. close() sets the stream's max timeout, which
shortens each wait to what is left, and stops reading frames at the
deadline, which matters when the peer keeps the socket full and no wait
happens at all. WebSocketSSLStream gains set_max_timeout(), the one
stream a WebSocket uses that lacked it.

A read() running on another thread when close() is called still waits
for the reply up to its own read timeout, as documented. The docs now
say how to bound it.
2026-11-17 00:56:06 -05:00
Graeme Smecher d7b0d4bedb Drain the socket before closing a TLS connection (#2673)
Server::process_and_close_socket() half-closes the socket and drains what
the peer is still sending before the final close (#2534), so that a close
with unread data, or with bytes still in flight, is not answered with a
RST. A RST can make the peer's stack discard a response it has received
but not yet read, as Windows does.

SSLServer::process_and_close_socket() still closed outright. After a
completed handshake, shut down TLS as before (sending close_notify when
the exchange completed) and then drain the same way. A failed handshake
closes outright as before.

Fix #2672
2026-11-16 23:53:56 -05:00
yhirose 4af42e1907 Don't reuse a client connection with unread data on it
A kept-alive connection was reused while bytes that belong to no
request were waiting on it, and the next request read them as its
response. Such bytes are not a response (RFC 9112 9.2). A server may,
for one, send a 408 before it closes an idle connection.

Close the connection when they show up: after a response, if bytes
past it are already buffered in the stream or the TLS library, and
before reuse, if anything is readable on the socket. As in curl, the
reuse check no longer asks the TLS library whether the peer closed, so
it is the same for every TLS backend. A TLS message on an idle
connection, such as a late session ticket, now costs a new connection.
2026-11-16 23:41:25 -05:00
yhirose 1b40ce850b Close the tunnel when the TLS handshake through a proxy fails
SSLClient::setup_proxy_connection() left the socket open when the TLS
handshake inside the CONNECT tunnel failed, so the client kept a
connection with no TLS session on it. Only the next request's reuse
check closed it, because tls::is_peer_closed() reports a missing
session as closed. Close it where the handshake fails, as
ensure_socket_connection() does for a direct connection.
2026-11-16 23:41:25 -05:00
metsw24-maxandyhirose f57672b390 take the request lock in open_stream_ before reusing the socket (#2669)
* take the request lock in open_stream_ before reusing the socket

* Hold the request lock only while open_stream_ takes the socket

The handle owns the connection once the socket is transferred to it, so
holding request_mutex_ while the stream wrote its request and read the
response head only made requests on other threads wait for the stream.
Also shorten the comments.

---------

Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
2026-11-16 22:08:13 -05:00
metsw24-maxandyhirose 654fd3546c resolve junctions and symlinks in canonicalize_path on Windows (#2670)
* resolve junctions and symlinks in canonicalize_path on Windows

* Shorten the canonicalize_path comment

---------

Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
2026-11-16 21:19:39 -05:00
yhirose 546e821c65 Add server_mount_fuzzer for static file serving (#2666)
Fuzz Server::handle_file_request through a mount point backed by a temporary directory that holds symlinks escaping the mount root, one of them into a sibling directory sharing its name prefix. Abort if a token stored outside the root appears in a response. The seeds are based on those of #2661.

Move FuzzedStream into fuzzed_stream.h so that both server fuzzers share it, and add FuzzableServer::MarkAsRunning(), without which a server that is not listening writes no file.
2026-11-12 21:52:33 -05:00
yhirose 53a7f4964e Add sse_parser_fuzzer for detail::SSEParser (#2665)
Feed the event stream to the parser in chunks and abort if the reported events, ids and retries differ from parsing it in one piece. Also run with a length limit to cover the early-stop path.
2026-11-12 21:11:48 -05:00
Arthur Chan 8c5fd6aa08 OSS-Fuzz: Improve url_parser_fuzzer with more cases (#2664)
Add cases for relative location resolution, dot segment removal, URI encoding and decoding, and filename sanitization.
2026-11-12 20:47:26 -05:00
Arthur Chan a2ebab183d OSS-Fuzz: Add new fuzzer/options targets websocket (#2663)
Add websocket_fuzzer covering frame reading, UTF-8 and close payload validation, and the upgrade response parser. Add websocket_fuzzer.options to raise max_len above the 64 KiB frame length boundary.
2026-11-12 20:47:22 -05:00
yhirose a39e081386 Test SSEParser directly instead of through a server
Move the SSE parsing tests from a fixture that started a server and an
SSEClient for every case to unit tests of detail::SSEParser, and turn
NonDigitRetryFieldIgnored, which slept for 500ms, into one of them. Add
tests that feed each stream split at every position and byte by byte,
and that cover the max event length and an event left incomplete at the
end of the stream.
2026-11-12 19:07:54 -05:00
yhirose 07142256ed Extract SSE event stream parsing into detail::SSEParser
Move the line splitting and event assembly out of
SSEClient::run_event_loop() into a parser that is fed chunks and reports
events, ids and retry values through handlers. The event loop is left
with I/O and reconnection, and the parser can be driven without a
server, which the following test and fuzzer changes rely on.

No behavior change.
2026-11-12 18:27:58 -05:00
userandastunm cc7a30f12f Preserve aliased keys when erasing entries (#2659)
Co-authored-by: astunm <astunm@users.noreply.github.com>
2026-11-11 14:32:09 -05:00
yhirose cdcd2ac6e6 Release v0.66.0 v0.66.0 latest 2026-11-10 17:19:39 -05:00
yhirose bd23a7b179 Keep the job queued when a dynamic worker thread cannot be created
ThreadPool::enqueue constructed the dynamic thread without handling
std::system_error. When thread creation failed (e.g. EAGAIN under a
pids limit), the exception escaped the accept loop and destroying the
pool's joinable threads called std::terminate. Treat the failure as
"no spare capacity" and leave the job for an existing worker.
2026-11-10 16:54:53 -05:00
metsw24-max 6d8dec3c3e Keep a redirect on the unix socket in ClientImpl::redirect (#2658)
A client using set_address_family(AF_UNIX) took the host of an absolute Location as a socket path, so a redirect to http://localhost/next failed and one to http://other.sock/next went to ./other.sock. Every hop now stays on the client's socket, as with curl's --unix-socket.

The same host is still decided from the Location, as curl does: only localhost (the Host this client sends) counts as this client, so a redirect to any other host goes through the cross-host client and drops Authorization and Cookie. That client reaches the original socket through addr_map_, and a host the caller mapped with set_direct_hostname_addr_map keeps its mapping. An https Location runs TLS over the socket with the Location's host.

The mapped-target lookup shared with make_connect_request_target is factored into detail::addr_map_target.
2026-11-07 10:22:00 -05:00
yhirose 7e5d0683ac Release v0.65.0 v0.65.0 2026-11-05 07:39:47 -05:00
yhirose 6df971434e Do not carry Request::params over to a redirect's Location
A request sent with send(Request) and params got those params appended
to the Location of a redirect that had no query, on the same origin or
another one. The Location alone names the next target, as it already
did for Get/Post(path, params).
2026-11-05 07:04:31 -05:00
Eminy 39c98d4671 Ignore a Range header with a unit other than bytes (#2657)
RFC 9110 14.2 says an origin server MUST ignore a Range header field
with a range unit it does not understand. parse_range_header() accepts
only bytes=, so a request such as "Range: items=1-2" got a 416 instead
of the full response. The unit is now checked first, and only a bytes
range is parsed and answered with a 416 when it is invalid.
2026-11-04 22:44:22 -05:00
Wei-Lun Liandyhirose a1ba73a256 Cap a send() or wolfSSL_read() length at INT_MAX (#2655)
* Cap a send() or wolfSSL_read() length at INT_MAX

On 64-bit Windows SocketStream::write() handed size to send(), whose
length is an int, so a single write of 4 GiB or more went out as its low
32 bits: a 4 GiB + 1 byte write sent one byte, and write_data() then
retried the remaining 4 GiB as zero-byte sends forever. The INT_MAX cap
that read() has was limited to 32-bit Windows in df20c27 (#1294), when
the body was still written with one unchecked write(); write_data() now
sends the rest, so the cap applies to every Windows build again.

The wolfSSL tls::read() had the same cast, so a read into a buffer over
INT_MAX failed. Cap it as the OpenSSL backend has done since f80864c.

* Drop the 4 GiB write test and shorten comments

The Windows-only test allocated 4 GiB and took about 5 s, which is too
heavy for CI.

---------

Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
2026-11-04 18:06:30 -05:00
metsw24-maxandyhirose 453b1b8613 keep a relative Location on its host in resolve_relative_location (#2654)
* keep a relative Location on its host in resolve_relative_location

* Shorten the comment on on_this_host in resolve_relative_location

---------

Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
2026-11-02 13:12:31 -05:00
Yoshiki Matsuda a18895f42e Let ClientImpl::stop() interrupt a request waiting on Windows (#2653)
stop() shuts the socket down to end a request in flight, but on Windows
that does not wake a read waiting in WSAPoll (#1396). The request then
waited until the server answered, the read timeout passed, or the OS
dropped the connection.

stop() now also sets a flag, and send_() gives the request's stream an
interrupt check that reads it, as SSEClient::stop() does since 801bf2e.
The wait for data then ends within 100 ms on every platform. A request
started after stop() clears the flag. A redirect or a digest retry sent
inside the request keeps it.

select_read_interruptible() now rounds its last wait up to a whole
millisecond. poll() takes milliseconds, and the wait ended up to 1 ms
before the deadline, which MaxTimeoutTest caught once every client
request waited through this function.
2026-10-30 13:40:50 -04:00
yhirose 24e58ce5b2 Release v0.64.0 v0.64.0 2026-10-29 17:08:34 -04:00
Kim, Hyuk a0ab09fc0a Add set_basic_auth and set_bearer_token_auth to WebSocketClient (#2652)
ClientImpl and SSLClient let callers set origin credentials with
set_basic_auth()/set_bearer_token_auth(), but WebSocketClient could only
take a fixed Authorization header through its constructor. Add both
setters: each connect() adds the Authorization header to the upgrade
request from the current credentials (Basic over Bearer), unless the
constructor headers already carry one, and never to the proxy's CONNECT
request.
2026-10-28 16:15:17 -04:00
Kim, Hyuk 290c4c0bbe Add set_tunnel_hostname_addr_map() and deprecate set_hostname_addr_map() (#2647)
Through a CONNECT proxy it is the proxy that connects to the origin, and the CONNECT target was always the original hostname, so there was no way to send the tunnel to another address. set_tunnel_hostname_addr_map() sets that target on SSLClient, Client and WebSocketClient. The CONNECT request's Host follows it, including the Digest retry, while the Host header, the SNI and the certificate check inside the tunnel keep the original hostname. The map is also copied to a redirect client.

set_hostname_addr_map() is deprecated in favor of set_direct_hostname_addr_map(), which behaves the same and names the connection it applies to. The two maps are independent and correspond to curl's --resolve and --connect-to; neither applies to the proxy's own address.
2026-10-27 21:09:25 -04:00
metsw24-max 25ba40f6fe Reject an EOF without close_notify in the Mbed TLS and wolfSSL read (#2649)
Mbed TLS and wolfSSL read a TLS EOF without close_notify as a clean end, so a response delimited by the connection close could be cut short and still come back complete. Both reads now end cleanly only on a close_notify, as on OpenSSL.
2026-10-27 18:31:55 -04:00
metsw24-max 578e4815b7 Treat ':' as a path separator in sanitize_filename on Windows (#2651)
On Windows a ':' ends a drive prefix or starts an NTFS stream name, so a filename such as "C:.." or "a.txt:stream" survived sanitize_filename and could replace the upload directory or write to an alternate data stream. Cut at ':' on Windows as well.
2026-10-27 18:16:21 -04:00
Wleeaf 252dbaa505 Remove dot segments from absolute redirect locations (#2648)
resolve_relative_location removed "." and ".." segments only from a relative-path Location. A Location that starts with "/" or an absolute URI was requested as is, so a redirect to /dir/../target went out as /dir/../target and a server that does not normalize the path answered 404.

RFC 3986 section 5.2.2 applies remove_dot_segments to the target path of every reference that has a path, and Go and urllib3 do the same. The loop now lives in detail::remove_dot_segments, and ClientImpl::redirect applies it to an absolute path as well. Relative locations resolve as before.
2026-10-25 22:56:32 -04:00
metsw24-max 8710fbd25d Refuse a ':' in a path a mount point serves on Windows (#2643)
On NTFS a ':' in a path names a stream of the file, so /dir/test.html::$DATA opened the default stream of dir/test.html and passed a set_pre_routing_handler() guard on /dir/test.html, and /setup.exe:Zone.Identifier served the stream Windows attaches to downloaded files. is_valid_path() now refuses a ':' on Windows.

Elsewhere a ':' is an ordinary file name character, so it is still served: a file such as 2024-01-01T10:00:00.log would otherwise answer 404. The test pins both sides.
2026-10-25 22:39:44 -04:00
metsw24-max 2f8b9f6f57 Guard last_event_id() against the async event loop in SSEClient (#2644)
last_event_id() returned a reference to a string that run_event_loop() assigns on its own thread under start_async(), so a caller reading it from another thread raced the writes. The member is now guarded by a mutex and the getter returns a copy.

The return type changes from const std::string & to std::string. Code that binds the result to a const reference or copies it still compiles.
2026-10-25 22:21:55 -04:00
metsw24-maxandyhirose 391de32115 finish short writes in write_websocket_frame and open_stream_ (#2642)
write_websocket_frame and the request-body write in open_stream_ called
Stream::write once and only failed on a negative return, so a short
write (one record per mbedtls_ssl_write call, or SO_SNDTIMEO expiring
after part of the data went out) dropped the rest of the frame or body.
Route both through write_data, as the other body writers do.

Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
2026-10-25 22:19:53 -04:00
metsw24-max ba32b468a4 Honour Keychain trust settings when loading macOS system CAs (#2641)
The macOS system CA loader took every certificate that has Trust Settings, in the system, admin or user domain, as a trusted root without reading those settings. A root set to "Never Trust" in Keychain Access, or trusted only for S/MIME or code signing, still verified TLS servers.

The loader now evaluates the settings for the SSL policy. A per-user setting overrides the admin domain, which overrides the system domain, and only TrustRoot and TrustAsRoot are loaded. The OpenSSL path shares the helper that the Mbed TLS and wolfSSL paths already use.
2026-10-25 21:41:36 -04:00
Luochenchen 76f84a5bfd Honor set_decompress(false) in response streams (#2640)
open_stream_() created a decoder whatever set_decompress() said, so a response stream decoded the body when decompression was disabled, and failed with UnsupportedContentEncoding in a build without that decoder. The buffered path already checks the option; the streaming path now does too. The new test covers open_stream() and stream::Get().

Fixes #2639.
2026-10-25 20:05:56 -04:00
yhirose d23d1b6029 Match a host name against subjectAltName only
OpenSSL 4.1 no longer falls back to the subject Common Name when it
checks a host name (RFC 9525), so SSLClientServerTest.TlsVerifyHostname
began failing in the Windows CI job that installs the newest OpenSSL 4.x.

Make every backend agree instead of following whichever OpenSSL is
installed. OpenSSL passes X509_CHECK_FLAG_NEVER_CHECK_SUBJECT to both the
handshake check and verify_hostname(), and Mbed TLS and wolfSSL drop their
own CN fallback. A certificate that names its host only in the CN is now
rejected on all backends, as it already was on BoringSSL.
2026-10-25 18:58:43 -04:00
Wleeaf 7d349cbeed Follow only 301, 302, 303, 307 and 308 redirects (#2646)
is_redirect treated any 3xx with a Location as a redirect, so with set_follow_location(true) a 304 or 305 that carried a Location was followed. Only 301, 302, 303, 307 and 308 are redirects a client follows (RFC 9110 section 15.4), as in Go, requests and urllib3. A Location is still required. The 304 check in process_request that is_redirect now covers is dropped.
2026-10-25 18:55:20 -04:00
WleeafandWleeaf 5ceef83c54 Open a new connection for a redirect after sending Connection: close (#2645)
Without keep-alive, the client sends "Connection: close", but before
following a redirect it disconnected only when the response repeated
the option. RFC 9112 (section 9.6) lets a server close the connection
without repeating it, and a client that sent "close" must not send
another request on that connection. The redirected request was written
to the closing connection and failed with "Failed to read connection"
whenever the server closed it first. The client now disconnects before
a redirect or an auth retry whenever the request asked for the close.

Co-authored-by: Wleeaf <179125857+wleeaf@users.noreply.github.com>
2026-10-25 17:19:34 -04:00
yhirose a210b43e11 Give the ARM cross-compile job's build step its own time limit
apt on the runner mirror sometimes crawls for minutes
(actions/runner-images#14823), and the cross compiler is large enough
that its install alone used up the job's 10-minute limit. Loosen the job
limit to 30 minutes and limit the build step, which normally takes a few
seconds, to 2 minutes.
2026-10-22 18:00:44 -04:00
yhirose 7d28b64a03 Build BoringSSL with clang in the Linux CI job
GCC 15 on ubuntu-latest (26.04) reports a false -Wstringop-overflow in
BoringSSL's Keccak code, and BoringSSL's -Werror makes the build fail.
2026-10-22 17:40:53 -04:00
yhirose 97a972fcef Give the issue-2431 job's test step its own time limit
apt on the runner mirror sometimes crawls for minutes
(actions/runner-images#14823), which used up the job's 5-minute limit
before the test finished. Loosen the job limit to 15 minutes and limit
the test step, which normally takes about 80s, to 3 minutes.
2026-10-22 17:40:53 -04:00
metsw24-max 276cf08e84 Refuse a proxied path that does not start with a slash (#2638)
In absolute-form the path follows the authority, so a path such as "@evil.example/x" made the proxy connect to another host and forward the origin's credentials. Both handle_request and open_stream_ now refuse such a path before anything is written.
2026-10-22 17:01:58 -04:00
yhirose b4b24589ad Refuse to serve a path segment ending in a dot or a space
Windows trims a trailing '.' or space from a file name, so a mount
point there served a file under a path other than the one a pre-routing
handler checked. A mount point now serves no path with a segment
ending in '.' or a space, on every platform; "." and ".." are among
them. The README no longer lists such names among the ones the file
system can still alias.
2026-10-22 11:16:27 -04:00
yhirose 70bed02bc9 Bound how much of an unread request body the server skips
After answering, with a status below 400, a request whose body the
handler left unread, the server read past the body to reuse the
connection up to the payload limit (100MB by default), with only each
read bounded by the read timeout, so a slow sender could hold a worker
for a long time. It now skips at most
CPPHTTPLIB_UNREAD_PAYLOAD_MAX_LENGTH (256KB, as Go's net/http does)
within the read timeout as a whole, and closes the connection
otherwise. An error response already closed the connection.
2026-10-22 10:59:57 -04:00
yhirose 5db950e1e1 Normalize the extension of a user MIME type mapping when it is set
set_file_extension_and_mimetype_mapping() kept the extension as given,
and a file's extension was looked up as it was, then in lowercase, so a
mapping given as ".pdf" never matched and one given as "PDF" matched
only files ending in ".PDF". The extension is now lowercased and loses
a leading '.' when it is set, as Apache's AddType does, and a file's
extension is looked up in lowercase only.
2026-10-22 10:21:06 -04:00
yhirose b8ae3e43d5 Answer 500 to an exception the exception handler gives no status
An exception handler that set no status sent its response with 200, or
with whatever status the throwing handler had set. The status is now
set to 500 before the exception handler runs, as ASP.NET Core does, so
it stays 500 unless the exception handler sets another one.

On a WebSocket upgrade, an exception from the pre-routing handler, the
pre-request handler or the sub-protocol selector skipped the exception
handler and dropped the connection without a response. Everything
before the protocol switches now goes through the same exception
handling as routing, so it is answered over HTTP. The WebSocket handler
itself still runs outside it.
2026-10-22 10:01:48 -04:00
yhirose 64b5c68a46 Run RequestWithoutContentLengthOrTransferEncoding on Windows too
The test was kept off Windows while reading the body of such a request
waited on the socket and could hang there. That wait was made
non-blocking soon after, and such a request no longer has its body
read at all, so nothing in it is specific to the platform any more.
2026-10-22 09:32:29 -04:00
yhirose 1c91d54b7c Read no body from an unframed request in builds without TLS too
A request with neither Content-Length nor chunked Transfer-Encoding
has no body (RFC 9112 6.3), and builds with TLS support read none.
Builds without it still read whatever followed such a request on a
connection about to close, up to EOF, as its body. Every build now
reads no body. Tests that exercised only that read, or passed whatever
the server did, are removed, and a new one checks in every build that
the bytes after such a request are not taken as its body.
2026-10-22 08:50:17 -04:00
yhirose fbd3e2cc9b Answer 501 to a request with a transfer coding besides chunked
A request whose Transfer-Encoding ended in chunked but listed more
(another coding before chunked, or chunked twice) was read with only
the chunked framing removed, so the handler got a body still in the
other coding. RFC 9112 6.1 lets a server answer 501 to a transfer
coding it does not implement. The server now accepts only a single
Transfer-Encoding field whose value is chunked, as Go and nginx do,
and answers any other list ending in chunked with 501 and closes the
connection. A list whose final coding is not chunked still gets 400.
2026-10-22 07:58:28 -04:00
yhirose 32a0ca8b35 Ignore Range on requests other than GET and HEAD
RFC 9110 14.2 defines range handling only for GET, and a server must
ignore a Range header on any other method. The server parsed it on
every method, so a POST with a Range got a 206 slice of the handler's
response, or a 416 when the Range was malformed. Range is now parsed
only for GET and HEAD; other methods get the whole response, and a
handler can still read the header itself.
2026-10-22 07:31:50 -04:00
yhirose b83b385ed4 Match trusted proxies as IP addresses
set_trusted_proxies() compared addresses as strings, so a proxy listed
as 10.0.0.2 was not trusted when a server listening on both IPv4 and
IPv6 saw it as ::ffff:10.0.0.2, and two spellings of one IPv6 address
did not match either. The listed addresses are now put in one spelling
when they are set, and so are the connection peer and each
X-Forwarded-For hop before they are looked up; an IPv4-mapped IPv6
address becomes its IPv4 address. The README now documents the setter.
2026-10-22 00:37:09 -04:00
yhirose f297616584 Note that TLS settings do not pass through a plain HTTP redirect hop
The README said a redirect keeps the TLS verification settings. They
pass only from one HTTPS hop to the next; an HTTPS hop after a plain
HTTP one starts over with the defaults.
2026-10-22 00:10:57 -04:00