verify_cert_with_windows_schannel() rejected a chain whenever
TrustStatus.dwErrorStatus was non-zero, before
CertVerifyCertificateChainPolicy() ran. The
CERT_CHAIN_POLICY_IGNORE_ALL_REV_UNKNOWN_FLAGS flag passed to that
policy check was therefore dead code: a certificate without revocation
information, or one whose CRL could not be fetched, failed with
CERT_TRUST_REVOCATION_STATUS_UNKNOWN.
Drop the pre-check so the SSL chain policy is the only judge.
Revocation checking becomes best-effort: a revoked certificate and
every other chain error are still rejected, while an undetermined
revocation status is accepted.
On a rejected chain, ssl_backend_error() now holds the policy status,
such as CERT_E_UNTRUSTEDROOT, instead of the trust status bit mask.
Adds an explicit namespace qualification and a forward declaration for
class WebSocketClient. This change avoids a name clash on code bases
containing unrelated WebSocketClient classes.
RFC 9110 14.1.2: if the representation is shorter than the suffix-length,
the entire representation is used. range_error computed a negative first
byte position for bytes=-8 on a 7-byte body and answered 416. Clamp it
at 0, as #711 did before the range handling was reworked.
Co-authored-by: youdie006 <youdie006@users.noreply.github.com>
* use the SAN type tag in Mbed TLS verify_hostname and get_cert_sans
Mbed TLS keeps a subjectAltName entry's GeneralName tag in buf.tag and the bare value in buf.p / buf.len. verify_hostname ignored the tag, so a dNSName whose bytes equal an address authenticated that IP host, and an iPAddress or rfc822Name was matched as a DNS pattern. get_cert_sans looked for the tag inside the value, so it reported no entries for an ordinary certificate, or part of a dNSName as an entry of its own.
* Shorten the SAN type comments
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
A SubProtocolSelector could return a value outside the client's
Sec-WebSocket-Protocol list, and the server sent it back as is. The
server now treats such a value as no selection.
* reject an unoffered subprotocol in the ws client handshake
* test the subprotocol check through WebSocketClient so the split build compiles
* Trim comments in the subprotocol check
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
A 303 response turns the follow-up request into a GET, but only the buffered body and headers were cleared. A content provider (sized or chunked) stayed on the request, so the original payload was sent again, and for a chunked provider the unframed chunks also broke the keep-alive connection.
The redirect also percent-decoded the Location path before sending it. That turned %23, %3F and %25 into a fragment, a query delimiter and a different octet, so the client requested a different resource than the one named, and made set_path_encode(false) fail on any Location containing %20. The path is now sent as given.
The heartbeat thread waited on ping_cv_ without a predicate, so a
spurious wakeup ended the wait early and sent a ping before
ping_interval_sec_ had elapsed. With max_missed_pongs enabled, the
early ping also counted toward the pong timeout.
Pass a predicate to wait_for so that the wait only ends when the
interval elapses or the connection is closed.
Reported in #2612.
An event with an empty id field must reset the last event ID, so that
no Last-Event-ID header is sent on reconnect. run_event_loop only
updated last_event_id_ when the id was non-empty, so it could not tell
an empty id field from an event with no id field, and kept sending the
stale ID.
Track whether an id field was seen with a has_id flag, as has_data
does for the data field, and add a regression test.
A client may pipeline its requests (RFC 9112 9.3.2). The server read the
following request(s) into a per-request SocketStream buffer, discarded them
with the stream after the first response, and then waited in keep_alive()
for socket data that never came, closing the connection after the
keep-alive timeout. Over TLS the bytes stayed decrypted in the TLS library,
where keep_alive() could not see them either.
Create one stream per connection and serve a request that is already
buffered (Stream::is_readable()) without waiting in keep_alive().
Keeping the buffer means an extra CRLF that some clients send after a
request body is now parsed as the next request line, which answered 400
and closed the connection. Ignore one empty line before the request-line,
as RFC 9112 2.2 recommends.
Fixes#2599
A field line without a colon kept the \r of a CRLF line ending in its
name, so "data\r\n" was not recognized. Strip the \r once per line
before parsing instead of from each value.
An event without a data field was neither dispatched nor cleared, so
its event type leaked into the next event and its id only reached
last_event_id after a later event was dispatched. Reset the message on
every blank line and record the id even when nothing is dispatched.
The parsing tests exercised a copy of parse_sse_line that had drifted
from the real one. Run them through SSEClient against a local server
instead, and add regression tests for the cases above.
SSE events may contain an empty data field, and a data field without a colon also has an empty value. Track whether a data field was seen separately from the accumulated payload so empty events are dispatched and leading empty lines are preserved. Add an integration regression test for both forms.
parse_port and the NO_PROXY CIDR prefix parsing each repeated the same
from_chars call, full-consumption check and range check. Move that into
a parse_int_in_range helper and use it in both places.
parse_port checked only the error code of from_chars, which stops at
the first non-digit, so http://host:80abc was accepted as port 80, and
a redirect Location with such a port was followed. RFC 3986 defines
port as *DIGIT. Require the whole string to be consumed, as #2590 does
for quality values.
Every backend loads the Windows ROOT and CA stores, but Windows adds a
root to them only when CryptoAPI needs it to build a chain. On a machine
that had not needed a root yet, the backend rejected the chain before
the CryptoAPI check ran, so Windows never fetched the root (for example
OpenSSL error 20 for accounts.spotify.com under Starfield Root G2).
With Windows verification enabled, the backend's chain verdict is no
longer used. Mbed TLS and wolfSSL skip chain verification during the
handshake, and the post-handshake verify result is ignored. The
CryptoAPI check becomes mandatory: a leaf that cannot be encoded now
fails the connection instead of skipping the check, and the chain must
allow server authentication, which the backends used to check.
A server certificate verifier works on the backend's chain verification,
so when one is set the backend still decides and CryptoAPI only adds its
own check, as before.
ClientCertMissing now disables hostname verification: cert.pem does not
match HOST, and on Windows the hostname check fails before the chain
check.
Refs #2596
CryptoAPI got only the leaf, so it fetched an issuer from the leaf's AIA
URL instead of using the intermediates the server sent. For
accounts.spotify.com that issuer chains to Certainly Root R1, which
Windows does not trust, while the server's own chain ends at Starfield
Root G2.
Add tls::get_peer_certs(), which returns the certificates the peer sent
in the same way get_ca_certs() returns the CA certificates, for every
backend. The CryptoAPI check puts them into a memory store that it
passes to CertGetCertificateChain(), skipping any certificate that
cannot be added. wolfSSL keeps the received chain only when built with
SESSION_CERTS; without it, CryptoAPI still gets the leaf alone.
Refs #2596
A request whose Content-Length was present but not a valid decimal length
(e.g. "42, 42", "+42", "0x2e" or empty) was treated as having no body
unless a handler read it: no 400 was returned, the body was not drained,
and the bytes after the header block were parsed as the next request on
the keep-alive connection. Reject such a request with 400 and close the
connection before routing, as RFC 9112 Section 6.3 requires.
The server also kept reading after a response that announced
Connection: close. A rejected request line or header block left the rest
of the message to be parsed as a new request, and an error response to a
bodyless request did the same with whatever followed. Close the
connection whenever the final response carries Connection: close
(RFC 9112 Section 9.6), and mark the two request-head rejection paths
closed explicitly as the other rejection paths already do.
SSEClient::wait_for_reconnect() sleeps in 100ms steps until the
reconnect interval has elapsed. With an interval of 0 (for example
"retry: 0" from the server, or set_reconnect_interval(0)) it never
slept at all, so a server that sends "retry: 0" and closes the stream
made the client reconnect in a tight loop. set_max_reconnect_attempts()
does not stop this either, because each successful connection resets
the attempt counter.
Always wait at least one step (100ms). Intervals of 1-99ms already
waited 100ms because of the step size, so only 0 and negative values
change behavior.
parse_sse_line checked only the error code of from_chars, which accepts
a leading '-' and stops at the first non-digit, so retry: -1 made the
client reconnect without waiting and retry: 10s set 10 ms. The SSE spec
ignores a retry value that is not all ASCII digits.
Share the server's request-target check as fields::is_request_target()
and use it in write_request_line too. The client previously used
is_field_value(), which let an embedded SP or HTAB through.
encode_path() only escaped CR/LF among the control characters, so with
path encoding enabled a path like "/a\tb" would now be rejected instead
of sent. Percent-encode every control character (0x00-0x1F, 0x7F).
RFC 9112 §3.2 does not allow control characters in the request-target,
and §2.2 requires a bare CR to be treated as invalid. parse_request_line
accepted them, so e.g. "GET /a\rb HTTP/1.1" was routed normally. Reject
any byte that is not VCHAR or obs-text with 400 Bad Request. obs-text is
still allowed since some clients send raw UTF-8 in the target.
req.path is percent-decoded, so a request like GET /%0D%0A... put a
literal CR/LF into the NGINX-style log lines and let a client forge
extra entries. Log the raw req.target (matching NGINX's $request) and
escape '"', '\', control and non-ASCII bytes as \xHH the way NGINX
does. Also note in the README logging section that req.path may
contain control characters and should be escaped before logging.
parse_quality accepted values such as q=0.5junk because it checked only the conversion error and ignored the returned end pointer. Require the numeric parser to consume the complete q parameter so malformed Accept values are rejected and invalid Accept-Encoding weights are ignored. Add regression cases for both headers.
A file served from a mount point or through set_file_content() left in two
writes, one for the status line and headers and one for the body, because the
body came from a content provider. A small file is now read into the header
buffer so the whole response leaves in a single write. Only file-backed
providers are coalesced this way: a user-supplied provider may produce its data
over time, and holding the headers back until it finishes would stall the
client.
A large set_content() body was copied into the header buffer before being
sent. A body of CPPHTTPLIB_SEND_BUFSIZ or more is now written directly after
the headers, which saves the copy at the cost of one extra write.
* Added a feature test to auto enable/disable CPPHTTPLIB_USE_NON_BLOCKING_GETADDRINFO
* turned status: WARNING 'GetAddrInfoExCancel is unavailable; disabling non-blocking getaddrinfo.' into a warning
* added ws2_32 for the GetAddrInfoExCancel. this catches previous false negatives
---------
Co-authored-by: Tobias Wallner <tobias.wallner@qtlabs.at>
open_stream wrote the request line and each header straight to the
socket, so a header rejected by check_and_write_headers left the request
line and the headers before it on the wire. Build them in a BufferStream
first and flush once, as write_request and the WebSocket handshake do.
process_request reads the response even after write_request fails, so
that an early response (e.g. 413/414) sent while the body is still being
uploaded is not lost. A request line or header rejected while the
request is being built in memory never reaches the socket, though, so
no response will come and the client blocked until the read timeout (or
until the server closed the idle connection).
write_request now reports such a local rejection, and process_request
returns immediately in that case. Socket write failures still read the
response as before.
write_request_line checked the request target for CR/LF but concatenated
the method verbatim. A method carrying CR/LF could smuggle a whole
request ahead of the real one, and the client would take the smuggled
request's response as its own. A method with a space or an empty method
put a malformed request line on the wire.
Require the method to be a token (RFC 9110 Section 9.1) before anything
is written. All three callers (the buffered client path, open_stream and
the WebSocket handshake) go through this function and fail with
Error::Write, as they already do for a rejected target.
Claude-Session: https://claude.ai/code/session_01NTDesJQTQPEuu4o4XCu69g
* reject control characters in chunk extensions in read_payload
* Bound every chunk-size line scan by the line terminator
read_payload() ended its scans of one line buffer two different ways: the
hex-size parse and the space skip that follows stopped on the NUL that
stream_line_reader::append() writes, while the new chunk-ext check walked
to an explicit end pointer. Compute that end pointer first and bound all
of them by it, so no scan depends on the buffer's NUL and the terminator
can never be read as line content.
The bare-LF branch is reachable only under
CPPHTTPLIB_ALLOW_LF_AS_LINE_TERMINATOR, where getline() ends the line on
an LF that is the terminator rather than extension text. Say so: the
comment below it explains why a bare LF inside the line is rejected, and
without that note the two read as contradictory. Its guard no longer
depends on the scan cursor either, since all it ever needed was a check
that there is a byte to look at.
* Reuse the chunked-body helper in the chunk-ext acceptance test
AcceptsChunkExtension repeated expect_chunked_body_rejected()'s body
verbatim apart from the expected status, so parameterise the helper on
the status and keep the rejection wrapper for the existing callers. The
decoded body is already checked by the /chunked handler, so asserting
the status is all the new test needs.
Also record why the control-character literal stays split: a hex escape
consumes every hex digit that follows it, so "\x01b" would be the single
byte \x1b rather than \x01 followed by 'b', and joining the halves would
quietly change what the test sends.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
tls::shutdown() on OpenSSL called SSL_shutdown() a second time to wait
for the peer's close_notify. An idle keep-alive client never sends one,
so closing its connection held the worker thread until the read timeout,
and Server::stop() waited for it. Send close_notify and return, as the
Mbed TLS and wolfSSL backends already do.
The server wrote 100 Continue as soon as it saw the expectation, before
pre_routing_handler, pre_request_handler, or routing ran. A request
those handlers rejected, or one that matched no route, still invited
the client to send a body the server would never read.
Defer the interim response until the body is about to be read. If the
request is answered without reading the body, 100 Continue is never
sent and the connection is closed, since whether and when the client
sends the body is unknown.
Also treat a 417 returned by expect_100_continue_handler as the final
response. It used to be written as a bare status line, after which the
request was processed and a second response was written.
The WebSocket upgrade path matched the route and switched protocols
without setting req.matched_route or calling pre_request_handler, so a
check placed there (e.g. authentication) never ran for WebSocket
routes. Set matched_route and run the handler before the upgrade; if it
handles the request, reply with a regular HTTP response instead of 101.
Also write rejected upgrade responses (from pre_routing_handler too)
with write_response_with_content, so they carry Content-Length. Without
it, a client reading the body waited until the keep-alive timeout.
`sed -i ''` is BSD-only: GNU sed takes the '' as the script and the expression as a file name, so `just release --run` failed on Linux before touching anything.
199d7ee made read() return the new ReadResult::Timeout for every read
timeout and leave the connection open. The compile-time server default
(CPPHTTPLIB_WEBSOCKET_SERVER_READ_TIMEOUT_SECOND, 300s) is always in
effect, so a handler written as `while (ws.read(msg))`, the form the
README's Quick Start uses, no longer ended when a peer went quiet:
Timeout is non-zero, so the loop ran its body again with the previous
message still in `msg`, and the worker the backstop is meant to reclaim
was never released. Nothing caught it because every test of the new
result set a timeout explicitly and checked the result by value, and the
heartbeat tests keep the connection alive with pings.
The two timeouts mean different things. One the caller sets through
set_read_timeout() is a request for control back, and is reported as
Timeout on a still-open connection. The compile-time default is a
backstop against a peer that has gone quiet, and elapsing it is now a
failure again: read() returns Fail and closes the connection, as it did
before 199d7ee. WebSocket tracks whether set_read_timeout() was called,
and WebSocketClient carries the same flag over to the WebSocket it
creates on connect().
Tests use the heartbeat binary, which compiles both defaults down to 3s:
a `while (ws.read(msg))` server handler runs its body once and exits
when the client falls silent, and a client that never set a timeout gets
Fail with the connection closed. The README and cookbook now say which
timeout produces Timeout.
Claude-Session: https://claude.ai/code/session_01EF5uZ1X2kaHhqJ8VgfjVaQ
ProxyTest, RedirectTest.HTTPBin*, KeepAliveTest and ProxyTest.SSLOpenStream
still sent their requests through the squid proxies to the external
httpbingo.org, so an upstream hiccup there failed CI with no code change
involved (KeepAliveTest.SSLWithDigest got a 502 on its first /get).
Switch them to the "httpbin" container (nginx + go-httpbin) that
BaseAuthTest/DigestAuthTest already use. go-httpbin serves /get,
/redirect/n and /digest-auth the same way, so the test logic is
unchanged; the SSL variants disable certificate verification for the
self-signed test cert, as BaseAuthTest.SSL does.
RedirectTest.YouTube* is left pointing at youtube.com since it exercises
a real cross-host, http -> https redirect chain.
Claude-Session: https://claude.ai/code/session_0148ZAzsuYRYXkwcA7UFeh95
The intermittent failures it was tracking stopped after the graceful
drain before close in 8e702d3: no windows-without-SSL test failure on
master since 2026-08-09. Drop the reporting step, the issues: write
permission it needed, and the run_tests step id that only it used.
Claude-Session: https://claude.ai/code/session_0148ZAzsuYRYXkwcA7UFeh95
* reject ambiguously framed responses in client read paths
* Accept non-chunked Transfer-Encoding responses in the client framing guard
RFC 9112 §6.3 treats requests and responses differently when the final
transfer coding is not chunked: a request's body length cannot be
determined and the server must answer 400, but a response's body simply
runs until the server closes the connection. read_content() and the
open_stream() body reader already do that, so such a response is not
ambiguous and rejecting it broke valid responses such as
"Transfer-Encoding: gzip" followed by a close.
Keep rejecting a Transfer-Encoding paired with a non-zero Content-Length,
which is the actual ambiguity, and drop the non-chunked clause from both
client read paths.
Tests: check that rejection surfaces as Error::Read, that a non-chunked
Transfer-Encoding response is read until close on both paths, and that
HEAD, 204 and 304 responses with both framing headers are not rejected.
Claude-Session: https://claude.ai/code/session_01JYPWKpbp4a881EdpEf2xSi
* Share the framing check and reuse existing test helpers
Factor "Transfer-Encoding with a non-zero Content-Length" into
detail::has_conflicting_content_length() next to
is_chunked_transfer_encoding(), and call it from the server request
guard and both client read paths so the rule and its RFC 9112 §6.3
rationale live in one place.
In the tests, drop the POSIX-only raw socket helper in favour of the
existing serve_single_response() and read_all(), which also lets the
tests run on Windows. Fold the stream-only test into the buffered one so
each case checks both Get() and open_stream(), and cover the HEAD/204/304
exclusion on the open_stream() path too.
Claude-Session: https://claude.ai/code/session_01JYPWKpbp4a881EdpEf2xSi
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* Reject a Range first-byte-pos that overflows ssize_t
parse_range_header initializes first to the -1 sentinel that means "no
first-byte-pos" and only overwrites it when detail::from_chars succeeds.
On std::errc::result_out_of_range the assignment is skipped and -1
survives, so "bytes=9223372036854775808-100" is parsed as the suffix
range "bytes=-100" and range_error serves the last 100 bytes instead of
returning 416.
Before the parser was rewritten onto detail::from_chars, std::stoll threw
std::out_of_range on the same input, the catch arm added in 8f8761e for
issue #705 returned false, and the request was answered with 416. The
catch arm is still there but from_chars reports through an error code, so
nothing reaches it any more.
get_header_value_u64 and parse_port already reject an out-of-range value
at their from_chars call sites; this was the remaining one that dropped
the error.
The last-byte-pos side is deliberately unchanged: -1 there is the
documented RFC 9110 14.1.2 "remainder of the representation" value, so an
oversized last-byte-pos stays accepted.
* Simplify the Range first-byte-pos overflow check
Parse the first-byte-pos straight into first, since a failed parse now
returns before first is read, and fold the overflow test into the
existing batch of rejected ranges. Also note on the last-byte-pos side
why an overflow there deliberately keeps -1.
Claude-Session: https://claude.ai/code/session_01JYPWKpbp4a881EdpEf2xSi
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* send each credential only to its own hop in write_request
An SSLClient behind a proxy sent Proxy-Authorization inside the TLS tunnel, where the origin reads it, and sent the origin's Authorization on the CONNECT request the proxy reads. Attach each only on the message its hop actually reads.
* Keep default headers off the CONNECT request
set_default_headers() is typically used for origin credentials such as
Authorization, Cookie or API keys, but they were also attached to the
CONNECT request an SSLClient sends to its proxy, in plaintext before the
TLS tunnel exists. Default headers now go only on requests the origin
reads, the same split the previous commit makes for set_basic_auth and
set_bearer_token_auth.
Claude-Session: https://claude.ai/code/session_01JYPWKpbp4a881EdpEf2xSi
* Simplify per-hop credential handling and its tests
Flatten the Authorization insertion in write_request into one guard with
an else-if (Basic already took precedence over Bearer), and shorten the
comments around it. Fold DefaultHeadersStayOffConnect into the
CredentialsStayWithTheirHop helper, which now takes the list of headers
that must reach only the origin.
Claude-Session: https://claude.ai/code/session_01JYPWKpbp4a881EdpEf2xSi
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
These tests exercise the squid proxies by hitting /basic-auth and
/digest-auth on an external httpbin-style site. That site's identity has
already moved twice (httpbin.org -> httpcan.org, per #2300) chasing
uptime, and httpcan.org itself is now down (Cloudflare 502 from its
origin), failing CI with no code change involved.
Adds two containers to the existing squid docker-compose stack instead:
go-httpbin (mccutchen/go-httpbin) as the backend, and an nginx sidecar in
front of it under the single "httpbin" hostname so both the NoSSL tests
(port 80) and the SSL tests, which CONNECT-tunnel through the proxy to
port 443, resolve the same name -- go-httpbin only listens on one port at
a time, so it can't serve both protocols itself. nginx uses the repo's
existing self-signed test cert; the SSL client tests already disable
verification for it like other self-signed-cert tests in this suite.
go-httpbin was picked over the more feature-complete kennethreitz/httpbin
after finding the latter accepts a wrong digest-auth username as long as
the password matches -- confirmed with a direct curl against the
container, unrelated to anything in this repo. go-httpbin correctly
rejects both. The trade-off is losing SHA-512 digest-auth coverage here,
since go-httpbin only implements MD5 and SHA-256; nothing else in the
suite exercises SHA-512 digest auth against a live server. Response body
assertions are adjusted to go-httpbin's actual JSON shape (an added
"authorized" field, no "algorithm" field), and the domain changes from
httpcan.org to the self-hosted "httpbin".
This only affects 'make proxy'/'make proxy_mbedtls'/'make proxy_wolfssl'
and the Proxy Test CI workflow -- the default 'make' target is untouched.
The previous commit pinned CI and the pre-commit hook to a fixed
clang-format 23.1.0, but the maintainer develops on macOS against
whatever version `brew` currently installs, which changes over time
as Homebrew updates the formula.
style-check now runs on macos-latest and installs clang-format via
`brew install`, so it tracks the same moving target the maintainer's
Mac does. The pre-commit hook switches from pre-commit's own pinned
mirror to a local hook that shells out to the system clang-format,
so a local commit and CI both go through the same Homebrew-installed
binary rather than two independently versioned copies.
Trade-off: this reintroduces the non-determinism a fixed pin avoids
-- a commit's style-check result can now change over time as Homebrew
updates the formula -- but that mirrors how the maintainer already
develops, which is the point.
Also install coreutils in CI: the style_check Makefile target needs
grealpath's --relative-to, which the macOS-native realpath lacks.
CI relied on ubuntu-latest's default apt clang-format (18.1.3), while the
pre-commit hook was pinned to a different 18.x build. Neither tracked a
specific, deliberately-chosen version, and the two could drift from each
other and from whatever a contributor has installed locally.
Both now pin the same clang-format 23.1.0 (latest stable), installed via
pipx in CI. Also scope the pre-commit hook's file matcher to the same set
of files test/Makefile's style_check target checks, since the previous
\.(cpp|cc|h)$ pattern reached into vendored code (test/gtest,
benchmark/crow) that must stay untouched.
Reformat httplib.h's brace-init spacing to match 23.1.0's output.
Defined next to its non-template overload, it landed in the part of the header
that test_split compiles into httplib.cc, leaving a user TU's instantiation
with nothing to link against. The WebSocketClient overload of the same name has
always sat up with the class definitions; this one belongs there too.
Only test_split sees it -- a header-only build instantiates the template
wherever it is written, which is why the regular test target stayed green.
read() collapsed every failure into Fail and marked the connection closed with
it, so a read timeout could not be used to get control back and send on the
same connection -- it killed the connection instead. The information was
already there and thrown away: SocketStream::read records Error::Timeout, and
read_websocket_frame flattened it into a bool.
ReadResult gains Timeout, reported only when the timeout elapsed on a frame
boundary with nothing consumed, which is the only case where the stream can be
read again. Every multi-byte field now loops until it has its bytes, which also
fixes a frame header straddling the read buffer's boundary failing the frame:
Stream::read is allowed to return less than asked for, and only the payload
was reading in a loop.
ws::WebSocket::set_read_timeout() lets a server handler bound its own reads,
and WebSocketClient::set_read_timeout() now reaches an already-open connection
instead of only seeding the next connect().
The read timeout macro splits in two. A client waits forever by default -- a
read timeout is the caller's tool for taking back control, not a liveness
check, which is ping/pong's job -- while a server keeps the 300s that reclaims
a worker from a peer gone quiet. Defining the old name still sets both.
Also record a reason on the two WebSocketSSLStream::read failure paths that
returned -1 without one, so get_error() cannot report a previous call's
timeout, and make SocketStream's read timeout atomic now that it can be
changed while a read is in flight.
* Don't compress a response whose handler already set Content-Encoding
* Don't compress a pre-encoded response served from a file
The guard that stands down when a response already names a content coding
covered the responses that settle their coding in `apply_ranges()`, but a
file-backed one settles it in `static_file_encoding()`, which asked the
content-type overload and so never saw the field. With static file
compression enabled, a mount point naming the coding for a tree of
build-time compressed assets, and a handler setting the field on a
`set_file_content()` response, both had their stored bytes compressed a
second time and a second `Content-Encoding` field line appended.
A file-backed response has not been given a content type by the time its
coding is decided, which is the only reason it could not go through
`encoding_type()`. It takes the type as an argument now, so both paths share
the one guard instead of carrying a copy each.
`Response::content_encoding_` becomes `content_coding_`, after what it
holds. It names the coding chosen for the body, which is what its own
comment already called it, while the old name read as the value of the
`Content-Encoding` field whose presence is exactly what forces the coding to
`None`.
README gains the behaviour, including the part that stays with the handler:
`Vary` is added only to a coding the server chose, so a handler that picks a
representation from `Accept-Encoding` has to add the field itself.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
parse_www_authenticate() accepted any WWW-Authenticate: Digest
challenge that carried at least one auth-param, so a server sending
e.g. Digest qop="auth" with no realm/nonce would make it through.
make_digest_authentication_header() then dereferences auth.at("realm")
and auth.at("nonce") unconditionally, throwing std::out_of_range with
no try/catch on the retry path, which terminates the client process.
Now require both realm and nonce before treating a Digest challenge as
usable, same as if no Digest challenge were present at all.
The reporting step for issue #2533 was gated on failure(), which is true
when any step in the job failed. Two build failures on feature branches
were posted to the issue as flaky test recurrences, with the body falling
back to "Could not extract failed test name" because no test had run.
Gate it on the test step's own conclusion instead, and skip posting when
no [ FAILED ] line turns up in the shard logs. The explicit failure()
stays because an if expression with no status check function gets an
implicit success().
CPPHTTPLIB_MULTIPART_FORM_DATA_FILE_MAX_COUNT is enforced only in
Server::read_content(), where parts are accumulated into req.form. The
streaming ContentReader path keeps nothing and was never in scope, but
this was undocumented (GHSA-923p-8q8g-xcqj). Note the split in the README
and show how to bound the part count from inside a ContentReader handler.
* Drop the claim that small bodies skip compression
There is no size threshold anywhere in the compression path.
encoding_type() gates on the content type and Accept-Encoding only, and
apply_ranges() compresses whatever body it is given, so a two-byte
text/plain response comes back gzipped at 22 bytes.
Say what actually happens and leave the decision to the handler.
* Compress static file responses behind an opt-in (Fix#2545)
apply_ranges() runs the compressor inside the branch it takes when
res.body is non-empty. A response served from a file leaves res.body
empty and sets content_length_, so it took the other branch, which
writes Content-Length and returns; encoding_type() was computed before
the split and never consulted on that side. The same bytes handed to
set_content() came back gzipped, which left set_mount_point() and
Response::set_file_content() as the one path that missed out.
Add Server::set_static_file_compression(), off by default so nothing
about an existing server changes. When it is on, the file-backed
provider is run through the compressor into res.body ahead of the rest
of apply_ranges(), so the response is framed the way set_content()
already frames one: it keeps its Content-Length, and HEAD still reports
the size a GET would return.
Ranges are answered from the identity representation, since RFC 9110
applies Range after content coding and slicing a compressed body would
mean compressing the whole file first. The ETag carries the coding it
belongs to, so a client that cached the compressed form revalidates
against its own validator rather than the identity one. Both the ETag
and the body take their coding from static_file_encoding(), so the two
cannot disagree.
Providers registered with set_content_provider() are left alone. zlib
buffers until its window fills, so running one through a compressor
would hold back writes that a caller expects to reach the peer as they
are produced.
The compressed bytes stay in memory until the response has been
written, so the peak cost scales with requests in flight.
set_static_file_compression_max_length() bounds it, defaulting to 4MB.
* Add a minimum size for static file compression
Compressing a file that already fits in a single 1500-byte MTU does not
get it to the client any sooner, and a file of a few bytes comes back
larger than it went in once gzip's header and trailer are added. Every
other server draws this line: nginx's gzip_min_length, Caddy's
minimum_length, IIS's minFileSizeForComp, CloudFront's 1000-byte floor.
The note this replaces told callers to decide in the handler. A response
served through set_mount_point() has no handler to decide in, so the
floor has to live in the server. It defaults to 1400 bytes, the size
that fits inside one MTU with room for headers.
set_static_file_compression_min_length() moves it, and
CPPHTTPLIB_STATIC_FILE_COMPRESSION_MIN_LENGTH sets the default at
compile time. The empty-file case keeps its own early-out so that a zero
floor still cannot turn an empty body into a 20-byte gzip stream.
The two bounds now read as a pair, so the documentation says what each
one is for: the lower bound is about what is worth compressing, the
upper bound about what one request is allowed to cost.
Every file under test/www except 1MB.txt is below the default floor, so
the tests that need a small file compressed lower it explicitly.
parse_disposition_params() and extract_media_type() both split on every
';' and then on every '=', with no idea that a parameter value can be a
quoted-string. RFC 9110 5.6.6 allows ';' and '=' inside one, so
filename="report=v2.pdf" came out as v2.pdf", and filename="a;b.txt" was
truncated at the semicolon and left a bogus parameter behind.
The same defect reached the boundary. RFC 2046 5.1.1 allows '=' in a
boundary, which forces a sender to quote it, so the common MIME form
boundary="----=_NextPart_000_0000_01D9" parsed as
_NextPart_000_0000_01D9".
Add split_unquoted(), which is split() with the one extra rule that a
delimiter inside a quoted-string is not a delimiter, and route both
parameter parsers through it. The key/value split, duplicated verbatim
in the two of them, moves into divide_param_pair(). That one divides at
the first '=' without tracking quotes: 5.6.6 makes the key a token, so
no quote can precede the separator, and reusing divide() keeps this off
the per-byte scan.
A backslash stays an ordinary character here. Both browsers and
httplib's own sender percent-encode '"' rather than escaping it, and
recognizing a quoted-pair without also unescaping it would just trade
one wrong value for another.
write_content_with_progress() advances its offset only by what the provider
writes, so a provider that reported success without writing anything and
without calling done() was handed the same offset and length again on the next
pass. With the peer still connected it spun there, re-entering the provider as
fast as the loop could run.
make_file_body()'s provider was one way to reach this and was fixed in #2566,
but any user-supplied provider can do the same. Treat a pass that makes no
progress as a short body, which is how done() called early is already handled.
make_file_body() measures the file once and that length is already the
response's Content-Length. The provider re-opens the file by path on each
call, so if the file has been truncated since, the read comes up empty and
the provider returned true without writing. write_content_with_progress()
advances its offset only by what was written, so it called the provider
again, got nothing again, and kept spinning until the peer gave up.
Return false instead, as every other failure in this provider does.
parse_accept_header() rejected any Accept value with a leading, trailing
or doubled comma, and Server::process_request() validates Accept before
routing, so "Accept: text/html," was answered 400 Bad Request on every
route.
RFC 9110 Section 5.6.1.2 requires a recipient to parse and ignore empty
list elements in a #rule list, so those values are legal. split() already
trims each element and skips the empty ones, which made the guard inside
the callback unreachable as well; drop both and let the empty elements
fall away. The header length limit bounds how many a sender can send, so
ignoring all of them cannot be used as a denial-of-service vector.
get_combined_header_value() keeps skipping empty field lines, but that
skip is no longer observable through a request now that a stray comma
parses cleanly, so it gets its own test.
Server::process_request() wraps only routing() in a try/catch.
Everything else the user supplies runs outside it:
- the content provider, from write_response_core()
- post_routing_handler_, error_handler_, logger_
- expect_100_continue_handler_
- a WebSocket handler, and pre_routing_handler_ on the upgrade path
An exception from any of those unwinds out of process_and_close_socket()
into the task queue, which calls the job without a catch, so it reaches
the top of a pool thread and terminates the process. One handler that
throws takes down every other connection the server is holding.
Add Server::serve_guarded() and run the serving loop through it in both
process_and_close_socket() overloads. The exception is not turned into a
500: by the time a content provider runs, the status line and headers
are already on the wire, so there is nothing left to replace. Report it
through the error logger as Error::UserCallbackException and drop the
connection, which is what the peer observes regardless. Requests on
other connections are unaffected, and the socket is still drained and
closed - which unwinding used to skip on the non-SSL path, since
drain_and_close_socket() sits after the call rather than in a scope
guard.
The error logger is a user callback too, so the report inside the guard
is itself wrapped: a throwing logger must not be able to open the guard
back up.
Adds ServerExceptionTest: a throwing content provider, post-routing
handler, WebSocket handler and error logger, plus the content provider
case against SSLServer, each checking that a later request on a new
connection still succeeds. Every test runs the server on a single worker
thread, so a guard that catches the exception but still loses the thread
shows up as the follow-up request never being served. Note that all of
them abort the test binary without this change - which is the bug, but
it means a regression here fails the run rather than one test.
write_content_chunked()'s sink treated "the provider wrote nothing" as
"the provider has finished":
data_available = l > 0;
so sink.write(p, 0) ended the loop. Only done()/done_with_trailer()
emit the terminating zero-length chunk, so the body was left
unterminated - and the function still returned Success, because the
post-loop check only reports the is_shutting_down() case. The peer waits
for a last chunk that never arrives, and on a keep-alive connection
anything written next is parsed as a chunk-size line.
A provider reaching a pass with nothing to hand over is ordinary:
popping an empty buffer off a queue, or a compressor that has consumed
its input without producing output yet. It is not the end of the
message.
Ignore zero-length writes instead. A zero-length chunk is the terminator
in chunked coding, so it must never be emitted mid-body either way, and
data_available is now controlled only by done()/done_with_trailer().
This matches write_content_without_length(), where the sink's write
never ends the body.
The old behaviour cannot have been relied on: it produced an
unterminated response, so a provider using it never worked in the first
place.
DataSink has four callbacks, but only write is assigned by every writer
that hands a sink to a content provider:
write_content_with_progress() write, is_writable
write_content_without_length() write, is_writable, done
write_content_chunked() all four
send_with_content_provider...() write
get_multipart_content_provider() write, done (cur_sink)
A provider that calls one of the unassigned ones invokes an empty
std::function and throws std::bad_function_call. Nothing on that path
catches it, so it unwinds out of the thread running the provider and
terminates the process. The README's own idiom is enough to hit it:
sink.done() is documented for the without-length overload, but a
provider registered through set_content_provider() with a length gets a
sink where done is empty.
Default the three optional callbacks instead. A sink is writable unless
a writer says otherwise, and a sink that cannot carry trailers still has
to finish, so done_with_trailer() falls back to done(). Capturing this
for that is safe because DataSink is neither copyable nor movable.
A no-op done() alone would only trade the crash for a hang on the two
length-framed paths: both loop until offset reaches the promised length,
so a provider that reports itself done without writing would be called
again immediately, forever. Both now record that the provider finished
and stop, and the short body is reported as a write error. The client
path gains that check for the compressor-failure exit as well, which
used to send a truncated request body without reporting anything.
cur_sink in get_multipart_content_provider() now forwards is_writable
from the outer sink, so a provider item asking whether it may keep going
gets the stream's answer rather than the default.
The accept loop in Server::listen_internal() classified accept() failures
by reading errno, but Winsock reports them through WSAGetLastError() and
never touches the CRT errno. Both retry branches were therefore dead code
on Windows, and every accept() failure fell through to the fatal path,
which closes the listening socket and ends listen().
That is reachable in normal operation: a peer resetting a pending
connection before it is accepted is enough, and descriptor or buffer
exhaustion shows up under load. One such event stopped the server from
accepting anything again.
Add is_accept_resource_error() and is_accept_transient_error() next to
is_connection_error(), which already abstracts the same errno vs
WSAGetLastError() difference, and use them in the accept loop.
The POSIX sets are widened to match the Windows ones rather than being
left as they were: ECONNABORTED is the POSIX spelling of the aborted
pending connection that motivates this, and ENFILE, ENOBUFS and ENOMEM
are resource exhaustion in the same sense as EMFILE.
When accept() failed for a reason the retry branches do not cover, the
loop closed svr_sock_ but left the descriptor in the atomic. Two things
go wrong from there:
- A later stop() reads the stale value and calls shutdown()/close() on
it. By then the OS may have reused the descriptor for an unrelated
socket (a worker's keep-alive connection, or one the application
opened), and that connection is torn down instead.
- keep_alive() in the worker threads watches svr_sock_ to notice that
the server is going away, so the workers keep waiting on a listening
socket that no longer exists.
Take the descriptor with exchange(INVALID_SOCKET) before closing it,
which is what stop() already does. That also settles the race with a
concurrent stop(): whichever side takes the descriptor closes it exactly
once, and the other sees INVALID_SOCKET and does nothing.
parse_multipart_boundary only rejected an empty boundary, so a request could
declare one as long as a header line is allowed to be. A stock server accepts
up to 8146 bytes there, which is what CPPHTTPLIB_HEADER_MAX_LENGTH leaves after
"Content-Type: multipart/form-data; boundary=".
FormDataParser searches the body for "--" + boundary + CRLF with a plain
substring scan. buf_find scans for that delimiter's first byte, always '-', and
at every position that matches calls start_with, which compares until the first
mismatch. A body of '-' makes every position a candidate, and a boundary of '-'
makes each candidate compare the whole delimiter before failing at the CRLF. The
worst case is the product of the body length and the boundary length, and only
the first factor was bounded.
Measured by driving the parser directly in 16 KB reads, Apple clang 17 at
-O2 -DNDEBUG, best of three runs on an otherwise idle machine. 100 MB of '-',
the default payload limit, costs 2.59 s of CPU with a 70 byte boundary and
281.83 s with an 8147 byte one, a factor of 109. The same shape shows at 8 MB:
0.211 s, 3.081 s, 11.359 s and 22.091 s for boundaries of 70, 1024, 4096 and
8147 bytes.
RFC 2046 5.1.1 caps a boundary at 70 characters, so honoring that limit bounds
the multiplier too. The limit applies to the value after unquoting, so a quoted
70 character boundary stays valid. Only the server receive path parses a
boundary out of a Content-Type, so what clients may send is unaffected, and the
boundaries the library generates itself are 45 characters.
expect_split_multipart_ok() carries a comment saying the request sends
"Connection: close" so the response drain ends as soon as the server has
answered, but the header itself never made it into the request, so both
callers kept idling until the read timeout instead.
Add the header. EpilogueSplitAcrossReadsIsIgnored and
InitialBoundarySplitAfterLongPreamble each drop from about 3.1s to about
0.11s.
* Bound the multipart parser's buffer while it waits for a boundary
FormDataParser accumulated the entire request body whenever the declared
boundary never appeared in it. State 0 returned without erasing anything, so
the buffer grew to the full payload (100 MB by default) and buf_find rescanned
all of it on every 16 KB read. The cost grew with the square of the body size:
50 MB of '-' took 198 s of CPU on one core, and the buffer pinned the body in
memory for the whole request. One unauthenticated request was enough, and the
parser runs for any multipart request even when the handler never looks at the
parsed result.
State 0 now keeps only the last dash_boundary_crlf_.size() - 1 bytes while it
waits, which bounds both the memory and the rescan without capping how long a
preamble may be. The same 50 MB body now takes 0.14 s and the buffer stays at
one read plus the boundary. A boundary split across reads still parses, which
is what de5a255 (#2159) gave up this erase for.
State 4 buffered without bound in the same way when a boundary was followed by
neither CRLF nor "--". No further data can make such a body valid, so it now
fails right away. That is only safe because the close-delimiter branch moves to
a new state 5 that discards the epilogue: it used to stay in state 4, so an
epilogue arriving in a later read fell into this same branch. An epilogue
beginning with CRLF was then parsed as a new part and the request was rejected
with 400, which state 5 fixes as well.
Affected since v0.23.0, where de5a255 replaced the erase that had kept the
buffer in check.
* Skip buffering the multipart epilogue
Once the close delimiter has been parsed the parser is in state 5 and discards
whatever follows, but it still copied each epilogue read into the buffer before
erasing it. Return before buffering so a large epilogue spread across several
reads is dropped without being copied in at all.
* Clean up the multipart parser tests and the state 4 branch
Review follow-ups on top of the previous two commits, no behavior change.
- Move the four new tests next to the rest of MultipartFormDataTest. They
had landed in the middle of the RedirectTest block.
- Use bind_to_any_port instead of the fixed PORT, as AGENTS.md requires for
newly added servers. NoInitialBoundaryParsingIsNotQuadratic holds its port
for a couple of seconds, which matters when the suite is run sharded.
- Send "Connection: close" from expect_split_multipart_ok. The server kept
the connection alive after answering, so the response drain idled until the
client read timeout; both tests drop from about 3s to about 0.11s.
- Drop the dead `dash_.size() > buf_size()` guard in state 4 and flatten the
nested else. The check above it already guarantees two buffered bytes, and
both CRLF and "--" are two bytes, so it can never fire. Removing it is what
makes the new comment's claim readable straight off the code.
* Rename the timing test's locals to avoid a Windows macro
MSVC's <rpcndr.h>, pulled in by <windows.h>, defines `small` as `char`, so
`auto small = ...` failed to compile on the Windows jobs. Same class of
problem as the std::min / std::max collision.
* Add Server::CustomRoute() for HTTP methods outside the built-in set
parse_request_line validates the request method against a fixed whitelist and
rejects anything else with 400 before routing runs. That blocks WebDAV, where
PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK and UNLOCK are ordinary methods
defined by RFC 4918, and it blocks extension methods such as UPnP's SUBSCRIBE.
The need has been open since #847.
Registering a handler is now what makes the server accept a method:
svr.CustomRoute("PROPFIND", "/dav/:id", handler);
Because custom methods go through the normal dispatch path, patterns work the
way they do for Get() and friends, and the request body is available in
req.body. Serving these methods through set_pre_routing_handler was never
enough: the body has not been read at that point, so PROPPATCH and LOCK, which
require one, could not be implemented at all.
A HandlerWithContentReader overload is available too. The content reader gate
in routing() also fires when a custom method carries no body, matching what
expect_content() does unconditionally for POST/PUT/PATCH/DELETE, so a body-less
PROPFIND (RFC 4918 treats one as allprop) reaches its handler instead of
falling through to 404.
Method names are validated as RFC 9110 tokens, and the ten built-in methods are
refused. Seven of them are dispatched by the if/else chain in routing() before
the custom tables are consulted, so a route registered for one could never
fire; CONNECT, TRACE and PRI carry protocol-level meaning this library does not
route. A refused registration makes is_valid() return false, so listen() fails
rather than starting a server holding a handler that would never run. This is
also why SSLServer::is_valid() now chains to Server::is_valid() instead of only
checking ctx_.
Servers that never call CustomRoute() keep the previous per-request cost: the
built-in method set is checked first and short-circuits, and the custom lookup
returns early on an empty map.
* Add cookbook recipe for custom HTTP methods
The CustomRoute() docs were a section inside S01, which pushed that page to 90
lines, the longest in the cookbook, and mixed a separate feature into a page
about registering GET/POST/PUT/DELETE handlers. Move the section into its own
recipe and give it room for the part that was missing: the OPTIONS handler
returning DAV: and Allow, which WebDAV clients probe for before anything else.
S01 goes back to 68 lines and keeps a pointer to the new page.
The recipe is titled after the API rather than after WebDAV, and says outright
that generating the 207 Multi-Status XML, interpreting Depth and managing locks
are the reader's job. Routing the method is all the library does.
S23 takes order 42, so the TLS, SSE and WebSocket recipes shift to 43-57. That
only moves the sort key. Filenames, the T01/E01/W01 labels, the published URLs
and every cross-reference are untouched.
Start-Process joins ArgumentList entries with spaces, so /DIR=C:\Program
Files\OpenSSL reached Inno Setup as /DIR=C:\Program and the install landed
there. Linking still succeeded, because the import libraries were present
under that path, and the failure surfaced only when gtest_discover_tests ran
the test binary: exit code 0xc0000135, DLL not found, since PATH pointed at
C:\Program Files\OpenSSL\bin.
Quote the value, and assert that the import libraries and runtime DLLs are
where we expect before exporting PATH, so a misplaced install fails loudly at
the install step instead of quietly at load time.
The Chocolatey openssl package hardcodes a versioned slproweb URL in its
install script, and slproweb keeps only the newest build of each OpenSSL
branch. Every OpenSSL release therefore deletes the file the current package
points at, and "windows with SSL" fails at the install step with a 404 until
someone respins the package. That is what broke the job today: the package is
still at 4.0.1 while slproweb has moved to 4.0.2.
slproweb publishes a JSON manifest of its current downloads, linked from the
download page and updated at the same time as the files themselves. Read that
and take the newest 64-bit 4.x installer from it, so the URL is always live.
The SHA512 in the manifest is verified before the installer runs.
The silent flags are the ones the Chocolatey package used. /DIR pins the
install location that the CMake step already finds, instead of relying on a
registry lookup. PATH and OPENSSL_CONF are exported the same way the package
set them.
Staying on 4.x is deliberate: it keeps this job on the OpenSSL 4.0 series
rather than dropping to the 3.6 that vcpkg would provide.
close() drained the peer's Close reply with its own frame read. If an
application reader thread was inside read() at that moment, two threads
parsed frames off one stream: read_websocket_frame()'s payload loop keeps
reading until it has the declared length, so bytes stolen by the drain
were silently replaced with bytes from further along the stream. The
in-flight message kept its correct length but got the wrong content.
Add a read_mutex_ that marks which thread owns the stream's read side.
read() holds it for the whole call. close() sends the Close frame, then
drains the peer's reply (RFC 6455 7.1.1) only if it can try_lock the
mutex; otherwise it returns immediately, leaving the stream entirely to
the thread already reading it. This also fixes close() blocking for the
full close timeout when a reader thread was parked waiting on a peer
that never replies.
Add WebSocketTest.CloseDoesNotStealBytesFromConcurrentRead, which drives
a raw TCP peer that stalls mid-payload to force the race; it fails
reliably against the old code and passes against the fix.
Update README-websocket.md: close() during a concurrent read() is now
supported.
A wss:// WebSocket enters a single TLS session from several threads: the
read path, the application's send()/close(), and the heartbeat ping thread.
The existing write_mutex_ only serializes writers, so a reader's SSL_read and
a writer's SSL_write (plus the SSL_peek in is_peer_closed() on the write path)
run concurrently on the same session. OpenSSL and the other backends forbid
concurrent access to one session, so this corrupts the record layer: messages
are silently dropped, and under ASan it shows up as a heap-buffer-overflow.
It affects wss:// only; plain ws:// is unaffected because the kernel allows
concurrent recv()/send() on a socket.
Route wss:// through a new WebSocketSSLStream that serializes every TLS call
with one per-stream mutex. The socket is kept non-blocking for the stream's
lifetime and each read()/write() performs a single non-blocking TLS call under
the lock, then waits for readiness with select() outside the lock. The lock is
therefore held only for CPU-bound work, so a reader blocked waiting for data
never stalls a concurrent sender.
Because the socket is non-blocking, a TLS call can stop needing either
direction, so read() also waits for writability on WantWrite and write() waits
for readability on WantRead. A read that shares its session with the send path
has to flush pending output before it can decrypt more input, and Mbed TLS
surfaces this on every mbedtls_ssl_read(). The read timeouts are atomic since
WebSocket::close() shortens them from the closing thread while the receive
thread is inside wait_readable().
SSLSocketStream is left untouched, so ordinary HTTP/HTTPS keeps its exact code
path and performance. The heartbeat ping thread also stays, so timer-driven
pings keep working as before.
Add test_websocket_thread_safety.cc, which drives send/close/heartbeat against
a concurrent reader over wss://. Built with ASan in CI, a regression surfaces
as a heap-buffer-overflow.
The harness had a single endpoint returning a 12-byte set_content() body.
That is the one case where the response line, the headers and the body
already share a single write(), so any change to the write path measured
as noise. Comparing a gather-write branch against its merge base reported
0.993x at p = 1.000 while the same branch moved static-file throughput by
a quarter and TLS throughput by nearly half in both directions.
The server now also serves a large set_content() body and small and large
files from a mount point, over HTTPS when a certificate is given, with
--path, --large-mib and --tls selecting the combination.
ab.sh now compiles the harness from the invoking worktree instead of each
ref's own copy, so both refs run an identical workload and a ref that
predates a harness change stays measurable. Only httplib.h varies, through
-I. --timeout is exposed because bombardier's 2s default aborts large TLS
responses, which then fails the non-2xx check.
Drop the now-redundant has_header guard (get_header_value already
returns "" for a missing header, which the length check rejects),
name the "Bearer " prefix once, and cite RFC 9110 to match the
file's convention. Convert the regression test to the table-driven
form used elsewhere in test.cc and move it out of the middle of
GetHeaderValueTest so that suite stays contiguous.
detail::parse_www_authenticate() assumed a single challenge starting at
the first space in the field value and read only its first occurrence,
so a Basic challenge listed before Digest (or split across two field
lines, as some servers do) hid the Digest challenge entirely, and a
second Digest challenge with different parameters (RFC 7616 offering
both SHA-256 and MD5) could mix params from both. Combine repeated
field lines the same way the other list-valued headers do, then split
on commas that aren't inside a quoted-string so a quoted realm can
contain a comma, and track which challenge each auth-param belongs to
by the auth-scheme token that starts it. Also require at least one
auth-param before reporting a Digest challenge as found, since an
empty challenge can't produce a usable Authorization header.
RFC 9110 7.8 defines Upgrade as a comma-separated list of protocols and asks
recipients to match each protocol-name case-insensitively; RFC 6455 4.2.1 asks
for a header field containing the value "websocket". Both handshake checks
instead read occurrence zero and required the whole field value to be exactly
"websocket", so a client offering "websocket, HTTP/3.0" -- or naming websocket
on a second Upgrade field line -- was answered 404 rather than 101.
This is the defect ffe2a1c fixed for Connection two lines below, and
has_header_token() is already called in both of these functions.
The client-side check loosens what we accept back from a server, which is the
same reading: a server answering 101 may name websocket alongside another
protocol, and rejecting that handshake was ours to get wrong.
The four ExpectTokenTest cases landed inside the #ifndef _WIN32 that guards the
10 GiB content-provider test below them, so Windows never compiled them and the
green Windows jobs said nothing about the fix. Nothing in them is POSIX-only --
they use the same helpers as ConnectionTokenTest, which sits outside any guard
-- so move them above the guard.
probe_expect() re-implemented send_request(), down to the create_client_socket
argument list. Call send_request() instead, with Connection: close so its read
loop ends at the response rather than idling to the read timeout; the Connection
check runs before the Expect block, so it does not disturb what is under test.
Move the new Content-Encoding case below its siblings. Appending it to the tail
of the comment block left the "whole token" paragraph reading as documentation
for a test about repeated field lines. The paragraph above it had been detached
from KnownEncodingWithoutSupportIsReported the same way one commit earlier; put
that one back too.
RFC 9110 Section 10.1.1 defines Expect as a comma-separated list, states that
its value is case-insensitive, and requires a server that receives a
100-continue expectation in an HTTP/1.0 request to ignore it. Comparing the
whole field value against "100-continue" met none of those.
An HTTP/1.0 request asking for 100-continue was answered with a 100 (Continue)
interim response, which that section forbids. "100-Continue" and
"100-continue, foo" were both read as no expectation at all, so a client that
waits for the interim response before sending its content waited for a response
that was never coming.
Route the check through has_header_token(), which walks every field line and
compares complete tokens case-insensitively, and skip it for HTTP/1.0. An
expectation cpp-httplib does not recognize is still ignored rather than
refused; the 417 the section offers for one is a MAY, not a requirement.
RFC 9110 Section 5.3 makes a Content-Encoding spread over several field lines
the same message as the comma-joined one, so the two have to be read the same
way. Reading occurrence zero did not: a response carrying "gzip" on two field
lines was decoded as a single gzip coding, so a body the sender says was
encoded twice came back after one pass -- still compressed, but presented to
the caller as decoded. The same value written as "gzip, gzip" on one line took
the pass-through path instead.
Read the combined value at both sites. A value naming several codings matches
none of the ones cpp-httplib implements, so both representations now take the
pass-through path that prepare_content_receiver() already documents for an
unrecognized coding.
This does mean a sender that repeats "Content-Encoding: gzip" on two lines for
a body it gzipped once no longer has that body decoded. There is no way to tell
that sender apart from one that really did encode twice, and the conservative
reading is the one the field value states.
is_brotli_encoding() and is_zstd_encoding() searched the Content-Encoding
value for "br" and "zstd" as substrings, while is_zlib_encoding() beside them
compared the whole value. So "fibre" and "librarian" were read as Brotli and
"x-zstd-ish" as Zstandard, and "gzip, br" -- a value naming two codings, which
cpp-httplib does not support -- was labeled Brotli and run through a Brotli
decompressor over gzip data.
RFC 9110 8.4.1 defines a content coding as a token, so compare the whole value
case-insensitively as the zlib check already does. A value naming several
codings no longer matches any of them and takes the pass-through path
prepare_content_receiver() already documents for an unrecognized coding.
contains_case_ignore() has no callers left.
RFC 9110 Section 7.6.1 defines Connection as a comma-separated list of
case-insensitive connection options, and Section 5.3 lets that list be split
across several field lines. Comparing the whole field value against a single
option gets both wrong.
A client sending "Connection: keep-alive, close" was answered without a
Connection header and its socket was kept open, so the close it asked for was
never performed and never announced. An HTTP/1.0 client asking for keep-alive
only got it by spelling the option exactly "Keep-Alive"; the lowercase form
everyone actually sends closed the connection instead.
Route the five Connection checks through has_header_token(), which already
walks every field line and compares complete tokens. Expect is left alone:
matching "100-continue" as a token would make an unrecognized expectation
alongside it look acceptable, where Section 10.1.1 asks for 417.
It was defined as a static inline above the border line so that the split
build would not turn it into an exported symbol of the shared library, which
kept abidiff from reporting an added function. That put an internal helper's
location at the mercy of a CI check rather than of where it belongs: it reads
a header field the same way get_header_value() and get_combined_header_value()
do, and it is the closest sibling of the latter, both being about a list-valued
field spread over several field lines.
Define it as a plain inline beside them and forward-declare it with the
split() family it calls. Adding a symbol is a source and binary compatible
change, so let abidiff report it.
RFC 9110 Section 5.2 and 5.3 define the combined value of repeated field
lines as their values joined by commas in the order they were received.
Several call sites read only the first occurrence and then split that on
commas, so whatever the later field lines carried was silently dropped: an
acceptable media type or content coding, an ETag, a WebSocket subprotocol, a
declared trailer name, or an address a proxy appended as its own line rather
than by extending the one it received.
Add detail::get_combined_header_value() and use it for Accept,
Accept-Encoding, If-None-Match, Sec-WebSocket-Protocol, Trailer and
X-Forwarded-For. Empty field lines are skipped so the combined value never
starts with a bare comma, which parse_accept_header() rejects outright.
Also drop the now-dead manual trimming in parse_trailers() and replace the
istringstream-based subprotocol tokenizer with detail::split(); split()
already trims each token and skips empty ones.
* Match the Connection "Upgrade" token exactly in WebSocket handshakes
The server and the client both looked for "upgrade" as a substring of the
Connection field value, so "notupgrade", "upgrade-not" and "xupgrade" all
passed as the standalone token the handshake requires. RFC 6455 4.2.1 asks
for an ASCII case-insensitive token match, and a value split across several
Connection lines was missed entirely because only the first line was read.
Parse the field as the comma-separated token list it is, across every line,
and reuse the same helper for the server request check and the client
response check.
Reported by gb1dev.
* Tidy up the Connection token helper
Move has_header_token() out of the WebSocket-only detail block and next to
the other header field helpers, forward-declaring it beside split(). Use the
existing split_find(), which drops the manual found flag and stops at the
first matching token.
Drive the client-side test from an ordinary Server route answering 101 with
a bad Connection value, rather than the hand-rolled listening socket copied
from the test above it.
* Keep the Connection token helper out of the split build's ABI
The split build strips inline from everything below the border line, so a
helper defined there becomes an exported symbol of the shared library and
abidiff reports it as an added function. The tests also could not see
is_websocket_upgrade() or websocket_accept_key(), since neither is declared
in the part of the header that survives the split.
Define has_header_token() as a static inline above the border, next to the
split() declarations its two call sites already sit below, and declare the
two WebSocket helpers the way ws::impl::read_websocket_frame() already is.
The shared library's exported symbols are now identical to master's.
decode_uri was a byte-for-byte copy of decode_uri_component: it decoded every
%XX, including escapes of the reserved characters that encode_uri leaves
literal. So decode_uri was not the inverse of encode_uri and promoted an
escaped delimiter into a real one -- decode_uri("http://h/a%2Fb") returned
"http://h/a/b". Keep escapes of the reserved set encode_uri preserves, matching
JS decodeURI; non-reserved escapes still decode.
Server::make_matcher() built a std::regex for every pattern that did not
contain "/:", even though most route patterns are plain literals with no
regular expression syntax in them. Matching those went through
std::regex_match on every request, for every registered route the
dispatcher scanned before reaching the one that matches.
PathParamsMatcher already performs an exact literal comparison when it
captures no parameter, so no new matcher class is needed: a pattern with
no regex metacharacter can simply use it. Add an early return for the
zero parameter case in PathParamsMatcher::match(), and select the matcher
by also looking for the 14 ECMAScript metacharacters instead of only for
"/:". Path params keep taking precedence, so a pattern that mixes both,
such as "/users/:id/(.*)", is unaffected.
Measured with clang -O2 on macOS, scanning routes that all miss until the
last one: at 100 routes a scan drops from 10.3us to 0.46us, and end to end
throughput rises by about 24%. At 1000 routes throughput is roughly 3
times higher. Registering 5000 routes drops from about 3.0ms to about
0.9ms, since no std::regex is built for literal patterns.
This also keeps CPPHTTPLIB_REGEX_ROUTE_PATH_MAX_LENGTH confined to the
routes it is meant for. That limit rejects overlong paths before calling
std::regex_match, but until now every literal route was a RegexMatcher
too, so a literal route longer than the limit stopped matching even though
no regular expression was involved. Literal routes no longer go through
RegexMatcher, so only real regex routes are capped.
Patterns containing a metacharacter keep their current behavior, so
"/index.html" still matches "/indexXhtml" the way it always has. One
visible change: a literal route no longer populates Request::matches,
which is now a default constructed std::smatch. Path parameter routes
have always behaved that way, and Request::matches only carries useful
information for regex routes.
RegexMatcher::match() called std::regex_match() directly on the
attacker-controlled request path. For quantified patterns such as "(.*)",
std::regex_match's recursive backtracking implementation (most acute on
libstdc++) recurses roughly once per matched character, so a long enough
path can exhaust the calling thread's stack and crash the process. Verified
against real GNU libstdc++: under the default thread stack size, a path of
a couple thousand characters against a simple quantified route pattern
reliably crashed the process, well within the existing 8192-byte request
URI limit.
Add CPPHTTPLIB_REGEX_ROUTE_PATH_MAX_LENGTH (default 256) and reject paths
longer than it before ever calling std::regex_match, treating them as a
non-match instead. Confirmed the fix eliminates the crash under the same
libstdc++ build and default stack size that reproduced it.
For a non-SSL request with neither Content-Length nor Transfer-Encoding,
Server::read_content_core() fell back to reading raw wire bytes with
detail::read_content_without_length() directly, bypassing the decompressor
wrapper that the length-framed and chunked paths already use. As a result,
payload_max_length only bounded the compressed bytes read off the socket,
not the decompressed size a handler could produce from them.
Route this fallback through detail::read_content(..., decompress=true)
instead, the same helper already used below for the length-framed and
chunked cases, so the decompressed-size guard applies uniformly.
* Gracefully drain socket before close in Server::process_and_close_socket
Closing a connection while the receive queue still has unread data,
or while bytes are still in flight, can make the OS send an abortive
RST instead of a graceful FIN. On Windows this surfaces as
WSAECONNABORTED/WSAECONNRESET on the peer's read, which can make an
otherwise fully-written response look like a failed request -- a
likely contributor to the ServerTest.HTTP2Magic flakiness tracked in
#2533.
Add detail::close_socket_gracefully(), which half-closes the write
side, drains any queued/in-flight bytes (bounded to 100ms / 1MB),
then performs the final shutdown+close. Use it in
Server::process_and_close_socket.
Root cause and fix mechanism identified by @Hyukya in #2533.
* Rename close_socket_gracefully to drain_and_close_socket
'gracefully' already means something specific in this codebase: whether
to send a TLS close_notify before closing (shutdown_ssl's
shutdown_gracefully param, ClientImpl::disconnect(gracefully),
tls::shutdown(session, graceful)). Reusing the word for an unrelated
TCP-level drain-before-close made the new function read as part of that
TLS machinery when it isn't. Rename it to describe what it does instead,
matching the existing close_socket/shutdown_socket and
WebSocketClient::shutdown_and_close naming.
The stricter ws::Result error checks added in 6018c7f and 86d0210
exposed two backend-parity bugs in setup_client_tls_session(), shared
by SSLClient and WebSocketClient since their TLS setup was merged:
- enable_server_hostname_verification(false) had no effect on Mbed TLS
or wolfSSL for DNS hosts: mbedtls_ssl_set_hostname() and
wolfSSL_check_domain_name() bind SNI and handshake-time identity
checking together, so the identity check ran regardless of the
option, failing the handshake before the post-handshake
server_hostname_verification check was ever reached.
- On a genuine wrong-hostname failure, Mbed TLS reported the generic
Error::SSLServerVerification instead of
Error::SSLServerHostnameVerification, because
MBEDTLS_ERR_X509_CERT_VERIFY_FAILED was mapped without looking at
which verify flag actually caused it.
Fixes:
- set_sni() now takes a verify_hostname flag. wolfSSL skips
wolfSSL_check_domain_name() when it's false. Mbed TLS can't request
SNI without also arming the CN/SAN check, so it installs a verify
callback that masks the mismatch flag instead - a self-contained one
when the session has no user verify callback of its own, so it never
reads the process-wide set_verify_callback() slot another client may
have populated (this was caught by ASAN as a stack-use-after-scope:
VerifyCallbackTest.VerifyContextFields leaves a dangling lambda
there because MbedTlsSession never had a reason to consult it
before).
- map_mbedtls_error() now takes the handshake's verify flags and
reports HostnameMismatch when CN/SAN mismatch is the only one set,
matching the wolfSSL mapping and the post-handshake identity check.
- The duplicated verify-flags/error-mapping/backend_code logic in
connect() and connect_nonblocking() is factored into
fill_mbedtls_tls_error(); the duplicated flag-clearing in the two
verify callbacks is factored into mbedtls_clear_cn_mismatch(); both
use the existing hostname_mismatch_code() accessor instead of the
raw Mbed TLS macro.
Also tightens SSLClientTest.ServerHostnameVerificationError_Online to
assert the specific error code now that all three backends agree,
rather than accepting Mbed TLS's old fallback value.
Verified full non-online suite green on OpenSSL (791), Mbed TLS (737),
and wolfSSL (735), plus the split build, plus the Online
hostname-mismatch test against badssl.com on all three backends.
WebSocketClient's TLS setup already threaded
ClientTlsSessionOptions::server_hostname_verification through
setup_client_tls_session(), the same path SSLClient uses, but never
exposed a way to set it: create_stream() called setup_client_tls_session()
without an options argument, so the default (verification on) was the
only reachable value.
Add the public setter, mirroring ClientImpl/SSLClient/Client, and wire
it into create_stream()'s ClientTlsSessionOptions. Last open item from
issue #2531's WebSocketClient/SSLClient API alignment.
Issue #2531 asked for connect() to expose error detail the way
ClientImpl/SSLClient do via Result, instead of collapsing every failure
into a bare bool. The groundwork (detail::ClientTlsSessionError) was
already laid during the WebSocketClient/SSLClient dedup but left
unwired.
- Add httplib::ws::Result: explicit operator bool(), error(), and
flattened upgrade-response accessors (status(), headers(),
get_header_value(), has_header()); ssl_error()/ssl_backend_error() on
SSL builds.
- Add Error::WebSocketHandshake for upgrade-validation failures
(non-101 status, bad Sec-WebSocket-Accept, bad Upgrade/Connection
headers).
- Extract detail::parse_status_line from ClientImpl::read_response_line
and reuse it in read_websocket_upgrade_response, replacing the
previous "HTTP/1.1 101" substring match with a proper parse. Non-101
responses now surface their status and headers instead of being read
and discarded.
- Wire WebSocketClient::create_stream() to capture ClientTlsSessionError
so TLS failures (SSLServerVerification, SSLServerHostnameVerification,
...) reach the caller with backend error codes.
- Update tests and README-websocket.md accordingly.
This is a source-breaking change for callers that assign the result to
bool (e.g. bool ok = cli.connect();); if (cli.connect()) and gtest's
ASSERT_TRUE/EXPECT_FALSE(...) macros are unaffected since operator bool
still participates in contextual conversion.
T04 (mTLS) had grown a "WebSocketClient" subsection describing
wss:// client certificates, and c12/t02 were getting similar
WebSocketClient asides for timeouts and CA paths. The Cookbook's
own index already separates WebSocket into its own category
(W01-W04) from TLS/Security (T01-T05) and Client (C01-C19), so
burying WebSocketClient specifics inside those pages fought the
site's structure.
Move that content into two new recipes under the WebSocket
category instead:
- W05: wss:// TLS setup (set_ca_cert_path CA directory parity,
PemMemory client certificate)
- W06: WebSocketClient's three timeouts, including the recently
added chrono overloads
T04, T02, C12, and W01 now carry a single reference link to the
new pages instead of duplicated explanations, matching the site's
existing cross-link convention.
While rewriting T04's client-side section, noticed it documented
SSLClient's file-path constructor but not its PemMemory one, even
though the server-side section covered both forms for SSLServer.
Added the missing PemMemory example so both sides are symmetric.
WebSocketClient::set_connection_timeout (both the time_t and
chrono overloads) was missing from README-websocket.md's API
reference and the timeout example, even though set_read_timeout
and set_write_timeout were both listed.
README.md never documented the PemMemory in-memory constructor
that SSLServer and SSLClient both have, so mTLS setup only showed
the file-path form. Add a "Mutual TLS (mTLS)" section covering
both forms for server and client, and note that
ws::WebSocketClient's wss:// constructor takes the same PemMemory
struct.
Also note, next to Client::set_interface, that WebSocketClient has
the same method, matching the existing cross-reference for
set_hostname_addr_map right below it.
Adds ws::WebSocketClient::PemMemory and a constructor overload that
installs an in-memory client certificate on the TLS context, enabling
mutual TLS for wss:// connections. The certificate is silently ignored
for ws:// URLs, consistent with the existing TLS-only setters such as
set_ca_cert_path().
Part of the interface alignment discussed in #2531.
Temporary instrumentation to track the intermittent windows-without-SSL
failures reported in #2533. When the job fails on a push, it posts the
run URL, commit, and per-shard failed-test lines as a comment on the
issue, building up failure-pattern history automatically.
This should be removed once the root cause is found and fixed.
SSLClient::initialize_ssl kept its own copy of the session setup that
detail::setup_client_tls_session already implemented for WebSocketClient.
Extend the shared function with the pieces only SSLClient needed - a session
verifier, an independent hostname verification flag, the context mutex,
Windows Schannel verification and error details - and let initialize_ssl
build a ClientTlsSessionOptions and call it. All of them default, so
WebSocketClient's call site is unchanged.
This settles one difference between the two: WebSocketClient used to call
tls::set_hostname for named hosts, which on OpenSSL turns on verification
during the handshake, while SSLClient always set SNI only and verified
post-handshake. The shared function now does the latter for both, so
tls::set_hostname loses its last caller and goes away, as does the
write-only SSLClient::verify_result_.
Certificate verification with a host name rather than an IP literal was the
one combination the WebSocket tests never covered, and it is exactly the
path this normalizes. WebSocketSSLDnsHostTest fills that in; cert2 gains a
DNS:localhost SAN so a name can be verified against it.
ClientImpl::prepare_default_headers and WebSocketClient::prepare_default_headers
each built the Host value with the same AF_UNIX special case and appended the
same User-Agent. Move both into detail:: so there is one copy.
The Host helper returns only the value, because the two callers disagree on
where it goes: ClientImpl prepends it per RFC 9110 5.3, WebSocketClient appends.
The User-Agent helper takes the Request, since it has to consult and set a
header rather than compute a string, and it stays inside ClientImpl's
content_receiver branch so that path keeps sending no User-Agent.
WebSocketClient::set_ca_cert_path took a single path and create_stream()
hardcoded an empty directory when calling detail::load_client_ca_config, while
ClientImpl has always accepted (ca_cert_file_path, ca_cert_dir_path = ""). Give
WebSocketClient the same signature and store the directory, so both clients
configure CA loading identically. The one-argument form is unchanged for
callers.
Also note at both call sites why the "load the CA config once" guard differs:
SSLClient needs call_once because one client serves concurrent requests, and
WebSocketClient does not because connect() is not safe to call concurrently
anyway.
WebSocketClient only accepted timeouts as (time_t sec, time_t usec), while
ClientImpl has taken std::chrono::duration overloads for its read, write and
connection timeouts for a long time. Add the same three overloads, forwarding
through the existing detail::duration_to_sec_and_usec helper so the split
matches ClientImpl exactly.
The template bodies go above the first split.py BORDER, next to the class, so
that the .h/.cc split keeps them in the header where instantiation needs them.
test.cc says right above the PORT constant that it is only for the
legacy fixtures and that new standalone tests must use
bind_to_any_port() instead. The six tests added with the insertion-order
work all took the shared PORT anyway, which made them one more thing
contending for it.
Move them to bind_to_any_port() plus listen_after_bind(), the pattern
the note asks for and the rest of the standalone tests already use.
HeadersOrderTest.ReceivedFieldsKeepTheirOrder sends a raw request rather
than going through Client, so send_request() gains an optional port that
defaults to PORT and leaves its other 36 callers alone.
Checked by holding PORT open from another process while running the six:
they pass, where before the change listen(HOST, PORT) would have failed.
SocketStream.wait_writable_INET binds PORT + 1 directly, and it was the
only raw listener in the file that did not set SO_REUSEPORT/SO_REUSEADDR
first. PORT + 1 is shared with three SSL redirect tests and with
VulnerabilityTest.CRLFInjectionInHeaders, so once any of those has run,
the TIME_WAIT entries they leave make bind() fail here.
That failure did not surface where it happened. The bind runs on a
worker thread, where a failed ASSERT_EQ only returns from the lambda, so
the test carried on and failed later at ASSERT_NE(disconnected_svr_sock,
-1) with nothing pointing at the port.
Within one run of the suite the test comes before everything that uses
PORT + 1, so a clean run passes; it failed when a previous process had
left TIME_WAIT behind, which made it look intermittent. Running any of
those four tests first and then this one reproduces it every time: 4 of
4 before this change, 4 of 4 passing after, with up to eight TIME_WAIT
entries on the port.
Call default_socket_options(), which is what the library does for its
own listeners and what the other raw listeners in this file already do.
parse_query_text() and normalize_query_string() both walk a query string
and both open with the same eight lines to cut one "key=value" span at
its first '='. Give that a name and call it from both.
divide() puts everything before the first delimiter in the left half and
the rest in the right, so a span with no '=' lands entirely in key and
leaves val empty. Both callers rely on that: parse_query_text() records
a bare "flag" with an empty value, and normalize_query_string() emits it
back without an '='. The helper's comment says so, since that is the
part of divide()'s behaviour a reader has to know to follow either
caller.
Seven accessors each spelled the same two lines:
auto r = x.equal_range(key);
return static_cast<size_t>(std::distance(r.first, r.second));
The containers behind them all became detail::insertion_ordered_multimap
along the way, so count() is available and says what these functions
mean. The work is the same either way: equal_range() scans to the first
match and distance() then walks the restricted iterator to the end,
comparing keys at each step, which comes to one pass over the entries,
and count() is one pass too. This is a readability change, not a faster
one.
Covers get_header_value_count, Request and Response
get_trailer_value_count, Request::get_param_value_count,
MultipartFormData get_field_count and get_file_count, and
Result::get_request_header_value_count. The distance() call in
get_param_values() stays, since it sizes a reserve() and needs the range
anyway.
FormFields and FormFiles were std::multimaps, which sort by field name.
RFC 7578 5.2 says a form processor "SHOULD send back results in order"
and that "Intermediaries MUST NOT reorder the results", so a handler
walking req.form.fields saw the parts alphabetised rather than as they
were sent, and a body received for forwarding could not be reproduced.
Point both at the insertion-ordered container #2523 generalized, with
std::equal_to since field names are case-sensitive. Entries sharing a
name already kept their relative order under std::multimap; what is
recovered here is the order across different names.
Server::read_content() keeps a FormFields::iterator alive across the
content callbacks that fill the part it points at, which is the one
thing this container could have broken: it is vector-backed, so a later
emplace can reallocate and leave an older iterator dangling. The code is
safe because the iterator is reassigned by the same emplace that could
reallocate, and is only read while the flag set alongside it says so.
ContentSurvivesContainerGrowth pins that down with 64 parts, enough to
grow the vector through seven reallocations; reverting the reassignment
makes it abort under ASan rather than fail quietly.
Growth also never copies a part's payload: the parser inserts the entry
with an empty content and appends the body bytes afterwards, and both
mapped types are nothrow-move-constructible, so a reallocation steals
the string buffers rather than deep-copying them.
Params was a std::multimap, which sorts by parameter name. Parsing a
query string therefore threw away the order it arrived in, and building
one back out of Params handed the caller an alphabetised query rather
than the one they wrote. ClientImpl::send() takes that path whenever a
request carries Params without a query already in its path, so a caller
signing its query string could not reproduce the order it asked for.
normalize_query_string() exists in part to work around exactly this.
Generalize the container #2520 introduced for Headers into
detail::insertion_ordered_multimap<Mapped, KeyEqual> and alias both
types to it. Params passes std::equal_to, so parameter names stay
case-sensitive, while Headers keeps matching field names
case-insensitively. The name says insertion_ordered because in STL
vocabulary std::multimap is the ordered one, which is the reading this
change exists to correct.
This is a correctness change, not a performance one. Measured on the
real paths against the previous implementation, parse_query_text() over
eight parameters goes 931.5ns -> 935.1ns and params_to_query_str()
374.7ns -> 377.2ns; both are inside the run-to-run noise. The container
is a small share of that work, most of which is the string building in
decode_query_component().
Params picks up the same API changes Headers took in #2520: iterators
follow std::vector rules, value_type is std::pair<std::string,
std::string>, and insert(hint, value) is gone. Headers itself becomes an
alias rather than a class, so the names it appears in mangle differently
again; #2520 has not shipped in a release yet, so this costs nothing on
top of the break already there.
RFC 9110 5.3 lets a coding list be split across several Transfer-Encoding
lines, which combine, in the order the lines were received, into one
comma-separated list. RFC 9112 6.1 then frames the message as chunked
only when chunked is the final coding of that combined list.
is_chunked_transfer_encoding() could not apply that rule while Headers
was an unordered_multimap, since the order of the lines was not
recoverable, so it fell back to reporting any message naming chunked on
any line as chunked. Headers now preserves the order the lines were
received in, so read the final coding directly: the last token of the
last line.
The fallback erred toward reporting chunked because mis-reading a
chunked message as unframed leaves its body in the socket, where a
keep-alive connection parses it as a smuggled request. That direction is
no longer needed. A request whose combined list ends in something other
than chunked is now reported as not chunked, and process_request()
answers 400 and closes rather than letting it reach the "no body" path,
which is what it already did for the single-line "chunked, gzip" form.
So `Transfer-Encoding: chunked` followed by `Transfer-Encoding: gzip` is
rejected instead of being read as chunked, and `gzip` followed by
`chunked` is still accepted.
A trailing line carrying no coding at all now leaves the combined list
ending in nothing rather than inheriting the coding from the line
before it.
The fuzzers build httplib.h and the targets under test/fuzzing, so a pull
request that touches neither has nothing for CIFuzz to exercise. Fuzzing
is by far the longest job in CI: 600 seconds of fuzzing on top of
building the OSS-Fuzz image, which came to 12m20s on a recent run while
every other job finished within 5m7s.
Filter the trigger by path rather than shortening fuzz-seconds. OSS-Fuzz
recommends 600 seconds as a minimum, and the budget is divided among all
of the project's fuzz targets, so with five targets a shorter run would
leave each one well under two minutes. Skipping the job for
documentation-only changes cuts the wait without giving up any fuzzing
on the pull requests that do reach the parsers.
RFC 9110 5.3 makes the order of header fields sharing a field name
significant, but Headers was a std::unordered_multimap, which gives no
ordering guarantee for equivalent keys. libstdc++ hands duplicates back
in reverse insertion order while libc++ uses insertion order, so
get_header_value() returned a different field depending on the platform,
and code that picks a value out of an accidentally or maliciously
duplicated field name had no way to say which one it wanted.
Replace Headers with a small container that keeps the fields in the
order they were received or set. Storage is a flat vector and lookup is
a linear scan, which beats hashing for the at most
CPPHTTPLIB_HEADER_MAX_COUNT fields a message carries. begin()/end() walk
every field, while find() and equal_range() hand back the same iterator
type restricted to one field name; equality compares only the position,
so a restricted iterator still compares equal to end(). Erasing an
equal_range() therefore removes only the fields with that name and
leaves interleaved fields alone.
std::multimap was the smaller change but sorts by field name, which
would stop control data such as Host from leading the message. Instead
Host is now prepended via emplace_front() so it keeps its place at the
front of a request.
Two side effects worth noting: incrementing past the last field of a
name now saturates at end(), so an out-of-range id passed to
get_header_value() returns the default instead of running off the
container as it did before; and iterators follow std::vector rules, so
they are invalidated by insertion.
Fixes#2509
* Skip the redundant readability poll on the first read of a request
process_server_socket_core() calls keep_alive(), which polls the socket
and only invokes the callback once it reports readable. The stream is
then constructed and its first read polls the very same socket again
before calling recv, asking the kernel a question that was answered
microseconds earlier.
Profiling a request shows how little else is going on: of the samples
taken while the worker was off the keep-alive wait, 85% sat in syscalls
(recv, send, and the two polls) and only 15% in header parsing, routing
and response serialization. Removing one of five syscalls per request is
worth more than anything reachable inside that 15%.
Let the caller hand the stream what it already knows. The hint is
consumed by the first read, so body reads and every later request keep
polling as before, and set_read_timeout() during a WebSocket upgrade is
unaffected. Nothing is skipped when a read finds the buffer empty on its
own. The accepted socket also carries SO_RCVTIMEO from listen_internal(),
so even a wrong hint could not block forever.
Measured with wrk -t2 -c8 over three interleaved runs, server CPU per
request drops from 33.4/32.5/29.7us to 23.6/22.9/23.9us, and throughput
rises 10-15%. The TLS path gets the same treatment - it shares
process_server_socket_core(), and SSLSocketStream::read() polls after
tls::pending() comes up empty - for a smaller 44.9->40.2us, crypto being
the larger cost there.
Removing the second poll, in write(), looks worth another 15% but is left
alone: it would rely on SO_SNDTIMEO being set on every socket a
SocketStream is built over, and the TLS write path retries on WantWrite
in a way that turns a send timeout into a much longer stall.
* Scan buffered bytes for line ends instead of reading one at a time
stream_line_reader::getline() pulls the request line and every header
through strm_.read(&byte, 1). A 300-byte header block therefore costs
300 virtual calls, each with its own bounds check and one-byte copy.
None of them are syscalls - SocketStream has already pulled up to 4KB
off the socket - so this is pure CPU spent one character at a time.
On a Linux CI runner read_headers() takes 2.44us for seven headers, of
which 1.53us is this loop. That is 63% of header parsing, and header
parsing is a bigger share of the total on Linux than the profiling on
macOS suggested: 3.8us of HTTP processing against 16us of server CPU
per request, versus roughly 3us against 30us on macOS. Cheap syscalls
leave parsing a larger slice of what remains.
Let a stream offer what it has already buffered, and scan that for the
terminator in one pass. Streams that do no buffering of their own report
none and keep the existing byte loop, so Stream subclasses outside the
library are unaffected, as is the TLS path - SSLSocketStream has no
buffer of its own to expose.
A bare LF still does not end a line in the default configuration, so the
scan looks for CRLF and carries the CR across a chunk boundary. Both
append overloads now share one path: the per-character one used to
decide where to write from fixed_buffer_used_size_ alone, which after a
bulk append had spilled into the growable buffer would send the byte to
a fixed buffer that ptr() and size() no longer read. That dropped a byte
from over-long request lines - caught by ServerTest.TooLongRequest and
AlmostTooLongRequest.
Explain why `addr_` is reset before `close()`: `munmap()` must not be
called with the sentinel. The test also checks `size()`, since the hazard
is a stale size paired with a sentinel `data()`.
The added test only exercised a 403 with a single range, leaving three
paths that were equally broken untested: a suffix range, whose first_pos
is still -1 when the bounds asserts are compiled out under NDEBUG; two
ranges, where apply_ranges() makes no boundary for a non-206 response and
the multipart branch wrote an empty one; and a 2xx that is not a 206,
which was validated but still announced the full content length.
Fold the copied route back into /streamed-with-range with a status query
parameter, following the ?error idiom already used there.
Since 8bba34e, prepare_content_receiver() rejected every non-empty
Content-Encoding that create_decompressor() could not handle, conflating an
unrecognized content coding with a known one whose support was not compiled
in. A response labeled `Content-Encoding: UTF-8` -- a header some servers
misuse to advertise a character set -- therefore failed with 415, which the
client surfaced as the generic Error::Read. Before that commit the raw body
came back untouched.
Restore that behavior: reject only a coding cpp-httplib recognizes but was
not built with, and leave an unrecognized one (including "identity") alone.
Match codings case-insensitively while here, as RFC 9110 8.4.1 requires;
otherwise `Content-Encoding: GZIP` would look unrecognized and hand back a
still-compressed body.
open_stream() had the mirror-image problem. It silently passed the payload
through whenever create_decompressor() returned null, so a gzip response on
a build without zlib reached the caller still compressed, and it never
checked is_valid() -- gzip_decompressor::decompress() only asserts, so a
failed inflateInit2() was undefined behavior in release builds. It now
applies the same policy as the buffered path.
Add Error::UnsupportedContentEncoding so callers can tell this apart from a
read failure, and report an unusable decompressor as Error::Compression. The
status read_content() writes was previously discarded into an uninitialized
dummy_status.
* Close the listening socket in stop() even when not serving
Server::stop() released svr_sock_ only under `if (is_running_)`, and
is_running_ is set inside listen_internal(). A server that bound with
bind_to_port() / bind_to_any_port() and never reached listen_after_bind()
therefore kept its listening descriptor for the life of the process:
~Server() is defaulted and svr_sock_ is a bare atomic<socket_t>, so nothing
else closes it. The port stayed held too.
Close whenever the socket is open instead. The exchange already made this
safe against a concurrent accept loop, so dropping the is_running_ gate also
removes a TOCTOU between the check and the exchange.
* Fail listen_after_bind() when stop() already closed the socket
With stop() now releasing a bound-but-not-serving socket, a stop() that
lands between bind and listen used to slip through listen_internal():
the accept loop saw INVALID_SOCKET, never iterated, and returned success
without ever serving, pulsing is_running_ just long enough that a
wait_until_ready() caller could miss it and spin forever. Return false
instead and mark the server decommissioned the way any failed listen
does, so waiters wake up.
Also drop the assert() in stop(). It read is_running_ and svr_sock_
separately, which is exactly the race the exchange removes: a second
stop() while the accept loop is still unwinding sees is_running_ true
and svr_sock_ already INVALID_SOCKET, aborting debug builds.
The regression test checks the released port with a raw connect()
instead of a Client request: against the old code the connection is
accepted into the backlog and never answered, which would hang the test
rather than fail it.
Follow-up to 49b921b.
Move the duplicated addr_map lookup into detail::apply_addr_map, shared by
ClientImpl::create_client_socket and WebSocketClient::connect.
Add a WebSocketClient test for a hostname mapped value, so that path has
the same coverage as the Client one. Guard its teardown with scope_exit:
a failing ASSERT_TRUE returns from the test body, and destroying a still
joinable std::thread calls std::terminate, taking the whole binary down.
Document set_hostname_addr_map in README. It had no entry at all.
addr_map_ only accepted IP literals as mapped values; a non-IP value was
passed as the `ip` argument and rejected by getaddrinfo's AI_NUMERICHOST
path. The lookup in ClientImpl::create_client_socket and
WebSocketClient::connect now checks the mapped value with
detail::is_ip_address: IP literals keep the existing AI_NUMERICHOST path,
while hostnames are passed as the connect host so they get resolved.
host_ is never touched, so it keeps supplying the Host header and SNI in
both cases.
is_ip_address() moved into the non-SSL detail block so that both client
code paths can use it without CPPHTTPLIB_SSL_ENABLED.
This also fixes the documented Unix domain socket client example, where
the mapped value is a socket path: it previously took the AI_NUMERICHOST
path and never reached the AF_UNIX branch in create_socket.
Follow-up to #2514. The rebuilt handshake wrote the request line straight
to the socket, so a header rejected by check_and_write_headers left a
truncated "GET /ws HTTP/1.1" sitting in the peer's buffer before the
connection was torn down, and every header cost its own small write.
Build the request into a BufferStream and flush it in one go, matching
ClientImpl::write_request. The new test drives a raw listener and asserts
the peer sees a clean EOF with zero bytes; without this change it observes
18.
Also fold WebSocketTest.HostHeaderInHandshake into
WebSocketTest.DefaultHeadersInHandshake, which covers the same Host
assertion through the capture helper #2514 introduced.
Replace the hand-built upgrade request string in
perform_websocket_handshake with the same Request,
write_request_line, and check_and_write_headers path used by
ClientImpl::open_stream. Add WebSocketClient::prepare_default_headers
to inject Host and User-Agent defaults; protocol-mandatory headers
(Upgrade, Connection, Sec-WebSocket-Key/Version) are always
overwritten.
The first run of this workflow exposed three problems.
Crow's amalgamated header includes <asio.hpp>, which no runner provides,
so the build died immediately. It compiled locally only because CPATH
happened to point at Homebrew's include directory. Install asio
explicitly, and on macOS pass its include path through CROW_CXXFLAGS.
The macos runner image has no Go, so bombardier could not be installed.
Add actions/setup-go, which also pins a known toolchain on Linux.
Worst of all, the ubuntu job reported success. `make ... | tee` returns
tee's status, so the failed build was invisible. Enable pipefail. That
alone is not enough: every recipe in benchmark/Makefile ends in `kill`,
so make still exits 0 when bombardier itself fails to run. Assert that
the expected number of "Reqs/sec" lines came out.
benchmark/Makefile has always been local-only, so the numbers it produces
were never recorded anywhere. Wire it up to a manual workflow so a run can
be kicked off and its output kept in the job summary.
This does not gate anything: it reports absolute throughput for the
current ref, with Crow v1.3.1 alongside for reference. Absolute req/s is
only comparable against other runs on the same runner type, which is why
the ref, runner and load parameters are recorded next to the numbers.
Use benchmark-ab instead when the question is whether a specific change
made things faster; comparing absolute numbers across runs cannot answer
that.
Linux and macOS only. Windows needs benchmark/Makefile rewritten first,
since it relies on nc, & and kill.
The existing test_benchmark asserts a single request completes within
5ms, which catches gross connection-setup regressions but cannot see
throughput changes: the effects we care about are tens of microseconds
per request, a hundredth of that resolution. There was no way to answer
"does this patch make the server faster" other than measuring by hand.
Absolute req/s is not usable for that. Running the same binary five
times on an idle 8-core machine gave 53.9k to 74.6k req/s, and shared CI
runners are noisier still, so a number printed per push says nothing.
Build both refs and measure them alternately in one session, flipping
the order each round to cancel ordering bias, then report only the ratio
of the medians. Whether that ratio means anything is decided by an exact
permutation test rather than by comparing the change against the min/max
spread - a single slow round is enough to make a spread check give up,
while the rank test rides it out.
Validated against a patch that removes a redundant poll() per request:
individual measurements ranged 41.7k-93.9k req/s, yet nine rounds
resolved a 1.244x speedup at p = 0.019. The same data truncated to five
rounds was inconclusive, so the workflow defaults to nine.
Manual dispatch only, Linux and non-SSL for now, and it never fails the
build - this is a measurement, not a test.
The accept queue was limited to 5 pending connections, which overflows
easily under connection churn or a burst of simultaneous connects (LB
health checks, thundering herd on restart). On overflow the kernel
silently drops the ACK rather than failing fast, so clients stall on
SYN/ACK retransmission backoff.
Benchmark (bombardier -c 10 -d 10s, 3 trials, interleaved with the old
binary) shows max latency dropping from 48-89ms to 5.8-11.2ms, while
p99 is unchanged - the fix affects only the extreme tail, as expected.
The upload example wrote each uploaded file using the filename supplied
verbatim in the multipart Content-Disposition header. A client could set
that filename to an absolute path or one containing "../" components and
cause the server to create or overwrite files outside the working
directory.
Reduce each client-supplied filename to its base name and reject the
request with 400 Bad Request if the result is empty, ".", "..", or
still contains a path separator (including colon for Windows drive
letters).
ClientImpl::open_stream() passed the caller-supplied path straight to the
request line, so it ignored path_encode_ entirely and always behaved as if
set_path_encode(false) had been called. The same path therefore produced
different bytes on the wire depending on which API was used:
Get() "/a b" -> GET /a%20b HTTP/1.1
open_stream() "/a b" -> GET /a b HTTP/1.1
A space is the request-target delimiter, so the streaming form is not merely
inconsistent: an RFC 9112 conformant server reads the target as "/a" and the
version as "b". Non-ASCII bytes and '+' diverged the same way, the latter
changing the value a server that decodes '+' as space sees.
Extract the path/query splitting and encoding out of ClientImpl::write_request
into detail::encode_request_target() and call it from both paths, so the
encoding rule lives in one place. open_stream() appends Params before
encoding, matching ClientImpl::Get(path, params), which builds its target the
same way.
Note a behavior change: with path encoding enabled, CR/LF in the target is now
percent-encoded and sent rather than rejected with Error::Write, matching
Get(). This is not a weakening of the CR/LF guard in write_request_line() --
that check is independent of path_encode_ and still backstops
set_path_encode(false), where encode_path() does nothing.
Mbed TLS has no SSL_peek() equivalent, so is_peer_closed() (called after
every SSL request write) probed liveness with a real 1-byte
mbedtls_ssl_read() and discarded whatever it read. If the response had
already arrived by the time the probe ran — plausible under CI load or
plain OS scheduling — the probe silently ate the first byte of the
status line, corrupting the response and surfacing as a fast
"Failed to read connection" failure.
This was the root cause of the long-standing MbedTLS-only CI flakiness
(ServerTest cases failing intermittently on Ubuntu and macOS), previously
worked around by reducing gtest shard parallelism. Fix: push the probed
byte back into MbedTlsSession and have tls::read()/pending() account for
it, so no data is lost.
Also fix a second, unrelated flake: ProxyTunnelTest.
OriginReturning407InsideTunnelDoesNotLeakProxyDigest used "localhost" for
its client while the test's proxy harness only listens on 127.0.0.1;
under dual-stack resolution this could race with another test's server
on ::1 using the same ephemeral port. Pin the test to 127.0.0.1.
With the root cause fixed, restore the mbedTLS CI jobs (ubuntu,
ubuntu-26.04, macOS) to the default shard count instead of the
previously reduced SHARDS=1/2 mitigation.
Only 8 of 423 call sites logged the actual error on failure. The
MbedTLS-backend flaky CI failure (connection-level ASSERT_TRUE(res),
~20-30ms) keeps landing on assertions without this diagnostic, so the
real error code has never been captured. Broadens the existing
GetWithRange-only logging (a4d7066) to every plain ASSERT_TRUE(res);
site, no behavior change.
The is_field_name(name) && is_field_value(value) predicate was repeated
across five output paths (set_header, write_headers, write_content_chunked
trailer, perform_websocket_handshake, check_and_write_headers). Introduce
fields::is_field_valid(name, value) and route all five through it so the
CR/LF-injection guard has a single definition. No behavior change.
Server::process_request only checked that trusted_proxies_ was
non-empty before deriving req.remote_addr from the X-Forwarded-For
header. It never verified that the actual TCP peer (remote_addr) was
itself one of the trusted proxies, so any client connecting directly
to the server could spoof remote_addr simply by sending an arbitrary
X-Forwarded-For header.
Now X-Forwarded-For is only honored when the connecting peer address
matches an entry in trusted_proxies_.
* Add Mbed TLS 4.x support (PSA Crypto) for macOS
Auto-detect Mbed TLS 4.x via MBEDTLS_VERSION_MAJOR and adapt the backend:
- Include psa/crypto.h and drop the headers removed in 4.x (ctr_drbg,
entropy, md5, sha*), gated behind the version macro.
- Compute MD5/SHA-256/SHA-512 via PSA (psa_hash_compute) and initialize
PSA Crypto once with std::call_once.
- Drop the explicit entropy/CTR-DRBG RNG (PSA provides the TLS RNG) and
skip the RNG-callback overloads of pk_parse_key/pk_check_pair on 4.x.
- Retry on a TLS 1.3 NewSessionTicket (the 4.x default) in connect, read,
write and is_peer_closed via a single mbedtls_is_session_ticket() helper,
so online HTTPS works, including large redirected downloads where the
ticket arrives mid-write.
Note V4 implies V3, so 3.x-only paths now check V3 && !V4.
Build systems (macOS): the CMake config and pkg-config shipped by Homebrew
resolve 4.x transitively, so CMakeLists.txt and meson.build need no change
for linking; the Makefile links libtfpsacrypto when present, else
libmbedcrypto.
Tests: generate the encrypted client key as both PBES2-AES (3.6+/4.x,
OpenSSL, wolfSSL) and PBES1-3DES (Mbed TLS 2.28) and pick by version, since
4.x dropped DES and 2.28 lacks PBES2. Also generate the IP-host certs in
test/meson.build to match gen-certs.sh and CMakeLists.txt.
* CI: test Mbed TLS 4.x on macOS, 3.x on Ubuntu 26.04
Homebrew's default mbedtls is now 4.x, so switch the macOS build and CI job
to it (drop the mbedtls@3 pin). That leaves 3.x (Ubuntu 24.04 apt ships 2.28,
macOS now 4.x) uncovered, so add an ubuntu-26.04 job whose apt provides Mbed
TLS 3.6. Net coverage: 2.28 (ubuntu-latest), 3.6 (ubuntu-26.04), 4.2 (macOS).
ubuntu-26.04 is a public-preview runner image; fold it into the main ubuntu
matrix once ubuntu-latest moves to 26.04.
* Document Mbed TLS 4.x support and libtfpsacrypto rename
Update README.md and the tour's TLS setup pages (en/ja) to note that
Mbed TLS 4.x is now auto-detected and that it renames libmbedcrypto
to libtfpsacrypto.
The post-response drain exists to keep unread framed body bytes from being parsed as a subsequent request on a persistent connection. Once response generation has committed the connection to close, there can be no subsequent request, so draining no longer provides that protection.
Continuing to drain is especially harmful when a ContentReader aborts an unterminated chunked upload: the server can wait indefinitely for the terminal chunk even after sending Connection: close. This delays the transport close that tells an in-flight uploader to stop and leaves a worker occupied consuming discarded data.
Use the finalized response Connection header as the single source of truth for whether to skip the drain. write_response_core already sets this header for keep-alive exhaustion, request-directed closure, handler-directed closure, and error responses. Marking connection_closed then terminates the keep-alive loop and closes the socket.
Add a raw-socket regression test whose ContentReader rejects the first chunk of an unterminated upload. The test verifies that the 409 response announces Connection: close and that the peer observes EOF rather than timing out while the server drains.
The multipart/form-data parser bounded the length of each individual
header line via CPPHTTPLIB_HEADER_MAX_LENGTH but did not cap how many
header lines a single part could contain. Within the overall payload
limit, a client could pack a large number of small header lines into
one part, each of which is parsed twice, inflating CPU usage.
Add a per-part header-line counter that resets when a new entry begins
and abort parsing once it reaches CPPHTTPLIB_HEADER_MAX_COUNT, mirroring
the limit already enforced by read_headers() for request headers.
The from_chars template is instantiated with unsigned types (e.g.
uint64_t for Content-Length), where unary minus on `result` triggers
MSVC warning C4146, failing the 32-bit build under warnings-as-errors.
Use `T(0) - result`, which yields the identical two's-complement value
without the warning.
Replace the strtoull + errno + cast-back dance in get_header_value_u64
with the existing hand-written detail::from_chars, which reports
result_out_of_range at size_t width. This detects the 32-bit truncation
case directly (instead of via a separate cast-back comparison), drops
the reliance on the global errno, and keeps the parsing locale-
independent and consistent with the rest of the codebase.
* reject crlf in request target in write_request_line
* declare write_request_line in test.cc for split builds
split.py strips `inline` and moves the definition into httplib.cc, so
detail::write_request_line is not visible from the split httplib.h and
test_split failed to compile. Re-declare it in test.cc, matching what the
base64_encode and getaddrinfo_with_timeout tests already do.
* Fail the request when write_request_line rejects the target
The CR/LF guard in write_request_line returns -1, but ClientImpl::write_request
ignored that return value. A rejected target therefore produced a request-line-
less request (headers only) that the client silently reported as a successful
send. This is the primary path reachable via a decoded redirect Location under
set_path_encode(false), and it also carries the CONNECT target.
Check the return value like ClientImpl::open_stream already does and fail with
Error::Write. Add an end-to-end test asserting the client refuses a CR/LF target
instead of putting it on the wire.
---------
Co-authored-by: Sayed Kaif <metsw24@gmail.com>
is_chunked_transfer_encoding compared the whole Transfer-Encoding field
value against "chunked", so a message whose final coding is chunked but
which names another coding first ("gzip, chunked", valid under RFC 9112
6.1) was read as unframed. On a keep-alive server the body was then left
in the socket and parsed as a smuggled request; open_stream repeated the
check case-sensitively with a raw ==, desyncing the client stream the
same way.
Rework the helper to match the last coding token case-insensitively and
route open_stream through it so both paths agree. The codings may also be
split across multiple Transfer-Encoding lines (RFC 9110 5.3); since
Headers is an unordered_multimap whose duplicate-key iteration order is
not portable, the final coding of a multi-line field cannot be
determined reliably, so treat any such message that names chunked as
chunked (fail safe: a mis-parse only closes the connection, whereas the
opposite error enables smuggling). A unit test covers the helper,
including order-independent multi-line cases.
Based on #2487 by @metsw24-max.
Cookie and Cookie2 headers were forwarded to the new host when following
a cross-origin redirect, even though Host, Proxy-Authorization, and
Authorization were already stripped. Add them to the removal list so
session cookies are not leaked to a different origin.
shutdown_and_close() freed the TLS session before ws_->close() sent the
WebSocket close frame. The WebSocket's SSLSocketStream keeps a raw pointer
to that session, so sending the close frame then read/wrote a freed SSL
object. Reorder so ws_->close()/ws_.reset() run while the session is still
alive, then free the session (GHSA-w7p7-f35j-mw7q).
Server::apply_ranges computed the correct Content-Length header for
body-based responses but never updated content_length_, so the
Logger callback always saw 0. Set content_length_ to the final body
size (post-range/post-compression) alongside the header.
SSLClient::~SSLClient() freed the TLS context before shutting down
the SSL session. mbedTLS sessions hold a raw pointer into the
context's mbedtls_ssl_config, so a live keep-alive session's
close_notify would read freed memory. Shut down the session first,
then free the context.
Add a regression test that destructs an SSLClient while a keep-alive
mbedTLS session is still open.
The quick preview used a nonexistent httplib::ws::Message type with
.is_text()/.data. The actual API, as shown in README-websocket.md,
uses a plain std::string with ws.read(msg).
Trailer field names and values written by write_content_chunked()'s
done_with_trailer lambda were never validated, unlike every other
header output path (set_header, WebSocket handshake, client request
headers). An application reflecting untrusted input into a trailer
via DataSink::done_with_trailer() could inject CR/LF sequences and
achieve HTTP response splitting.
Skip trailer fields with invalid names or values, matching the
silent-skip behavior of set_header().
Cookbook body links referenced sibling pages with a bare slug
(e.g. `c14-keep-alive`). Under the pretty-URL layout each page lives
in its own directory, so these resolve against the page's own
directory and 404. Prefix them with `../` to match the convention
already used in the tour and llm-app sections.
Verified clean with `docs-gen check`.
std::isalnum and std::isdigit consult the global C locale, so a byte
like 0xC5 can classify as alphanumeric once an embedder calls
setlocale() (observed on macOS). HTTP grammars are defined over ASCII,
so raw bytes must be classified without regard to the locale.
Add detail::is_ascii_digit/is_ascii_alpha/is_ascii_alnum and use them
at every classification site: multipart boundary validation, token
checks, URI encoding, range header parsing, is_numeric, is_hex, and
IPv4 host detection. Also unify the hand-written digit range checks in
from_chars, URL parsing, and parse_ipv4 onto the same helpers.
With these in place nothing uses <cctype> anymore, so the include is
dropped.
On glibc >= 2.34, getaddrinfo_a is no longer provided by a standalone
libanl shared library — it is built directly into libc. Newer
architectures (e.g. loongarch64, riscv64) have never shipped a separate
libanl, causing the meson build to fail with:
C++ shared or static library 'anl' not found
Signed-off-by: Pluto Yang <yangyj.ee@gmail.com>
The WebSocket upgrade request always appended ":port" to the Host
header, violating RFC 6455 Section 4.1 which says the port should be
included only when it is not the default (80 for ws, 443 for wss).
Some CDNs alter routing when the Host header carries an explicit
default port.
Build the Host header with detail::make_host_and_port_string, which
also brackets IPv6 literal hosts correctly.
Same header-injection vector as the name/filename fix: item.content_type
was concatenated into the part's Content-Type header unescaped, so
embedded CR/LF could inject arbitrary part headers.
Escape CR -> %0D and LF -> %0A via escape_multipart_field with a new
escape_quote = false mode. '"' is left intact since it is legal in
Content-Type values (e.g. quoted charset parameters) and appears outside
a quoted-string context here.
Building a multipart/form-data body outside Client::Post/Put/Patch
(e.g. to feed a custom content provider or compose with other body
sources) previously required calling several detail:: functions
directly, which is not a stable interface.
Introduce MultipartFormDataWriter, a small public class that owns the
boundary and delegates to the existing detail:: serializers: whole-body
serialization with known content length, and per-part framing
(item_begin/item_end/finish) for streaming. Also expose
is_valid_multipart_boundary() so callers using an explicit boundary can
validate it without exceptions.
detail::serialize_multipart_formdata_item_begin() concatenated item.name
and item.filename into the Content-Disposition quoted-string without any
escaping. A '"' in either value silently terminated the quoted-string
early, and embedded CR/LF allowed injecting arbitrary part headers or
forging part boundaries when a filename comes from external input.
Escape both values following the WHATWG HTML standard ('"' -> %22,
CR -> %0D, LF -> %0A), which matches what browsers send in
multipart/form-data bodies. Behavior only changes for inputs that
previously produced malformed HTTP.
set_path_encode(false) only suppressed encoding of the path part. The
query part was always run through normalize_query_string(), which
decodes then re-encodes each key/value pair regardless of the flag.
That round-trip is lossy for pre-encoded payloads: re-encoding emits
sub-delimiters literally (%2C->",", %24->"$", %3B->";", ...) and turns
%20 into "+", which a strict RFC 3986 server decodes back as "+"
(0x2B) rather than a space (0x20), corrupting binary query data.
Honor path_encode_ for the query as well: when disabled, append the
caller-supplied query verbatim. Add a regression test asserting on the
raw request target, since the server decodes "+" as space and would
otherwise mask the difference.
The cert_ip_cn.pem and cert_ipv6.pem certificates added in ba390f2 were
only generated by gen-certs.sh, which the Makefile-based Linux/macOS CI
uses. The Windows CI builds with CMake, whose own certificate-generation
block was not updated, so cert_ipv6.pem was missing there and
SSLClientServerTest.TlsVerifyHostnameIpv6San failed on is_valid().
Mirror the two openssl commands into test/CMakeLists.txt to keep both
certificate-generation paths in sync.
An IP-literal host must only be authenticated via a matching iPAddress SAN,
never via the certificate's Common Name (RFC 9110), as the OpenSSL backend
already does through X509_check_ip. The Mbed TLS and wolfSSL backends instead
fell back to the CN when no IP SAN matched, and recognized IPv4 only.
This is a more complete solution for #2476, which gated the CN fallback for
IPv4 hosts only; here the same gap is closed for IPv6 as well, and IPv6
iPAddress SANs are actually matched.
- Add impl::parse_ip_address() to parse IPv4/IPv6 literals into raw bytes
- Match IPv6 (16-byte) iPAddress SANs, not just IPv4
- Skip the CN fallback for IP-literal hosts (both IPv4 and IPv6)
- Remove the unused SSLClient::verify_host* dead code
- Add regression tests and test certificates for the IP-host cases
OpenSSL 4.0 deprecates X509_STORE_get0_objects() and
X509_NAME_get_text_by_NID(), producing warnings on the OpenSSL
backend (surfaced by Debian bug #1138434, which also reported a
const-conversion build error already fixed in v0.43.1).
- Fetch CA store objects via X509_STORE_get1_objects() (thread-safe,
OpenSSL 3.3+) in get_ca_certs() and get_ca_names(), releasing the
snapshot with a scope_exit guard; keep using get0 on older OpenSSL,
BoringSSL, and LibreSSL
- Extract the subject CN in get_cert_subject_cn() with
X509_NAME_get_index_by_NID()/X509_NAME_get_entry() instead of the
deprecated X509_NAME_get_text_by_NID()
- Delegate SSLClient::verify_host_with_common_name() to
tls::get_cert_subject_cn() to drop the remaining deprecated call
Verified against OpenSSL 4.0.1, 3.6.2, and 3.0: builds warning-free
and passes the full test suite.
For connections to IP-literal hosts with server certificate
verification enabled, the Mbed TLS and wolfSSL backends downgraded the
verification mode before the handshake because no hostname could be
bound for in-handshake checks:
- SSLClient skipped certificate chain validation entirely; only the
post-handshake identity check (IP SAN match) remained, so any
untrusted certificate carrying a matching IP SAN was accepted
- The WebSocket client skipped verification altogether on Mbed TLS,
accepting any certificate
Keep the verification mode enabled for IP hosts and verify the
certificate identity post-handshake via tls::verify_hostname(), which
supports IP SANs on all backends. The WebSocket path now performs the
same post-handshake identity check as SSLClient. On Mbed TLS, sessions
explicitly opt out of in-handshake hostname verification (mandatory
since Mbed TLS 3.6.4) and the post-handshake check covers identity
instead; DNS hosts still bind the hostname during the handshake. Also
stop sending SNI for IP hosts on Mbed TLS and wolfSSL (RFC 6066).
Add an explicit opt-in for loading system CA certs alongside a custom
CA, addressing the request in #2471. The default behavior is unchanged:
a custom CA remains exclusive.
- Add Client/SSLClient/WebSocketClient::enable_system_ca(bool); the
policy carries over to redirect clients
- Extract the CA loading policy into detail::load_client_ca_config()
shared by SSLClient and WebSocketClient, making WebSocketClient
exclusive by default (it previously always merged system certs)
- Make the WebSocketClient TLS context live as long as the client,
fixing a use-after-free when reconnecting after set_ca_cert_store()
- Free the source store in the Mbed TLS and wolfSSL set_ca_store()
backends, honoring the take-ownership contract (memory leak)
- Verify IP hosts against IP SANs in the OpenSSL set_hostname() backend
so WebSocket connections to IP hosts can use full verification
Since the TLS abstraction layer was introduced, SSLClient::set_ca_cert_store()
handed the store to the TLS context without leaving any trace on the client.
As a result:
- load_certs() merged system CA certs into the user-provided store,
silently broadening the trust set (a custom store used to suppress
system CA loading)
- Client::load_ca_cert_store() went through the native store path,
bypassing the PEM retention used for redirect transfer, so CA certs
were not carried over to clients created for HTTPS redirects
- The Windows Schannel verification skip for custom CA certs did not
trigger
Track custom store assignment with a flag checked by load_certs() and
the Schannel path, and route Client::load_ca_cert_store() through the
PEM-based SSLClient path so the CA data survives redirects.
Previously, for regular handlers the request body was read in routing()
before the route was matched, so the pre-request handler always saw an
already-read body. The ContentReader path, in contrast, ran the
pre-request handler before the body was read. This inconsistency made
it impossible to reject a request (e.g. failed per-route authentication
via req.matched_route) without buffering a potentially large body.
Move the read_content() call into dispatch_request(), after route
matching and the pre-request handler, so both paths behave the same:
route matching -> pre_request_handler -> body read -> handler. A
request rejected by the pre-request handler no longer reads the body
at all; the existing keep-alive drain logic still consumes any framed
body afterwards.
Note: code that referenced req.body or body-derived form fields inside
the pre-request handler will now see an empty body. Inspect headers,
path, query parameters, or matched_route instead.
Also document the handler execution order in README and update the
pre-request cookbook pages (en/ja).
The constructor stored only uc.path in path_, discarding uc.query, so the
WebSocket upgrade handshake sent the Request-URI without the query string.
Append the query to path_ so query parameters (e.g. auth tokens) are sent.
The release check treated runs with an empty conclusion as failures.
Now it inspects each run's status and aborts with an error if any CI
check is still running, so releases wait until CI completes.
* add set_hostname_addr_map to WebSocketClient
* add WebSocketTest unit test cases
* SpecifyServerIPAddress_AnotherHostname
* SpecifyServerIPAddress_RealHostname
* Change wrong_ip from 0.0.0.0 to 192.0.2.1
Use 192.0.2.1 (RFC 5737 documentation address) to ensure it acts
as a non-routable address and does not alias to loopback.
* Fix style check
* set short timeout in WebSocketTest.SpecifyServerIPAddress_RealHostname
cannot reach wrong_ip
The auto-added `Expect: 100-continue` (for bodies >= 1024 bytes) decided
whether to withhold the request body based on raw socket readability via
select_read(). Over TLS, post-handshake records such as TLS 1.3 session
tickets make the socket readable without any HTTP response being
available, so the client withheld the body and then blocked reading a
response that never came, failing with `Failed to read connection`.
Decide based on whether a status line can actually be read within the
100-continue timeout instead: temporarily shorten the read timeout, try
to read the status line, and if none arrives, send the body and proceed
as usual (matching curl). This keeps the `100 Continue` and early
final-response paths working while no longer being fooled by TLS records.
Add a regression test using a raw OpenSSL server that never sends
`100 Continue`.
* Route proxy-enabled checks through is_proxy_enabled_for_host helper
In preparation for NO_PROXY support (#2446), centralize the proxy-enabled
decision in a single helper so the upcoming bypass logic can be added in
one place rather than to six divergent call sites. The helper's body for
now is identical to the existing condition; the host parameter is unused
until set_no_proxy() lands.
Refactored sites:
ClientImpl::create_client_socket
ClientImpl::handle_request (HTTP request rewrite)
ClientImpl::setup_redirect_client
ClientImpl::process_request (SSL is_proxy_enabled flag)
SSLClient::setup_proxy_connection
SSLClient::ensure_socket_connection
The two prepare_default_headers Proxy-Authorization injection blocks
(currently gated only on proxy auth credentials being set) are
intentionally not wrapped here. Doing so would change behavior in the
rare misconfiguration case where credentials are set without set_proxy,
so the gating is deferred to the NO_PROXY commit where it becomes
meaningful.
No behavior change. All 608 unit tests and the 22 squid-backed proxy
tests pass.
* Add detail::parse_proxy_url with control-char and scheme validation
Building block for the upcoming set_proxy_from_env (#2446). Parses
"http(s)://[user[:pass]@]host[:port][/...]" into a detail::ProxyUrl
struct.
Rejects:
- empty input
- any control character (< 0x20 or 0x7F), including CR/LF/NUL — these
would otherwise let a malicious env value inject extra header lines
into a CONNECT request or Proxy-Authorization header
- schemes other than http and https
- ports outside [1, 65535]
- malformed IPv6 host literals (validated via inet_pton(AF_INET6))
- non-numeric or trailing-garbage port strings
Notes:
- userinfo is split on the LAST '@' so passwords containing '@' are
preserved in the password field
- if no port is present, defaults to 80 (http) / 443 (https)
- integer parse goes through detail::from_chars to stay compatible
with -fno-exceptions builds
The helper has no callers yet; it lands consumer-side when
set_proxy_from_env arrives. All 608 unit tests pass.
* Add NO_PROXY parsing and matching helpers in detail namespace
Building blocks for the upcoming Client::set_no_proxy (#2446):
- NoProxyEntry / NoProxyKind: parsed list entry (wildcard, hostname
suffix, IPv4 CIDR, IPv6 CIDR)
- NormalizedTarget: pre-normalized form of the connection's target
host (lowercase, brackets stripped, trailing dot stripped, with
inet_pton already attempted)
- parse_no_proxy_entry / parse_no_proxy_list: token / list parsing.
Port-specific entries are rejected by design — cpp-httplib's other
host-keyed APIs (e.g. set_hostname_addr_map) are hostname-only, so
supporting host:port for NO_PROXY alone would be inconsistent.
- ipv4_in_cidr / ipv6_in_cidr: CIDR membership. IPv4 special-cases
prefix=0 to avoid the (1u << 32) shift UB. IPv6 uses byte-wise
memcmp plus a masked partial-byte compare.
- normalize_target: prepares the target host for matching. Routes
every IP literal through inet_pton so "127.0.0.1" vs
"127.000.000.001" vs decimal-form integers cannot be used to bypass
a NO_PROXY entry via alternate string forms.
- host_matches_no_proxy: matches a normalized target against an
entry list. Hostname suffix matching uses a dot-boundary rule so
"evilexample.com" does NOT match the entry "example.com". IPv4 and
IPv6 entries match only their own address family — IPv4-mapped IPv6
("::ffff:1.2.3.4") is not cross-matched against IPv4 entries.
These helpers have no callers yet; they land consumer-side in the
upcoming set_no_proxy / set_proxy_from_env commits. All 608 unit tests
pass.
* Add Client::set_no_proxy and wire NO_PROXY into proxy decision
Implements the user-facing half of #2446 (set_proxy_from_env follows in
the next commit). When a NO_PROXY pattern matches the target host, the
client now bypasses the configured proxy and the corresponding
Proxy-Authorization header is suppressed.
Public API:
- Client::set_no_proxy(const std::vector<std::string> &patterns)
Patterns: "*", hostname suffix (e.g. "example.com" or
".example.com"), IPv4/IPv6 CIDR (e.g. "10.0.0.0/8", "fe80::/10"),
or single IP literals. Replaces any previous list. Malformed
entries are silently dropped.
Internals:
- is_proxy_enabled_for_host now consults no_proxy_entries_, normalizing
the target through inet_pton so leading-zero or alternate-form IPs
cannot be used to bypass an entry.
- prepare_default_headers gates both Proxy-Authorization injection
blocks (basic and bearer) on is_proxy_enabled_for_host(host_).
Previously, Proxy-Authorization was sent whenever proxy auth
credentials were configured, even when the request was going direct
to the target. With NO_PROXY now in play, that path would leak
proxy credentials to the destination server — analog of the
redirect-leak class of bugs (cf. CVE-2023-32681 in Python requests,
GHSA-6hrp-7fq9-3qv2 in cpp-httplib).
- setup_redirect_client now takes the redirect target host as a
parameter and re-evaluates is_proxy_enabled_for_host against it.
no_proxy_entries_ is always copied to the redirect client so the
bypass policy follows across redirects. This is the cross-origin
leak surface that GHSA-c3h8-fqq4-xm4g lives in; centralizing the
decision through is_proxy_enabled_for_host removes the chance of
branch divergence.
- copy_settings copies no_proxy_entries_.
The slight behavior change for the rare misconfiguration "set
proxy_basic_auth without set_proxy" — Proxy-Authorization is no longer
sent in that case — is deliberate. The header has no addressee when
the proxy is unset.
All 608 unit tests and 22 squid-backed proxy integration tests pass.
* Add Client::set_proxy_from_env with httpoxy mitigation
Final user-facing piece for #2446. Reads proxy-related environment
variables and configures the client.
- HTTPS clients (SSLClient) read https_proxy / HTTPS_PROXY
- HTTP clients read http_proxy (lowercase only — see below)
- Both also read no_proxy / NO_PROXY
- Returns true if at least one variable was found and applied
The lowercase-only http_proxy rule mitigates httpoxy / CVE-2016-5385.
In CGI / FastCGI environments the uppercase HTTP_PROXY collides with
the HTTP_* namespace used to expose request headers, so a remote
attacker controlling the "Proxy:" header can inject a proxy URL.
cpp-httplib follows curl, Go, and Python requests in honoring only
the lowercase form. https_proxy/HTTPS_PROXY and no_proxy/NO_PROXY do
not have this problem because their names don't begin with HTTP_.
Scheme dispatch uses virtual is_ssl(): an SSLClient picks
https_proxy and a plain ClientImpl picks http_proxy. There is
intentionally no cross-scheme fallback — the two variables describe
different traffic.
set_proxy_from_env() reads getenv() synchronously and is documented
as "call once at startup" — concurrent setenv from other threads is
undefined.
All 608 unit tests pass.
* Add NO_PROXY behavior tests
27 black-box tests exercising the public Client API only (no detail::
calls, BORDER-friendly; no EXPECT_NO_THROW, -fno-exceptions-friendly).
In-process proxy mock + target server. Each test asserts which side
of the routing decision each request landed on, and what headers (in
particular Proxy-Authorization) the receiving side saw.
Coverage:
Suffix matching (dot-boundary rule)
- exact-host match
- subdomain match
- "evilexample.com" does NOT match "example.com" ← regression
guard for the classic NO_PROXY suffix-match pitfall
- "example.com.evil.com" does NOT match
- leading-dot pattern still matches the bare domain (Go/curl
convention)
- case-insensitive
- trailing-dot host normalization
Wildcard
- "*" bypasses everything
IP normalization
- exact IPv4 match
- "::1" matches "0:0:0:0:0:0:0:1" via inet_pton
- IPv4-mapped IPv6 ("::ffff:127.0.0.1") is NOT cross-matched
against an IPv4 entry
CIDR
- basic v4 in-cidr / not-in-cidr
- "0.0.0.0/0" (prefix=0; verifies no shift UB)
- bare IP treated as /32
- malformed prefix (/33) silently dropped → no NO_PROXY effect
Proxy-Authorization handling
- suppressed when NO_PROXY matches the target
- sent when NO_PROXY does not match
Backward compat
- default behavior unchanged when set_no_proxy is never called
Parsing edge cases
- port-specific entries ("host:port") rejected
- empty / whitespace tokens dropped
Cross-origin redirect (analog of GHSA-6hrp-7fq9-3qv2)
- redirect target in NO_PROXY → redirect leg goes direct, no
Proxy-Authorization carried over
set_proxy_from_env (Unix only — uses setenv/unsetenv)
- lowercase http_proxy applied
- uppercase HTTP_PROXY ignored (httpoxy / CVE-2016-5385)
- NO_PROXY-only env returns true and applies the bypass list
- CRLF in env value rejected (cf. CVE-2026-21428)
- empty env value treated as unset
635 tests (608 prior + 27 new) pass under both the regular and the
split builds.
* Document set_no_proxy and set_proxy_from_env in README
Adds two subsections under "Proxy server support":
- "Bypass the proxy for specific hosts (NO_PROXY)" — set_no_proxy,
pattern syntax, dot-boundary rule, IP normalization, limitations
(no port-specific entries, no v4-mapped v6 cross-match, replace
semantics).
- "Read proxy settings from the environment" — set_proxy_from_env,
which variables are read, the lowercase-only http_proxy rule with
an inline httpoxy / CVE-2016-5385 explanation, threading
expectations.
Documentation only. Closes the doc gap from #2446.
* Document NO_PROXY and set_proxy_from_env in cookbook c16-proxy
Replaces the now-incorrect Note at the bottom of c16-proxy ("cpp-httplib
does not read HTTP_PROXY...") with the actual API.
JA is the master per the project's translation workflow; the EN
translation lands in the same PR. Both pages remain `status: "draft"`
for normal review.
Adds two sections:
- Bypass the proxy for specific hosts (set_no_proxy):
pattern syntax, dot-boundary rule, case-insensitivity, IP
normalization via inet_pton, port-specific-entries unsupported,
malformed entries dropped.
- Read proxy settings from the environment (set_proxy_from_env):
which variables are read, lowercase-only http_proxy with an
inline httpoxy / CVE-2016-5385 explanation, threading caveat.
* Simplify NO_PROXY implementation per review
Apply seven post-implementation cleanups:
- Move ProxyUrl, ProxyEnvSettings and most helper forward declarations
below the BORDER. Only NoProxyKind/NoProxyEntry/NormalizedTarget stay
above (they are used as ClientImpl members or by inline cache state).
This shrinks the public header surface area considerably.
- Drop ProxyUrl::scheme: the field was write-only after parsing. Track
is_https as a local during parse_proxy_url and use it for the
default-port branch directly.
- Hoist the duplicate is_proxy_enabled_for_host(host_) gate in
write_request: the previous form had two adjacent gates bracketing
an unrelated end-server bearer-token block. Reordering puts the two
proxy-auth blocks together under a single gate.
- Drop the redundant trim_copy + empty-check inside parse_no_proxy_list:
detail::split already trims each token and skips empties, so the inner
work was dead code.
- Cache normalize_target(host_) on the client. host_ is const, so the
normalized form is invariant for the client's lifetime. The gate is
called up to 7 times per request when NO_PROXY is configured;
caching avoids repeating two heap allocations + two inet_pton calls
per request. Cross-host calls (only setup_redirect_client passing
next_host) still compute fresh.
- Trim narrative comments in setup_redirect_client and
set_proxy_from_env: replace WHAT-narration with single-line WHY
statements.
- Drop test comments that paraphrased their own test name.
All 635 unit tests pass under both the regular and split builds.
* Inline proxy URL parsing and env reading; drop intermediate structs
The previous design had two intermediate structs that existed only to
ferry parsed values between helper functions and the consuming method:
- detail::ProxyUrl: filled by parse_proxy_url, drained back into
proxy_host_ / proxy_port_ / proxy_basic_auth_* by set_proxy_from_env.
- detail::ProxyEnvSettings: bundle of two ProxyUrl + a NoProxyEntry
vector returned by read_proxy_env, drained by set_proxy_from_env.
Both bundles had exactly one producer and exactly one consumer. Drop
them and let the parsing flow directly into ClientImpl state:
- New private member ClientImpl::apply_proxy_url(url) parses a proxy
URL and, on success, assigns the result to proxy_host_, proxy_port_,
and proxy_basic_auth_*. Same validation as before (CRLF rejection,
scheme allowlist, port range, IPv6 bracket validation), same commit-
on-success ordering — the local variables are kept until every check
has passed so a malformed URL leaves no partial state.
- set_proxy_from_env now reads getenv() directly, dispatches between
https_proxy / http_proxy via virtual is_ssl(), and applies via
apply_proxy_url. NO_PROXY is parsed in place via parse_no_proxy_list.
Net effect:
- Two structs and two free helper functions removed (~150 lines of
declaration + body deleted).
- set_proxy_from_env body grows ~20 lines (still well under 50).
- Per-request hot path is unchanged (NoProxyEntry / NormalizedTarget
cache stays). Setup path is marginally faster (no intermediate
string copies through ProxyUrl / ProxyEnvSettings).
635 unit tests pass under both the regular and split builds.
* Trim doc comments to match the rest of httplib.h
The new code carried inline doc comments (15-line set_no_proxy block,
18-line set_proxy_from_env block, plus narrating comments inside parser
bodies, plus section dividers in the test file) that were heavy
compared to the rest of the codebase — neighboring setters like
set_proxy / set_proxy_basic_auth carry no doc at all, the test file
does not use sub-section dividers, and the README / cookbook already
document the behavior in detail.
Removed:
- Public-API doc blocks on set_no_proxy and set_proxy_from_env.
- Narrating comments inside parse_no_proxy_entry, normalize_target,
apply_proxy_url, host_matches_no_proxy that were just describing
the obvious code structure.
- Multi-line BORDER-rationale meta comments.
- In-test sub-section dividers ("// ---- Hostname suffix matching",
etc.) and per-class doc comments on the test fixtures.
- Test-side comments that paraphrased their own test name.
- Redundant ordering comments inside setup_redirect_client.
Kept:
- Security WHY comments (CRLF rejection, dot-boundary suffix matching,
httpoxy / CVE-2016-5385, GHSA-6hrp-7fq9-3qv2 analog, CVE-2026-21428).
- Regression-target WHY comments (UB shift on prefix=0).
- Non-obvious external knowledge (detail::split already trims).
635 unit tests still pass under both the regular and split builds.
* Add NO_PROXY tests covering edge cases found during PR review
Three regression guards added during review of an alternate NO_PROXY
implementation (PR #2449). All three pass on the current implementation
and surface bugs in the alternate one:
- BareIPv6LiteralMatchesIPv6Cidr: a host given as a bare IPv6 literal
(no surrounding brackets) must still be recognized as IPv6 for CIDR
matching. An implementation that only detects IPv6 when the host
string starts with '[' would split the host at the first ':' and
misclassify it as a hostname.
- TrailingDotOnEntryIsNormalized: trailing dots must be canonicalized
on BOTH sides — host and entry. An implementation that strips the
host-side trailing dot only would fail to match host "example.com"
against entry "example.com." because the substring lengths differ.
- ValidEntryWithSurroundingWhitespaceStillMatches: an entry with
leading/trailing whitespace must still match. An implementation
that feeds raw tokens directly to inet_pton would reject valid
CIDRs (" 10.0.0.0/8 ") because of the spaces.
635 unit tests pass.
* Unify IPv4/IPv6 CIDR matching into a single byte-buffer helper
Adopts the unified 16-byte address representation suggested by the
alternate NO_PROXY implementation in PR #2449. Both v4 and v6 entries
now share one storage type and one matcher; the v4/v6 distinction is
only the address-family flag and the max prefix length.
- detail::NoProxyEntry: replaces in_addr v4_net + in6_addr v6_net
with a single IPBytes net (std::array<uint8_t, 16>). v4 occupies
the first 4 bytes, v6 fills all 16.
- detail::NormalizedTarget: replaces in_addr v4 + in6_addr v6 with
a single IPBytes ip.
- Replaces detail::ipv4_in_cidr and detail::ipv6_in_cidr with one
detail::ip_in_cidr that takes the address, the network, the prefix
length and the family's max bits (32 for v4, 128 for v6). The mask
is constructed by the byte-fill approach from the previous v6
helper, which is straightforward to read and avoids the shift UB
that the v4 helper had to special-case.
- The NoProxyKind enum keeps IPv4Cidr / IPv6Cidr as separate values
so the match dispatch stays explicit and IPv4 entries cannot
accidentally cross-match an IPv6 target (the same address-family
isolation the previous code had).
Net change: -28 lines + -1 helper function. All 30 NoProxyTest cases
plus 643 unit tests pass under both the regular and split builds.
* Drop set_proxy_from_env per #2446 discussion
Per @unterwegi's feedback in #2446, environment variable handling
conflicts with cpp-httplib's long-standing policy of explicit
configuration (e.g. set_ca_cert_path requires explicit paths instead
of reading SSL_CERT_FILE / SSL_CERT_DIR). The NO_PROXY matching logic
is the genuinely tricky part worth keeping in the library; getenv
parsing is trivial and is left to the caller.
- Remove Client::set_proxy_from_env, ClientImpl::set_proxy_from_env,
and ClientImpl::apply_proxy_url
- Remove ScopedEnv test helper and env-driven NoProxyTest cases
- Replace the "Read proxy settings from the environment" docs with a
short snippet showing how to parse no_proxy and feed set_no_proxy()
- Keep set_no_proxy() and all NO_PROXY pattern matching intact
* docs: blend NO_PROXY env-var note into c16-proxy cookbook style
Match the granularity of the surrounding sections: imperative heading,
inline paragraph instead of a heavyweight callout, and a simpler getenv
snippet without the C++17 if-init.
* Skip digest 407 retry when target is bypassed by NO_PROXY
Before this fix, a NO_PROXY-bypassed origin that returns
407 Proxy-Authentication-Required with a Digest challenge would
trigger the same retry path the proxy uses, computing a
Proxy-Authorization header from proxy_digest_auth_* and sending the
user's proxy credentials directly to that (potentially hostile)
origin.
A 407 from a direct origin is semantically meaningless — RFC 9110
defines it strictly as a proxy response. Skip the retry when the
current target is not actually going through the proxy and let the
407 propagate to the caller unchanged.
Regression test BypassedTargetReturning407DoesNotLeakProxyDigest
Credentials reproduces the leak without this gate.
* Make set_no_proxy safe across redirects and keep-alive
Two correctness bugs that the dynamic NO_PROXY API exposed:
1. Multi-hop redirect through a bypassed host lost the proxy.
setup_redirect_client only copied proxy_host_/port and the proxy auth
credentials when is_proxy_enabled_for_host(next_host) was true. After
a chain like A (proxied) -> B (NO_PROXY-matched, direct) -> C, the
redirect client built for B had no proxy configured, so the further
B -> C hop went direct even when C should have been proxied. Copy the
proxy configuration unconditionally and let is_proxy_enabled_for_host
gate at send time. The next_host parameter is no longer needed and
removed from the signature.
2. Keep-alive socket reuse with a stale bypass decision. set_proxy() /
set_no_proxy() left the existing keep-alive socket open, so the next
request reused a socket pointed at the previous endpoint (proxy vs
origin) while write_request emitted the new request-line form
(absolute vs relative URL). Add invalidate_keep_alive_socket() and
call it from both setters; the helper handles the in-flight case by
deferring the close.
Regression tests MultiHopRedirectThroughBypassedHostKeepsProxy and
KeepAliveSocketInvalidatedOnSetNoProxy reproduce each bug without the
respective fix.
* Tighten NO_PROXY entry parsing
Three small parser fixes surfaced during code review:
- Accept bracketed IPv6 entries like "[::1]" and "[fe80::]/10". Users
coming from URL syntax naturally write the bracketed form; previously
it was silently rejected because inet_pton does not accept brackets
and the subsequent ':' check tripped.
- Reject malformed trailing-slash CIDRs like "127.0.0.1/" instead of
silently treating them as /32 (or /128). A typoed entry quietly
turning into a single-host bypass changes semantics with no
diagnostic.
- Delete detail::parse_no_proxy_list — leftover from the removed
set_proxy_from_env path, no longer called from anywhere.
New regression tests: BracketedIPv6EntryAccepted,
BracketedIPv6CidrEntryAccepted, TrailingSlashCidrIsRejected.
* Refactor: introduce disconnect() and remove invalidate_keep_alive_socket
Replace the repeated `shutdown_ssl + shutdown_socket + close_socket`
pattern with a single `disconnect(bool gracefully)` helper. Used by
`stop()`, the send_() peer-closed and epilogue branches, and the close
in process_request after a non-keep-alive response.
Drop `invalidate_keep_alive_socket()` — its body collapses to a
`lock + disconnect()` pair which is now inlined in `set_proxy()` and
`set_no_proxy()` directly.
Also simplify `setup_redirect_client`: drop the now-unused next_host
parameter and the verbose comment block; the per-target proxy decision
is re-evaluated at send time anyway.
Net -47 lines in httplib.h.
* Fix MultiHopRedirect test on Windows; trim NoProxyTest comments
The bypass leg redirected to "http://localhost:<port>/...", but on
Windows `localhost` resolves to ::1 first while the mock server is
bound to 127.0.0.1, causing the redirect leg to time out. Use the
literal 127.0.0.1 in the Location and switch the NO_PROXY entry to
match, so the test exercises the same multi-hop path on every
platform.
Also trim the heavier inline comments and EXPECT messages I added on
recent NoProxyTest cases so they match the surrounding test style.
* Consolidate NoProxyTest server boilerplate; drop hardcoded sentinel ports
Add a small ScopedServer helper to no_proxy_test that wraps the
bind/listen/thread/cleanup dance (~13 lines per server before). Use it
to rewrite the four big tests (Redirect, BypassedTarget407, MultiHop,
KeepAlive), shaving ~100 lines.
Also drop the hardcoded port-1 / port-80 sentinels that violated the
"new standalone tests MUST use bind_to_any_port" convention and risked
collisions across gtest shards: re-use existing dynamic ports
(target.port() / bypass_server.port()) instead.
Verified pass under 4-shard parallel run.
* Trim README NO_PROXY section to match surrounding granularity
The block had ballooned to 62 lines while neighboring subsections
(Authentication, Proxy server support, Range, Redirect) are 13-18 each.
Collapse to a single code example + one-line behavior summary; point at
the cookbook for the entry-form details, env-var parsing snippet, and
httpoxy note that used to live inline.
Adds Ubuntu and macOS CI jobs that build BoringSSL from source and exercise cpp-httplib's existing OpenSSL backend path (continue-on-error: best-effort). Makes SSLClientServerTest.TlsVerifyHostname backend-aware (BoringSSL is SAN-only per RFC 6125 §6.4.4). README notes BoringSSL as a best-effort variant with the C++14 and SAN-only caveats.
* Replace deprecated SecTrustCopyAnchorCertificates on macOS
SecTrustCopyAnchorCertificates was deprecated in macOS 13. Switch to
SecTrustSettingsCopyCertificates, iterating over the System, Admin, and
User trust domains to retain equivalent coverage of anchor certificates.
* Restrict Keychain cert loading to macOS
TARGET_OS_MAC is true on all Apple platforms including iOS, tvOS, and
watchOS, which caused the keychain enumeration path to be compiled on
iOS where SecTrustSettingsCopyCertificates is unavailable.
Narrow the auto-enable and the Security.h include guards to
TARGET_OS_OSX, and emit an explicit #error when the user defines
CPPHTTPLIB_USE_CERTS_FROM_MACOSX_KEYCHAIN on a non-macOS Apple platform,
directing them to use set_ca_cert_path() with a bundled CA file.
Addresses the iOS build break reported in #2454.
* Add iOS header parse check to CI
Run a cross-compile syntax check against the iOS SDK to catch
accidental use of macOS-only APIs or guards (e.g. TARGET_OS_MAC vs
TARGET_OS_OSX) that would silently break iOS builds. Also verify that
defining CPPHTTPLIB_USE_CERTS_FROM_MACOSX_KEYCHAIN on iOS fires the
expected #error.
iOS is not officially supported as a runtime target; this job only
guarantees the header stays parse-clean on iOS toolchains.
* The current implementation of "Server::stop()" doesn't throw an exception, so why not specify this explicitly?
* Adding the missing "noexcept" to the declaration
parse_header() applied decode_path_component() to every header value
except Location and Referer, after is_field_value() validation. Wire
sequences like %0D%0A passed the check and expanded into literal CR/LF
inside stored values, enabling response splitting, log injection, and
proxy smuggling. %3D/%2C/%3B also flipped Cookie and X-Forwarded-For
boundaries against WAFs inspecting the wire form.
RFC 9110 §5.5 specifies header values as opaque octets. Drop the
decoding and the Location/Referer special case (originally workarounds
for the same auto-decode misbehavior; redundant once decoding stops).
Applications that need URI semantics should call decode_uri_component()
or decode_path_component() on the result explicitly.
Add regression tests covering CRLF injection, %3D/%2C/%3B boundary
characters, UTF-8 and %uXXXX sequences, browser-style Referer URLs
containing %0A (issue #2033), and the explicit-decode migration
pattern.
* Make ThreadPool ctor exception-safe on partial thread creation
If std::thread construction throws partway through the ThreadPool
constructor (e.g., pthread_create returns EAGAIN under thread-resource
pressure), the partially-built threads_ vector would destruct joinable
std::thread objects, calling std::terminate(). Wrap the spawn loop and,
on failure, signal shutdown to the workers already created, join them,
and rethrow.
Adds a reproducer test in test_thread_pool.cc that interposes
pthread_create at link time to deterministically fail the second call,
gated to POSIX + exceptions-enabled builds.
Fix#2444
* Strip ASAN from test_thread_pool to coexist with pthread_create override
Linux libasan installs its own pthread_create interceptor; our in-binary
symbol override sits on top of it and corrupts ASAN's thread bookkeeping,
which surfaces as "Joining already joined thread" on the very first test.
Disable ASAN for this small unit-test binary -- ThreadPool memory behavior
is still exercised under ASAN by the main `test` binary.
strtoul silently accepts a leading "-" and wraps via unsigned
arithmetic, so chunk-size "-2" produced ULONG_MAX-1, bypassing the
ULONG_MAX guard and letting a client drive the server toward unbounded
allocation.
Replace strtoul with a manual hex parser that requires at least one hex
digit, detects size_t overflow per digit, and accepts only chunk-ext or
end-of-line after the digits (RFC 9112 §7.1).
When the upstream request to httpbingo.org transiently fails, cli.Get()
returns nullptr and the next line dereferences it (res->status / res->body),
producing a SEGV in std::string::begin() under ASan. Sibling templates in
the same file already use ASSERT_TRUE(res != nullptr); apply the same
guard to the four Get() call sites in KeepAliveTest so a flaky network
turns into a clean test failure instead of a crash.
The hosted windows-latest runner is migrating from VS 2022 to VS 2026
(NOTICE: windows-2025 -> windows-2025-vs2026 by 2026-05-12). The
hardcoded path C:\Program Files\Microsoft Visual Studio\2022\Enterprise
no longer exists on the new image, so vcvarsall.bat silently fails and
'cl' is not on PATH.
Resolve the install path via vswhere.exe (stable location, version
agnostic) and exit if vcvarsall.bat fails so future breakage surfaces
immediately instead of as a confusing 'cl not recognized' error.
The test referenced detail::can_compress_content_type, which lives below
the split BORDER in httplib.h and is therefore not visible to test.cc in
test_split / Windows-CMake builds. EXPECT_NO_THROW also expanded to a
try/catch that would not compile under -fno-exceptions. The OSS-Fuzz
reproducer in test/fuzzing/corpus already serves as the regression test
for #508087118 and is exercised by make fuzz_test.
When a glob like clusterfuzz-testcase-minimized-foo_fuzzer-* did not
match anything, bash passed the literal pattern through. The standalone
runner then tried to open it, tellg() returned -1, and the resulting
size_t cast (SIZE_MAX) crashed std::vector with length_error. This made
fuzz_test fail loudly during bisects to commits before a corpus file
landed. Filter each glob through a -f test so unmatched patterns are
silently skipped with a "(no XXX corpus)" notice, mirroring what was
already done for url_parser_fuzzer.
str2tag_core is recursive (one frame per character), so a long runtime
input such as a fuzzer-supplied Content-Type would overflow the stack.
Rewrite the runtime entry point str2tag() iteratively while keeping the
recursive constexpr str2tag_core for compile-time UDL evaluation. The
hash output is unchanged for all inputs.
Same root cause as #508342856 (fixed in 2d2efe4): an oversized
Content-Length value (here 4467440718547775) caused res.body.reserve()
to attempt a multi-petabyte allocation. The UBSAN fuzzer job surfaced
it as a std::bad_alloc-driven abort, while the ASAN job for #508342856
reported it as allocation-size-too-big. The payload_max_length_ cap
introduced in 2d2efe4 already addresses both.
A malicious or malformed server response with an enormous Content-Length
header (e.g. 20000000000) caused the client to call res.body.reserve(len)
with the untrusted value, triggering OOM before read_content's
payload_max_length_ check could take effect. Cap the pre-reservation
at payload_max_length_, since reading more than that is never useful.
* Restore getaddrinfo_a path with proper worker-completion wait (#2431)
5ebbfee dropped the Linux/glibc getaddrinfo_a branch entirely to avoid
the stack-use-after-free reported in #2431. That sidestepped the bug
but lost the asynchronous-resolution capability getaddrinfo_a is meant
to provide.
Bring the getaddrinfo_a branch back with the actual fix on the
cancellation path: after gai_cancel() — which is non-blocking and may
return EAI_NOTCANCELED while the resolver worker is still mid-operation
— call gai_suspend() with no timeout in a loop until gai_error() stops
returning EAI_INPROGRESS. Only then is it safe to destroy the
stack-local gaicb. freeaddrinfo() is also called on any partially
populated ar_result so that error paths do not leak.
This is the approach suggested in the issue body, with gai_suspend
substituted for the busy-poll over gai_error.
The issue-2431 reproducer test (run under ASAN with sinkhole DNS) is
unchanged and continues to drive the cancel path; it now exercises the
restored getaddrinfo_a code rather than the std::thread fallback.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Simplify getaddrinfo_a branch (idiomatic init, scope_exit, fewer comments)
- Value-initialize gaicb / sigevent / timespec with {} instead of memset
- Replace the two manual freeaddrinfo calls with a scope_exit guard, with
request.ar_result reset to nullptr on the success path to release
ownership to the caller (matches the addrinfo cleanup pattern used in
detail::create_socket and friends)
- Inline the single-call wait_for_request_done lambda
- Drop the (const struct gaicb *const *) cast — the array decays without
it under C++11
- Tighten the leading comment to the one load-bearing fact (#2431) and
the trade-off about pathological DNS waits; remove a stale claim that
the inner loop handles EAI_INTR (the loop checks gai_error, not the
gai_suspend return value)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add test for #2435 mmap::open with concurrent writer
Verifies that detail::mmap can open a file held open with GENERIC_WRITE
by another handle (e.g. an active log file). Currently fails on Windows
because CreateFile2 omits FILE_SHARE_WRITE.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix#2435: allow mmap to open files held open for writing
Add FILE_SHARE_WRITE to the share mode passed to ::CreateFile2 so
detail::mmap can open a file even when another process holds it open
with GENERIC_WRITE (e.g. an active log file). Without this, CreateFile2
fails with ERROR_SHARING_VIOLATION because the new opener's share mode
must permit the existing handle's access mode.
This brings the Windows path's behavior in line with the POSIX path
which uses ::open(O_RDONLY) and is unaffected by other processes'
write handles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The macos-latest runner is consistently slower than ubuntu-latest for
the ASAN+mbedTLS test binary, and SHARDS=2 still flakes there on the
ServerTest fixture's rapid bind/connect cycle against a fixed port.
Serialize fully (SHARDS=1) on macOS only; ubuntu mbedTLS stays at 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
With the close_notify mid-response fix and SHARDS=2 mitigation, the
mbedTLS legs run reliably on both ubuntu and macos. Drop the
continue-on-error escape hatch so future regressions actually break the
build.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Under ASAN+mbedTLS, the default 4-way sharding loads CI runners enough
that timing-sensitive ServerTest cases (Delete, PostMethod2, GetStreamed,
...) flake on what looks like first-request keep-alive reuse. Reducing
to 2 shards halves contention and historically stabilizes these on local
runs. The total test time goes up roughly 1.5x (still well under the job
budget) which is an acceptable trade for reliability.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The mbedTLS backend's read() returned -1 with err.code = PeerClosed when
the peer sent close_notify, while OpenSSL and wolfSSL surface it as 0
(clean EOF). The result was that an SSL response without Content-Length
or chunked Transfer-Encoding — terminated by connection close — was
reported as "Failed to read connection" on mbedTLS, even though the
body had been fully delivered.
Translate PeerClosed into a return value of 0 to match the other
backends. This re-enables SSLTest.ResponseBodyTerminatedByConnectionClose
on mbedTLS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These tests reach out to external services (httpbin, YouTube, ...) and
flake on CI runners whenever those services are slow or unreachable.
The previous shard runner script silently masked these failures; now
that runs report them faithfully, default the filter to -*_Online.
Override via workflow_dispatch + the gtest_filter input to include
them when explicitly desired.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Skip SSLTest.ResponseBodyTerminatedByConnectionClose under
CPPHTTPLIB_MBEDTLS_SUPPORT until the close_notify-mid-response handling
is brought into parity with the OpenSSL and wolfSSL backends. The test
verifies a successful read past the server's close, which mbedTLS
currently reports as an I/O error.
Mark the mbedTLS matrix legs (ubuntu and macos) as
continue-on-error: true. Several timing-sensitive ServerTest cases
(PostMethod2, GetStreamed, Brotli, ...) flake under ASAN+mbedTLS in
ways unrelated to cpp-httplib code; isolating these into a non-blocking
slot keeps master green while the flakiness is investigated separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a libwolfssl entry to lsan_suppressions.txt to mirror the existing
libcrypto rule: the wolfSSL ECC subsystem caches per-handshake buffers
that are only freed at library shutdown, which the test binaries do
not perform. These are not leaks in cpp-httplib code.
Disable fail-fast on the ubuntu / macos / windows matrices so a failure
in one TLS backend does not cancel the others; with the runner now
detecting failures correctly, we want to see the full picture per run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These tests wrote to a hardcoded "/tmp/" path which does not exist on
Windows, causing the file write to silently fail and the subsequent
make_file_body / make_file_provider call to return zero-sized data.
Use a relative path under the test working directory instead so the
test runs identically on every platform.
Also dump the shard log when a shard's process exits non-zero even
when the gtest summary appears clean (e.g. sanitizer report after
the suite, or assertion-based abort) — previously such failures were
detected only via overall rc and showed no diagnostic output.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous logic considered a shard "passed" if its log contained any
[ PASSED ] line, missing the case where some tests pass and some fail
(both [ PASSED ] N tests. and [ FAILED ] M tests, listed below:
appear in the gtest summary). Exit codes from the test binaries were
also ignored.
Now require both: an [ PASSED ] line, no [ FAILED ] line, and a
zero exit code. Track each shard's PID so wait can surface non-zero
exits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Linux/glibc branch of detail::getaddrinfo_with_timeout used
getaddrinfo_a(GAI_NOWAIT) with a stack-local struct gaicb. On the
connection-timeout branch it called gai_cancel(), which is non-blocking
and may return EAI_NOTCANCELED -- in that case the resolver worker
thread is still alive and writes back to ar_result on the now-destroyed
stack frame after the function has already returned.
Drop the entire #elif _GNU_SOURCE && __GLIBC__ branch and let glibc
fall through to the existing std::thread + std::shared_ptr<State>
implementation that the file already uses for other Unix systems. That
path captures shared ownership in the resolver lambda, so the state
outlives the caller's frame whether or not the worker finishes in
time -- no stack frame is ever referenced after return.
The reproducer added in #2433 (issue-2431 repro CI job) goes from
hanging at job teardown to passing in ~25s with this change.
* Add reproducer for #2431 (getaddrinfo_a use-after-free)
On Linux/glibc, getaddrinfo_with_timeout() runs DNS asynchronously via
getaddrinfo_a(GAI_NOWAIT) using a stack-local gaicb. When gai_suspend()
hits the connection timeout, gai_cancel() is called and the function
returns immediately — but gai_cancel() is non-blocking and can return
EAI_NOTCANCELED, leaving the resolver worker thread alive and still
referencing the destroyed stack frame.
Adds three opt-in gtest cases (GetAddrInfoAsyncCancelTest.*) that
exercise the cancel path repeatedly. They are gated on Linux/glibc +
CPPHTTPLIB_USE_NON_BLOCKING_GETADDRINFO at compile time, and on the
CPPHTTPLIB_TEST_ISSUE_2431=1 env var at runtime, so a normal `make
test` run is unaffected.
Also adds a dedicated CI job (issue-2431-repro) and a Docker-based
local runner (test/run_issue_2431_repro.sh) that sinkhole UDP/53 so
the timeout branch is taken, and run the test under ASAN/LSAN. With
the bug present these runs are expected to fail; with a fix applied
they should pass.
Refs: https://github.com/yhirose/cpp-httplib/issues/2431
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix split build for #2431 reproducer tests
The new GetAddrInfoAsyncCancelTest cases call detail::getaddrinfo_with_timeout
directly. In split builds (make test_split) split.py moves the definition into
httplib.cc and strips `inline`, so the symbol is not declared in the public
httplib.h and test.cc fails to compile -- breaking the ubuntu/test-no-exceptions
CI jobs that the PR description says should be unaffected.
Add a forward declaration in test.cc, gated by the same #if as the tests
themselves, so it links against the split-build symbol without changing the
header-only build.
* Cap issue-2431 repro job at 5 minutes
The bug manifests as orphan getaddrinfo_a resolver workers that keep the
runner from completing job teardown -- the previous run had all steps
succeed in ~1m37s but then hung in "Cleaning up orphan processes" for
~57m before GitHub force-killed the job.
A job-level timeout-minutes makes the failure signal fast and predictable:
bug present -> killed at 5 min, bug fixed -> ~2 min pass. Step-level timeout
isn't enough since the hang is in post-job cleanup, not the test step.
* Enable ASAN detect_stack_use_after_return for #2431 repro
The bug is a textbook stack-use-after-return: a stack-local struct gaicb
is destroyed when getaddrinfo_with_timeout returns after gai_cancel()
yields EAI_NOTCANCELED, then the still-live resolver worker thread writes
back into the freed frame. ASAN's detect_stack_use_after_return is the
direct detector for exactly this pattern -- enabling it lets the failure
surface as a clear ASAN diagnostic during the test run instead of as an
orphan-process hang at job teardown.
* Revert ASAN detect_stack_use_after_return for #2431 repro
The option did not detect the bug in CI -- the resolver worker write
likely lands on the heap (via the gaicb's pai pointer) or happens after
the test process exits, neither of which stack-use-after-return can
catch. Roll back to relying on the job-level timeout: bug present ->
post-cleanup hangs ~8min then job-level timeout cancels at 10min total;
bug fixed -> job completes in ~2min.
* Switch issue-2431 repro to a delayed loopback DNS test fixture
The previous repro setup dropped UDP/53 outright, which made glibc's
resolver hang forever on every lookup -- the worker never actually
received a response and so never reached the buggy write-back path
that #2431 is about. As a result, neither the broken HEAD nor the
fix made any visible difference in CI: both produced "tests pass +
post-cleanup hangs ~10min" because the orphan resolver thread is a
structural property of *any* getaddrinfo path on a hung resolver,
not a property of the bug.
Replace the sinkhole with a small loopback test fixture
(test/dns_test_fixture.py, ~50 lines, stdlib only) that answers DNS
queries after a 3s delay -- longer than the test's 1s timeout. An
iptables NAT rule routes the test job's lookups to the fixture
without touching /etc/resolv.conf, so the rest of the runner's DNS
behaviour is unaffected.
With ASAN's detect_stack_use_after_return enabled, the worker's
late write-back into the destroyed gaicb stack frame is now caught
as a stack-use-after-return diagnostic, so the broken HEAD fails
fast at the test step (clear red) and the fix turns the same job
green in well under a minute.
Same fixture is wired into both the GitHub Actions job and the
docker-based test/run_issue_2431_repro.sh script, so local repro on
macOS and CI repro on Linux exercise the identical path.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix#2427
* Use setarch -R on Linux to fix ASAN crash on WSL2
WSL2 uses high-entropy ASLR which conflicts with ASAN's shadow memory
requirements, causing the ASAN runtime to crash at startup. Running tests
via setarch -R (ADDR_NO_RANDOMIZE) disables ASLR for the test process,
allowing ASAN to initialize correctly.
Previously, dynamic threads exited as soon as their current task
completed and the queue was empty. This caused excessive thread
creation/destruction under bursty or long-lived workloads (e.g., SSE
streaming), degrading tail latency. Now dynamic threads loop back and
wait for CPPHTTPLIB_THREAD_POOL_IDLE_TIMEOUT (3s) before exiting,
allowing them to be reused for subsequent tasks.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Added `set_websocket_max_missed_pongs` method to configure unresponsive-peer detection.
- Updated README and documentation to clarify WebSocket limitations and features.
- Introduced tests for detecting non-responsive peers and ensuring responsive peers do not trigger timeouts.
The server listens on AF_INET6 only (::1), so the test fails:
[ RUN ] WebSocketIntegrationTest.SocketSettings
test/test.cc:17160: Failure
Value of: client.connect()
Actual: false
Expected: true
Fixes#2419.
Co-authored-by: Jiri Slaby <jslaby@suse.cz>
adds support for pre-existing `zstd::libzstd` which is useful for
projects that bundle their own zstd in a way that doesn't get caught by
`CONFIG`
Signed-off-by: crueter <crueter@eden-emu.dev>
* Remove 32-bit limitation
* Fix build problems
* Add 32-bit disclaimer and fix MSVC x86 warnings
- Move 32-bit warning to top of README with strong disclaimer
- Add static_cast<size_t> to fix truncation warnings on 32-bit MSVC
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Update README for clarity and quick start instructions
- Refine default config.toml with hostname and base path
- Adjust index.md files for consistent heading levels
- Simplify CSS for code block styling and remove unnecessary theme switching
- Refactor SiteConfig to derive full base URL from hostname and base path
- Update MarkdownRenderer to remove light theme handling
- Implemented a search button in the header of each documentation page.
- Added a search modal that allows users to input search queries.
- Integrated a JavaScript search feature that fetches and displays results from a new `pages-data.json` file.
- Each documentation page now includes a search overlay for improved navigation.
- Updated the main JavaScript file to handle search logic, including result highlighting and navigation.
- Created a `pages-data.json` file containing metadata for all documentation pages to facilitate search functionality.
- Added a favicon link to all tour pages in the Japanese documentation.
- Updated navigation links to include SVG icons for Home and GitHub.
- Changed language button to include an SVG icon for better visual representation.
- Improved theme toggle button to use SVG icons for light and dark modes.
- Refactored the documentation build commands in the justfile for clarity and consistency.
* [CMake] New component MbedTLS
New component MbedTLS.
* Fix case
Fix case: HTTPLIB_REQUIRE_OPENSSL=OFF; HTTPLIB_REQUIRE_MBEDTLS=ON
* [CMake] Test target MbedTLS::tfpsacrypto
[CMake] Test target MbedTLS::tfpsacrypto.
* [CMake] Test MbedTLS::mbedx509
[CMake] Test MbedTLS::mbedx509.
* Revert "[CMake] Test MbedTLS::mbedx509"
This reverts commit 1d0b91f59a.
* Revert "[CMake] Test target MbedTLS::tfpsacrypto"
This reverts commit bf099f6264.
* Fix problem caused by the recent performance improvement
* wolfSSL support
wolfSSL support.
Partly solve https://github.com/yhirose/cpp-httplib/issues/2371. Only
meson is missing.
* Solve https://github.com/yhirose/cpp-httplib/issues/2361
Solve https://github.com/yhirose/cpp-httplib/issues/2361.
Apply `WARNING`.
* Fix variable
Fix variable.
* [CMake] Solve incompatibilities with loop
Solve incompatibilities with loop.
* Fix
Fix.
* Remove debug prints
Remove debug prints.
* [CMake] Fix bug
Prevent a bug aus the required and if available libraries are checked
independently from each other. A could be chosen in required but B could
be chosen in if available and everything would pass.
* Remove debug print
Remove debug print.
* Restore change
Restore change.
---------
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* Add initial documentations
* Update documentation for Basic Client and add WebSocket section
* feat: add a static site generator with multi-language support
- Introduced a new Rust-based static site generator in the `docs-gen` directory.
- Implemented core functionality for building sites from markdown files, including:
- Configuration loading from `config.toml`.
- Markdown rendering with frontmatter support.
- Navigation generation based on page structure.
- Static file copying and output directory management.
- Added templates for base layout, pages, and portal.
- Created a CSS file for styling and a JavaScript file for interactive features like language selection and theme toggling.
- Updated documentation source with new configuration and example pages in English and Japanese.
- Added a `justfile` target for building the documentation site.
* Add language/theme toggle functionality
- Created a new Japanese tour index page at docs/ja/tour/index.html
- Implemented navigation links for various sections of the cpp-httplib tutorial
- Added a language selector to switch between English and Japanese
- Introduced theme toggle functionality to switch between light and dark modes
- Added mobile sidebar toggle for better navigation on smaller screens
* WebSocket support
* Validate selected subprotocol in WebSocket handshake
* Fix problem with a Unit test
* Dynamic Thread Pool support
* Fix race condition in new Dynamic ThreadPool
* Remove macOS select() fallback
macOS has supported `poll` for a long time now, so there's no need for
the specific `select` code paths.
With this commit, we can successfully build on visionOS.
Signed-off-by: Adrien Gallouët <adrien@gallouet.fr>
* Fix coding style
Signed-off-by: Adrien Gallouët <adrien@gallouet.fr>
---------
Signed-off-by: Adrien Gallouët <adrien@gallouet.fr>
* Fix compilation on BoringSSL by replacing ASN1_TIME_to_tm
BoringSSL doesn't expose `ASN1_TIME_to_tm`.
This patch switches to using `ASN1_TIME_diff` to calculate `time_t`.
This is supported by OpenSSL, LibreSSL, and BoringSSL, and also avoids
the platform-specific `timegm` vs `_mkgmtime` logic.
Signed-off-by: Adrien Gallouët <adrien@gallouet.fr>
* Format code
Signed-off-by: Adrien Gallouët <adrien@gallouet.fr>
* Use detail::scope_exit
Signed-off-by: Adrien Gallouët <adrien@gallouet.fr>
---------
Signed-off-by: Adrien Gallouët <adrien@gallouet.fr>
* Correct sign comparison error with sk_X509_OBJECT_num
In some build configurations, sk_X509_OBJECT_num (from BoringSSL) returns a size_t. Comparing this directly against a signed int loop counter triggers -Werror,-Wsign-compare.
Instead, move the count into a local int variable so the compiler uses implicit conversion to ensure type consistency during the loop comparison.
* Update httplib.h
* Update httplib.h
Missed a s/int/decltype(count)
* Fix#2339
* Fix CI errors
* Fix Windows build error
* Fix CI errors on Windows
* Fix payload_max_length initialization in BodyReader
* Initialize payload_max_length with CPPHTTPLIB_PAYLOAD_MAX_LENGTH in BodyReader
* Update README and tests to clarify payload_max_length behavior and add no limit case
* Fix server thread lambda capture in ClientVulnerabilityTest
* Add C++ modules support
* Add module examples
* Missing semicolon
* Update GitHub Actions script and create a modules updating script
* Name the unused param
* Use the guarded/direct export of header approach
* Update CMakeLists.txt
Co-authored-by: Andrea Pappacoda <andrea@pappacoda.it>
* Update CMakeLists.txt
Co-authored-by: Andrea Pappacoda <andrea@pappacoda.it>
* Split scripts into split.py and generate_module.py
---------
Co-authored-by: Andrea Pappacoda <andrea@pappacoda.it>
* Add Tests for case sensitive hostname verfication
* Modify check_host_name to perform case insensitive comparisons during ssl hostname validation
---------
Co-authored-by: Tabor, Chris <chris.tabor@commscope.com>
Both APIs conceptually do the same thing: "use these CA certs for
verification." However, set_ca_cert_store() falls through to the else
branch in load_certs() where system certs are added to the user's
custom store, defeating the purpose of certificate pinning.
This change makes set_ca_cert_store() behave consistently with
set_ca_cert_path() by checking ca_cert_store_ before loading system
certificates.
Added test to verify system certs are not loaded when custom store is set.
Co-authored-by: Your <you@example.com>
* Resolve#2262
* Enhance request handling on Windows by adding early response check for large request bodies
* Enhance early response handling for large requests with long URIs on Windows
* Fix problem with 'windows without SSL`
* Fix payload limit enforcement for requests without Content-Length on Windows
- Enable MSG_PEEK on Windows (non-SSL builds) to detect payloads without Content-Length
- Only use MSG_PEEK when payload_max_length is set to a finite value to avoid blocking
- Use read_content_without_length for actual size checking to support any payload limit
- Set 413 Payload Too Large status before rejecting oversized requests
This fixes three test cases on Windows:
- RequestWithoutContentLengthOrTransferEncoding (no payload limit)
- NoContentLengthPayloadLimit (8-byte limit)
- NoContentLengthExceeds10MB (10MB limit)
* clang-format
* Ensure payload_max_length_ is respected for compressed payloads
* Fix Denial of service (DOS) using zip bomb
---------
Co-authored-by: Hritik Vijay <hey@hritik.sh>
On Windows cpp-httplib defines ssize_t, therefore applications needing
to define ssize_t for their own needs or are using libraries that do
require a means to avoid a possible incompatible redefinition.
Signed-off-by: Ralph Sennhauser <ralph.sennhauser@gmail.com>
To avoid surprises in the projects consuming the library, don't define
BUILD_SHARED_LIBS option ourselves but just use its value, if provided,
to define HTTPLIB_SHARED option which can be also set directly to
specify whether we should build static or shared library.
Closes#2263.
* Fix HTTP 414 errors hanging until timeout
* All errors (status code 400+) close the connection
* 🧹
---------
Co-authored-by: Wor Ker <worker@factory>
When the `SSLClient` is used to connect to a plain-HTTP server (which
can happen in clients due to some end-user misconfiguration) it can
return a failure from the `send()` call without setting the `Error`
reference to the corresponding error code. This can cause problems to
callers, that may expect that, when the check like this is passed on
the response:
```c++
if (res.error() == Error::Success)
```
then they can access the response contents with `res.value()`. When
`SSLClient`'s connection fails - the contents `unique_ptr` is not set
and an attemt to access it causes UB.
This change fixes the `SSLClient::create_and_connect_socket` method
making sure that, the `Error` value is set correctly when the
`is_valid()` check fails.
* Fix use of dangling references
When the resolve thread is detached, local variables were still used, which could lead to a program crash.
* Add test to verify dangling ref fix
* Add missing brace initialization
* Assert that the remaining fields are really zeroed
* Fix use of chrono literals
* Initialize start time for server
By initializing start_time_ for server, I hope to measure the time taken to process a request at the end maybe in the set_logger callback and print it.
I only see current usage in client with server retaining the inital min value
* Add test to verify start time is initialized
* Address review comments
* run clang format
* osx: fix inconsistent use of the macro `TARGET_OS_OSX`
Fixed the build error on iOS:
```
httplib.h:3583:3: error: unknown type name 'CFStringRef'
870 | CFStringRef hostname_ref = CFStringCreateWithCString(
```
Note, `TARGET_OS_OSX` is defined but is 0 when `TARGET_OS_IOS` is 1,
and vise versa. Hence, `TARGET_OS_MAC` should have been used, that is
set to 1 for the both targets.
* improve: non-blocking getaddrinfo() for all mac target variants
`TARGET_OS_MAC` should have been used, that is set to 1 for all other
targets: OSX, IPHONE (IOS, TV, WATCH, VISION, BRIDGE), SIMULATOR,
DRIVERKIT.
This is a follow-up to commit 4ff7a1c858,
which introduced new simplified build options and deprecated the old
ones. I forgot to also change the various get_option() calls,
effectively rendering the new option names useless, as they would not
get honoured.
On 32-bit Windows, meson setup fails with an unclear error:
meson.build:25:16: ERROR: Could not get define 'CPPHTTPLIB_VERSION'
The actual problem is that httplib.h #errors out.
Have the Meson logic explicitly check for a 32-bit host and warn or error,
matching the check in httplib.h. Phrase the Windows error in a way that
triggers WrapDB CI's unsupported architecture check.
The "cpp-httplib_" prefix of build options is now dropped, as Meson
build options are already namespaced for each project. The old names
remain as deprecated aliases for the new ones.
This new option automatically enables the new non-blocking name
resolution when the appropriate libraries are found, automatically
adding them to the list of required dependencies. It will gracefully
fall back to the old behaviour when no library is found.
This complements commit ea850cbfa7.
* Add proxy test on CI
* Add Brotli and Zstd dev packages to proxy test workflow
* Fix Docker Compose command for GitHub Actions compatibility
* Add proxy readiness check and netcat dependency
* Use netcat-openbsd instead of virtual netcat package
* Add proxy startup delay and debug logging
* Add test for multipart form data with boundary split
Add a test for multipart form data requests with a large header which
leads to a split inside the boundary because of the read buffer size
inside the SocketStream class.
* Fix bad request for multipart form data with boundary split
Fix a bad request (400) response for multipart form data requests with
a large header which leads to a split inside the boundary because of the
read buffer size inside the SocketStream class. The parse function
inside the MultipartFormDataParser wrongly erase the receive buffer if
it doesn't find the boundary inside the buffer during first call.
* Fix#2157
* Fix Windows build error: wrap std::max in parentheses to avoid macro conflict
- On Windows, max/min are often defined as macros by windows.h
- This causes compilation errors with std::max/std::min
- Solution: use (std::max) to prevent macro expansion
- Fixes CI build error: error C2589: '(': illegal token on right side of '::'
Fixes: error in coalesce_ranges function on line 5376
* Adds headers to multipart form data
Adds a `headers` field to the `MultipartFormData` struct.
Populates this field by parsing headers from the multipart form data.
This allows access to specific headers associated with each form data part.
* Adds multipart header access test
Verifies the correct retrieval of headers from multipart form data file parts.
Ensures that custom and content-related headers are accessible and parsed as expected.
* Enables automatic test discovery with GoogleTest
Uses `gtest_discover_tests` to automatically find and run tests, simplifying test maintenance and improving discoverability.
* Removes explicit GoogleTest include
* Refactors header parsing logic
Improves header parsing by using a dedicated parsing function,
resulting in cleaner and more robust code.
This change also adds error handling during header parsing,
returning an error and marking the request as invalid
if parsing fails.
* clang-format corrected
* Renames variable for better readability.
Renames the `customHeader` variable to `custom_header` for improved code readability and consistency.
* typo
* fix(parser): Limit line length in getline
Prevents potential infinite loop and memory exhaustion in
stream_line_reader::getline by enforcing max line length.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* fix: increase default max line length to 32k
LONG_QUERY_VALUE test is set at 25k.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* test(client): expect read error with too long query
Adds a test case (`TooLongQueryValue`) to verify client behavior
when the request URI is excessively long, exceeding
`CPPHTTPLIB_MAX_LINE_LENGTH`. In this scenario, the server is
expected to reset the connection.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
---------
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* Support zstd also via pkg-config
It doesn't always provide cmake config
* Find zstd with pkg-config also in non-required case
Code by @sum01, slightly modified
afunix.h uses types declared in winsock2.h, and so has to be included
after it. Including afunix.h first will result in a somewhat unhelpful
compilation error:
error C3646: 'sun_family': unknown override specifier
Signed-off-by: Piotr Stankiewicz <piotr.stankiewicz@docker.com>
* Add zstd support
* Add zstd to CI tests
* Use use zstd cmake target instead of ZSTD. Use cmake variable for found packages
* Add missing comment for HTTPLIB_REQUIRE_ZSTD
* Fix test.yaml rebase error
* Use zstd::libzstd target
* Add include and library paths to ZSTD args
* Run clang-format
* Add zstd to httplibConfig.cmake.in
By making the random engine thread_local, each thread now has its own
independent random sequence, ensuring safe concurrent access.
Additionally, using an immediately invoked lambda expression to
initialize the engine eliminates the need for separate static seed
variables.
* Wrap poll()/WSAPoll() in a function
Instead of using a macro for poll() on Windows, which breaks when the
implementation is compiled separately, add a detail::poll_wrapper()
function that dispatches to either ::poll() or ::WSAPoll().
* Build compiled library on Windows
Redirect tests fail using httpbin.org or nghttp2.org/httpbin. The
location header value contains a string representation of a Python byte
string (e.g., b'http://www.google.com/'), which results in a 404 error.
* Revert "Fix typo in meson.build (#2070)"
This reverts commit 5c0135fa5d.
* Revert "build(meson): automatically use poll or select as needed (#2067)"
This reverts commit 2b5d1eea8d.
* Revert "Make poll() the default (#2065)"
This reverts commit 6e73a63153.
* Remove select() and use poll()
- Replace is_readable() with wait_readable() and is_writable() with
wait_writable() in the Stream interface.
- Implement a new is_readable() function with semantics that more
closely reflect its name. It returns immediately whether data is
available for reading, without waiting.
- Update call sites of is_writable(), removing redundant checks.
* server_certificate_verifier extended to reuse built-in verifier
* code cleanup and SSLVerifierResponse enum clarification as per @falbrechtskirchinger comment
* cleanup
* clang-format
* change local var verification_status_ declaration to auto
* change local var verification_status_ to verification_status
* clang-format
* clang-format
---------
Co-authored-by: UrosG <uros@ub330.net>
Add include_httplib.cc to the main test executable (already done in
Makefile), and add include_windows_h.cc to the main test executable on
Windows to test if including windows.h conflicts with httplib.h.
* Add workflow_dispatch with Google Test filter
Add the workflow_dispatch trigger to the test.yaml workflow. Includes an
input for an optional Google Test filter pattern.
* Add OS selection to workflow_dispatch
* Fix wording
This CI workflow checks ABI compatibility between the pushed commit and
the latest tagged release, helping preventing accidental ABI breaks.
Helps with https://github.com/yhirose/cpp-httplib/issues/2043
Consider Content-Length and Transfer-Encoding headers when determining
whether to expect content. Don't handle the HTTP/2 connection preface
pseudo-method PRI.
Fixes#2028.
Modify for OpenVMS x86 C++. Make tests on OpenVMS currently not supported due to no cmake support.
Changes tested on OpenVMS clang C++ and Fedora & GCC
RFC-9110 '14.1.2. Byte Ranges':
A client can limit the number of bytes requested without knowing the
size of the selected representation. If the last-pos value is absent,
or if the value is greater than or equal to the current length of the
representation data, the byte range is interpreted as the remainder of
the representation (i.e., the server replaces the value of last-pos
with a value that is one less than the current length of the selected
representation).
https://www.rfc-editor.org/rfc/rfc9110.html#section-14.1.2-6
The testsuite checks for the exact size of the compressed content. The
exact size can change if the zlib library is using a different strategy.
In thise case using zlib-ng results in a slightly larger content leading
to a failure in the test.
Check that the compressed content is less than 10MiB which is a tenth of
the orignal content and proves that compression works.
Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
* build(meson): bump minimum version to 0.62.0
This allows making some minor cleanups
* test(meson): fix SSLClientServerTest.* tests with OpenSSL 3.2.0
Since OpenSSL commit
<https://github.com/openssl/openssl/commit/342e3652c791bdb06e08abcc169b4456c83ccd00>,
the default X.509 certificate format generated with the `openssl req`
command has been changed to X.509 v3 from X.509 v1.
For some reason, this change breaks cpp-httplib's SSLClientServerTest.*
tests.
To fix the test failures, this patch passes the '-x509v1' flag instead
of '-x509' when OpenSSL 3.2.0 or newer is detected. To detect the
version of a command line utility, Meson 0.62.0 or later is required.
Fixes <https://github.com/yhirose/cpp-httplib/issues/1798>, but only for
the Meson build system.
* Renamed enable_server_host_verification to enable_server_hostname_verification and added Error::SSLServerHostnameVerification
* Add some Open SSL function calls
* Code cleanup
* Fix#1431
This patch is necessary to build cpp-httplib in Crashpad, itself in
Chromium, using BoringSSL. Details at [1].
The fixes include:
- Library version check: tolerate BoringSSL as an alternative to
OpenSSL 3.
- Don’t call `OPENSSL_thread_stop`, which is not in BoringSSL.
- Use `SSL_get_peer_certificate` (deprecated in OpenSSL 3), the old
name for `SSL_get1_peer_certificate`, because the new name is not in
BoringSSL.
- Call `SSL_set_tlsext_host_name` directly instead of making an
`SSL_ctrl` call that BoringSSL does not support. The feared
-Wold-style-cast warning that occurs when buidling with OpenSSL is
not triggered in BoringSSL.
[1] https://chromium.googlesource.com/crashpad/crashpad/+/1a62a0182557c89494676c06611f1ca731bcb2db
The test used the hardcoded long value for 64 bit machines even on 32
bit ones, leading to test failures. With this patch the max long length
is obtained using std::numeric_limits<long>::max(). Thanks to q2a3z for
the hint!
Fixes: https://github.com/yhirose/cpp-httplib/issues/1795
* New function SSLServer::update_certs. Allows to update certificates while server is running
* New function SSLServer::update_certs. Added unit test
* avoid memory leaks if linked with static openssl libs
---------
Co-authored-by: CEU\schielke <Rainer.Schielke@heidelberg.com>
* New function SSLServer::update_certs. Allows to update certificates while server is running
* New function SSLServer::update_certs. Added unit test
---------
Co-authored-by: CEU\schielke <Rainer.Schielke@heidelberg.com>
Release builds result in the following warning because `content_length` param was used only inside asserts:
1> cpp-httplib\httplib.h(4933,45): warning C4100: 'content_length': unreferenced formal parameter
* Windows has WSAAccept() which will create sockets inheriting flags from
the server socket
* Linux has accept4() which has a flags argument supporting SOCK_CLOEXEC
* Implement string divider to replace splitter
* Divide query string in half
* Add a test case for query values containing the '=' character
* Add test cases for string divider
* Fix warnings
* Move httplibConfig.cmake.in to cmake dir
Just makes more sense to put it there I suppose.
* Cmake install README & License
Seems to make sense since you might already do this as a package
manager, or an end user might want them anyways.
The locations are just based on standard Linux locations using
GNUInstallDirs, so it should be sane on other machines too.
using lambda expression as Handler in set_error_handler will cause ambiguous.
Template forwarding can forward HandlerWithResponse to the correct overloading function
* Unimportant cleanup of FindBrotli
Some whitespace, removal of an unused var, and add a comment about
the missing version support
* Fix incorrect var in FindBrotli
Not much to say, it was just the wrong one. Not sure how that happened,
maybe a holdover from when I did an overhaul at some point..
* Tiny useless edit to FindBrotli
Just a little tweak to how I was postfixing those -static strings to the
lib names. Mostly pointless..
* Simplify some FindBrotli code
Nothing much, just reducing redundant stuff & a cleanup of a comment.
* Ignore PkgConf in FindBrotli if looking for static
As per the issue mentioned in this comment, static PkgConf support
is broken upstream. After testing, I found it'll just accept shared
even if you try to find static, so I'm merely disabling that feature
for this FindBrotli module.
That said, you can still get the static libs if you have them installed,
but PkgConf will be ignored even if found in favor of a regular
find_library call. Otherwise you'd end up with shared libs even if you
ask for static (with PkgConf installed).
TLDR: actually fail correctly when static libs aren't found when PkgConf
thought they were.
* Reorder cmake docs a bit
Just wanted to group the more related build options together.
Also removed a pointless reference to the old reasoning for the required
min ver since it's 3.14 now anyways.
* Fix outdated cmake comment
We don't use Git to find the version string anymore.
Just updated to match what it's actually used for now.
* Group options and build-tree vars in Cmake
Doesn't really change anything, I just wanted to clean these up a bit.
* Fix how we set HTTPLIB_IS_USING_XXX vars in Cmake
Prevents us acidentally using libs when the user didn't want them
actually used. This could happen if they set the option to OFF but their
own project itself is using the lib, thus we'd find and use it anyways.
Ref #1602 to see an example of this already happening before.
This is merely apply that kind of fix to all 3 of our deps, instead of
just OpenSSL.
* Minor formatting/comment change to Cmake
Pointless, but these things were bothering me..
For very busy servers, the internal jobs queue where accepted
sockets are enqueued can grow without limit.
This is a problem for two reasons:
- queueing too much work causes the server to respond with huge latency,
resulting in repetead timeouts on the clients; it is definitely
better to reject the connection early, so that the client
receives the backpressure signal as soon as the queue is
becoming too large
- the jobs list can eventually cause an out of memory condition
* Fix: Query parameter including query delimiter ('?') not being parsed properly
* Add details::split function with and without m argument to allow split parameters with/without counter
* Revert changes in SplitTest.ParseQueryString
* Fix readability warnings
Did not fix readbility-qualified-auto, will do a separate pull request
* Revert changes where meaning is lost
* Revert some style changes
The exact circumstances when this false positive is triggered are quite
tricky to reproduce, but it happened reproducibly with g++ 13.1 and 13.2 in
a close-source SW I'm working on. The fix even improves performance by a
very tiny bit: There is no need to copy the std::smatch, having a const
reference is enough.
Just as a side note: -Warray-bounds seems to cause trouble in other
projects, too, so e.g. the Linux kernel has disabled since June 2022.
* ClientImpl: Connection=close header control moved from process_request to send_
* Connection=close header control moved from send_ to handle_request
* SSLClient::connect_with_proxy error handling improved
* to_string definition added for Error::ProxyConnection
* Comment improvement
---------
Co-authored-by: akinelden <akin.elden@gmail.com>
* Fix successful decompress reported as Error::Read
Streams less than 4096 bytes are sometimes reported as failed reads because stream_.avail_in is not reduced to 0. The next iteration of the loop finds `prev_avail_in == strm_.avail_in` and return false. `ret = inflate(...)` returns Z_STREAM_END on the first iteration of the loop indicating that inflate is finished. This fix prevents the second iteration of the loop from failing.
* Fix successful decompress reported as Error::Read
- Add unit tests for raw deflate that illustrates the decompression failure when there are extra trailing bytes
GoogleTest, starting with vesion 1.13.0, requires C++14 to build. This
patch enables C++14 if GoogleTest 1.13.0 or newer is detected when
compiling the tests with Meson, making it possible to use new GTest
versions.
You can find GoogleTest's release notes at
<https://github.com/google/googletest/releases/tag/v1.13.0>.
We already detect OpenSSL in our tree for another project, but don't want to
link httplib against OpenSSL or with SSL support.
Allow us to `set(HTTPLIB_IS_USING_OPENSSL FALSE)`.
Removed multiple old-style (C) casts and replaced them with the
appropriate _cast's. This makes httplib a better citizen inside projects
that are compiled with all warnings enabled.
Unfortunately one warning remains as a result of invoking an
OpenSSL macro (at least on Linux), that would have to be fixed
upstream.
Also fixed a few casts for invocations of setsockopt.
setsockopt takes a const void* for the value pointer, not a char*.
Well, except on Windows ...
Co-authored-by: Andre Eisenbach <git@4ae.us>
This allows disabling the use of C++ exceptions at CMake configure time.
The value is encoded in the generated httplibTargets.cmake file and will
be used by CMake projects that import it.
* Explicitly #include <utility> for use of std::move
* Move not copy Logger arg from Client to ClientImpl
* Move not copy, set_error_handler Handler to lambda
* Remove null statement in non-empty if/else block
I guess it was a relic from a time before the other statement was added.
---------
Co-authored-by: Daniel Boles <daniel.boles@voltalis.com>
* Reuse gtest on the system
Try to use gtest on the system if found. Avoid using recent CMake
features.
* Set CMP0135 only when using FetchContent
* Add /bigobj option for MSVC
* Support also cmake between 3.14 and 3.20
Older versions provided only GTest::Main target, not currently used
gtest_main. Provide backward compatibility also to old cmake versions.
* Remove redundant variable checking
---------
Co-authored-by: Jiwoo Park <jiwoo_90@naver.com>
* Support loading system certs from Keychein on MacOS
* review improvements: add deps to meson.build and improve conditional expressions in cmake
* fix tabs
* fix tabs
* review improvements
* fix after review
* additionally load root certs from the system root keychain
* cmake fix
* fix
* small refactoring
* small refactoring
---------
Co-authored-by: Sergey Kazmin <sergey.kazmin@kaspersky.com>
* Add test/CMakeLists.txt to enable testing with CTest
Add HTTPLIB_TEST option
Downlaod GoogleTest source using FetchContent module
Generate cert files to test httplib with OpenSSL
* Generate cert2.pem with a new certificate request
* Use the latest GoogleTest library
* add support for requests with both MultipartFormDataItems and ContentProviders
* rework implementation
* use const auto & and fix offset calculation
* fix zero items
* snake case variables
* clang-format
* commonize get_multipart_content_provider, add Put() with MultipartFormDataProviderItems
* fix linker multiple definition error
* add test MultipartFormDataTest.DataProviderItems
* Add EINTR and EAGAIN judge for accept
* Add EINTR signal tests
* Cancel win32 and win64 compile on signal unittest
Co-authored-by: yukun.yu <yukun.yu@alibaba-inc.com>
* Move next job in task queue rather than copy
* Notify waiting thread after unlocking mutex
* Add unit test for TaskQueue
* Don't use C++14 feature in test code
* handle socket options for UNIX socket same as others
* set FD_CLOEXEC by default
* invoke `socket_options` callback if set
* Offer Client info even on UNIX socket based Server
HTTP Request header "REMOTE_PORT" contains client process id if possible
when Server works on UNIX socket.
* retrigger checks
* retrigger checks
* add support macOS
Co-authored-by: Changbin Park <changbin.park@ahnlab.com>
* SocketStream need to check connectivity for writability.
When the stream is used for SSE server which works via chunked content
provider it's not possible to check connectivity over writability
because it's wrapped by DataSink. It could make the server stalled, when
the provider wants to only keep connection without send data and certain
amount of clients consumes entire threadpool.
* add unittest for SocketStream::is_writable()
SocketStream::is_writable() should return false if peer is disconnected.
* revise broken unittest ServerTest.ClientStop
DataSink could be unwritable if it's backed by connection based. Because
it could be disconnected by client right after enter centent provider.
Co-authored-by: Changbin Park <changbin.park@ahnlab.com>
Release 0.11 broke backwards compatibility, meaning that different
cpp-httplib versions are compatible with each other only if the major
and minor version numbers are the same.
This patch reflects this in the build systems.
See #1209 for some more context.
* httplib.h
add multipart formdata for PUT in addition to POST as some REST
APIs use that.
Factor the boundary checking code into a helper and use it from
both Post() and Put().
* test/test.cc
add test cases for the above.
* Add support UNIX domain socket
* `set_address_family(AF_UNIX)` is required
* add unittest for UNIX domain socket
* add support UNIX domain socket with abstract address
Abstract address of AF_UNIX begins with null(0x00) which can't be
delivered via .c_str() method.
* add unittest for UNIX domain socket with abstract address
Co-authored-by: Changbin Park <changbin.park@ahnlab.com>
* Add get_socket_fd method to Client and ClientImpl, add according unit test
* Change name get_socket_fd to get_socket
* Change name get_socket to socket
Co-authored-by: ata.yardimci <ata.yardimci@erstream.com>
* ssize_t redefinition on Windows - int/int64 vs long/long long
* Define ssize_t as __int64 for _WIN64, not long long
Co-authored-by: iamttaM <9880090+oculusbytes@users.noreply.github.com>
* resolve problem: http server can't send file large than 2GB.
add unit test for http server send large file.
add /bigobj compile option to msvc x64.
* disable unit test "ServerLargeContentTest" due to out-of-memory on GitHub Actions.
- Enable THREADS_PREFER_PTHREAD_FLAG to use -pthread where supported
- Remove low-level compile features (closes#1272)
- Remove unneeded DESTINATION options where possible
Thanks to abf3a67dd0 the use of python3
isn't required anymore to configure the build, so I moved the
find_program('python3') inside the "if compile" block.
This makes it possible to configure cpp-httplib on systems where python
isn't available with tools like muon: https://sr.ht/~lattis/muon/
* Add large data transfer test
* Replace `SSL_read` and `SSL_write` with `ex` functions
* Reflect review comment
* Fix return value of `SSLSocketStream::read/write`
* Fix return value in the case of `SSL_ERROR_ZERO_RETURN`
* Disable `LargeDataTransfer` test due to OoM in CI
Meson only runs required targets. The key_encrypted_pem and
cert_encrypted_pem targets added in 020b0db090
and 8191fd8e6c weren't added to the list
of targets required by the test target, so the generation of the
encrypted certs was skipped, resulting in the failure of
BindServerTest.BindAndListenSeparatelySSLEncryptedKey.
* meson: fix regression that broke extracting version
In commit 33f67386fe the code that
heuristically parsed the version broke due to the version being moved
around into a more easily accessible define.
While we are at it, pass the exact path of httplib.h to un-break usage
as a meson subproject. This was broken in commit
8ecdb11979 which checked the return code
of trying to get the version; it was always broken, but formerly failed
in silence and resulted in no version number.
* meson: use the compiler builtins to extract the version from the header
As a convenient string define, it is now possible to ask the
preprocessor what the version of cpp-httplib is. This can be used from
meson too, in order to avoid encoding C++ file structure into python
regexes.
A pkg-config file was previously installed only if cpp-httplib was being
built as a compiled library.
Since architecture-independent .pc files
can exist in /usr/share/pkgconfig, it can be useful to install one even
when installing the header-only version (for example, it could be used
by third party projects to easily find out if cpp-httplib is installed
and its version, using something like Meson's `dependency()` or CMake's
`pkg_check_modules()`).
The change makes the Meson build behave a bit more like the CMake one,
as it also always installs a CMake Config file, but here the pkg-config
file gets installed to the correct architecture-independent directory
(`datadir` represents /usr/share on Linux and simiar systems).
Lastly, I made some minor cleanups.
This change is based on RFC7230, § 3.5 'Message Parsing Robustness': "Although the line terminator for the start-line and header fields is the sequence CRLF, a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR."
* Fix#1041
* Fixed problem with is_socket_alive
* Adjust the way to check if the sockt is still alive.
* Revert "Adjust the way to check if the sockt is still alive."
This reverts commit 6c673b21e5.
* Adjust is_socket_alive according to the code review
* Allow to specify server IP address
* Reimplement in set_hostname_addr_map
* Add tests for set_hostname_addr_map
* Fix tests after implement set_hostname_addr_map
* SpecifyServerIPAddressTest.RealHostname typo
This integrates the "main" test suite (test/test.cc) in Meson.
This allows to run the tests in the CI with the Meson-built version of
the library to ensure that nothing breaks unexpectedly.
It also simplifies life of downstream packagers, that do not have to
write a custom build script to split the library and run tests but can
instead just let Meson do that for them.
* Full Meson support
cpp-httplib can be now built with Meson even in compiled library mode.
The library is built with LTO, supports OpenSSL, zlib and Brotli,
and the build system also generates a pkg-config file when needed.
Compared to the CMake file this one is quite small (more than five times
smaller!), and maintaining it won't be an issue :)
* meson: automatic versioning
In order to test the split version (.h + .cc via split.py):
- Added a test_split program in the test directory whose main purpose is
to verify that it works to compile and link the test case code against
the split httplib.h version.
- Moved types needed for test cases to the “header part” of httplib.h.
Also added forward declarations of functions needed by test cases.
- Added an include_httplib.cc file which is linked together with test.cc
to verify that inline keywords have not been forgotten.
The changes to httplib.h just move code around (or add forward
declarations), with one exception: detail::split and
detail::process_client_socket have been converted to non-template
functions (taking an std::function instead of using a type parameter for
the function) and forward-declared instead. This avoids having to move
the templates to the “header part”.
When using the split version of httplib.h the templated implementation
of e.g. Client::set_connection_timeout ends up in httplib.cc and
therefore results in a linker error since the needed template
specialization has not been instantiated. Fix this by moving the
implementation of template methods into the part that ends up in
httplib.h after the split.
Fixes#1008.
The RedirectToDifferentPort.Redirect test assumes that port 8080 and
8081 are available on localhost. They aren’t on my system so the test
fails. Improve this by binding to available ports instead of hardcoded
ones.
- Added shebang and made the script executable.
- Added help text.
- Added -o/--out argument for specifying output directory.
- Added -e/--extension argument for specifying file extension of the
implementation file.
- Made the script find httplib.h next to split.py instead of the current
working directory. This makes it possible to call the script from
another directory.
- Simplified code structure slightly.
- Improved variable naming to follow Python conventions.
* Add header-only Meson support
This allows users to call `dependency('httplib')` and have the include
directory automatically configured
* Rename `httplib` to `cpp-httplib`
* Fix virtual call in ClientImpl::~ClientImpl()
This fixes a warning in clang tidy:
> Call to virtual method 'ClientImpl::shutdown_ssl' during
> destruction bypasses virtual dispatch
ClientImpl::~ClientImpl() calls lock_socket_and_shutdown_and_close()
that itself calls shutdown_ssl(). However, shutdown_ssl() is virtual
and C++ does not perform virtual dispatch in destructors, which results
in the wrong overload being called.
This change adds a non-virtual shutdown_ssl_impl() function that is
called from ~SSLClient(). We also inline sock_socket_and_shutdown_and_close()
and removes the virtual call in ~ClientImpl().
* Inline and remove lock_socket_and_shutdown_and_close()
The function only has one caller.
* Fix client.cc code, since res.error() without operator overloading causing error in Xcode
* Add unit test to check new error to string with operator overloading
* Add inline as requested in code review comment
* Special function to encode query params
* Fix #include <iomanip>
* Added unescaped charsets to encode_query_param
* Unit tests for encode_query_param
Add unit test for issue #682 fixed in PR #728, which does not contain
the test of its own.
The test creates a fake SSL server, inherited from SSLServer, which
does not create an SSL context. When an SSL client attempts to send it
a request, it gets a timeout error. Prior to PR #728, the client would
wait indefinitely
Co-authored-by: Michael Tseitlin <michael.tseitlin@concertio.com>
* ssl-verify-host: fix verifying ip addresses containing zero's
If the subject alternate name contained an ip address with an zero
(like 10.42.0.1) it could not successfully verify.
It is because in c++ strings are null-terminated
and therefore strlen(name) would return a wrong result.
As I can not see why we can not trust the length returned by openssl,
lets drop this check.
* ssl-verify-host: add test case
lets try to validate against 127.0.0.1
Co-authored-by: Daniel Ottiger <daniel.ottiger@ch.schindler.com>
SSL connection is performed in two steps:
First, a regular socket connection is established.
Then, SSL_connect/SSL_accept is called to establish SSL handshake.
If a network problem occurs during the second stage, SSL_connect on
the client may hang indefinitely.
The non-blocking mode solves this problem.
Co-authored-by: Michael Tseitlin <michael.tseitlin@concertio.com>
* *Add server fuzzer target and seed corpus
* Add fuzz_test option to Makefile
* Fix#685
* Try to fix Github actions on Ubuntu
* Added ReadTimeoutSSL test
* Comment out `-fsanitize=address`
* Rebase upstream changes
* remove address sanitizer temporarily
* Add separate Makefile for fuzzing
* 1. Remove special char from dictionary
2. Clean fuzzing/Makefile
* Use specific path to avoid accidently linking openssl version brought in by oss-fuzz
* remove addition of flags
* Refactor Makefile
* Add missing newline
* Add fuzztest to github workflow
* Fix
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* Backport std::make_unique from C++14.
* Replace shared_ptr with unique_ptr for better performance.
Co-authored-by: Ella <maxutong16@otcaix.iscas.ac.cn>
* Use move semantics instead of copy for functions
In some cases, a few more copies could be prevented by changing function definitions to accept parameters by const-ref, rather than by value, but I didn't want to change public signatures.
* Fix two use-after-move errors
* Add `cache_control` parameter to `set_mount_point`
Specifies the Cache-Control header value to return when specified. For example:
```
svr.set_mount_point("/assets", "public/assets", "public, max-age=604800, immutable");
```
* Add default for cache_control
Default to "no-cache", which is implicitly what is happening today.
* Change set_mount_point to accept Headers
* Don't use C++17 destructuring
* Fix memory leak due caused due to X509_STORE
* Add test for repro and address sanitizer to compiler flags
* Add comment
* Sync
* Associate ca_store with ssl context within set_ca_cert_store()
* Split SlowPost test
* Fix#674
Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* Fix parsing to parse query string with single space char.
When passed ' ' as a query string, the server crashes cause of illegal memory access done in httplib::detail::split. Have added checks to make sure the split function has a valid string with length > 0.
* Fix parsing to parse query string with single space char.
* Fix server crash caused due to regex complexity while matching headers.
While parsing content-type header in multipart form request the server crashes due to the exhaustion of max iterations performed while matching the input string with content-type regex.
Have removed the regex which might use backtracking while matching and replaced it with manual string processing. Have added tests as well.
* Remove magic number
Co-authored-by: Ivan Fefer <fefer.ivan@gmail.com>
Co-authored-by: yhirose <yhirose@users.noreply.github.com>
Co-authored-by: Ivan Fefer <fefer.ivan@gmail.com>
* Fix parsing to parse query string with single space char.
When passed ' ' as a query string, the server crashes cause of illegal memory access done in httplib::detail::split. Have added checks to make sure the split function has a valid string with length > 0.
* Fix parsing to parse query string with single space char.
Correct the working directory of the execute_process call, to get the version from git tags.
The working directory should be the the directory of this libary. Otherwise, if you include httplib
as a git submodule and call add_subdirectory, the execute_process command will be run in a wrong
directory leading to error.
If you didn't have Git installed, execute_process never declared the
error variable, which led to it never parsing the header for a version.
So if Git wasn't installed, the version variable never got declared,
which caused errors.
This fixes that.
* Update httplib.h
When you disconnect and reconnect from the network, your network stack rewrites and updates /etc/resolv.conf accordingly. This configuration file is needed by the DNS resolver in the C library. The C library reads the DNS configuration from /etc/resolv.conf the first time, and caches it. It doesn't check, with every lookup, if the contents of /etc/resolv.conf have changed.
the solution is to add a call to res_init(), defined in resolv.h
* Update httplib.h
Had to create a custom FindBrotli package, as not all users have
PkgConfig installed (which Brotli uses). This file gets installed
alongside httplibConfig.cmake for the end-users convenience.
Set BROTLI_USE_STATIC_LIBS to ON if you want to find the static libs
instead of default shared.
Adds the HTTPLIB_REQUIRE_BROTLI (default off) and HTTPLIB_USE_BROTLI_IF_AVAILABLE
(default on) options, which work in the same manner as the other optional/required
dependency options.
Moved the scattered linking and definitions to a single call.
Updated some documentation about the new options.
Improved the in-tree support by setting the HTTPLIB_IS_USING_XYZ
variables in the main CMakeLists (as well as having them in the
httplibConfig.cmake file).
Fixes#582
Seems certain targets/hosts failed without these, as "_MSC_VER" is
undefined on MinGW, which caused the 'pragma comment(lib "libname")' to
fail.
Fixes#575
This gets us the full version (aka with the patch version), instead of
just major and minor version from user agent.
Falls back to the user agent if it fails.
It pulls the version from the user-agent string in the header, so it
will not need to be manually adjusted. This version file is installed so
that you can check for a specific version with find_package(httplib)
Also added HTTPLIB_INCLUDE_DIR (root path without header name), and
HTTPLIB_LIBRARY (only if compiled).
Added HTTPLIB_VERSION, and HTTPLIB_FOUND (although it's recommended
to check if the target exists).
Updated CMakeLists documentation for all this.
This option (default OFF) automatically splits the file (with split.py)
into a header & source file, then compiles it as a shared/static
library. This requires an installed Python v3 executable to work.
This also adds a HTTPLIB_IS_COMPILED boolean that's available after a
finfind_package(httplib) call.
Note that the minimum Cmake version increased to 3.12 because of FindPython3.
Hopefully this isn't a problem, as it's already 3 years old at this point.
* Revert "Removed CMakeLists.txt. (Fix#421)"
This reverts commit 8674555b88.
* Fail if cmake version too old
Previous behaviour is just a warning.
* Improve CMakeLists
Adds automatic dependency finding (if they were used).
Adds a way to require a specific version in the find_package(httplib) call.
You should link against the httplib::httplib IMPORTED target, which is
created automatically.
Add options to allow for strictly requiring OpenSSL/ZLIB
HTTPLIB_REQUIRE_OPENSSL & HTTPLIB_REQUIRE_ZLIB require the libs be found, or the build fails.
HTTPLIB_USE_OPENSSL_IF_AVAILABLE & HTTPLIB_USE_ZLIB_IF_AVAILABLE silently search for the libs.
If they aren't found, the build still continues, but shuts off support for those features.
* Add documentation to CMakeLists.txt
Has info on all the available options and what targets are produced.
Also put some things about installation on certain platforms.
In case we want to send a lot of data,
and the receiver is slower than the sender.
This will first fill up the receivers queues and after this
eventually also the senders queues,
until the socket is temporarily unable to accept more data to send.
select_write is done with an timeout of zero,
which makes the select call used always return immediately:
(see http://man7.org/linux/man-pages/man2/select.2.html)
This means that every marginal unavailability will make it return false
for is_writable and therefore httplib will immediately abort the transfer.
Therefore make this values configurable in the same way
as the read timeout already is.
Set the default write timeout to 5 seconds,
the same default value used for the read timeout.
According to RFC 3493 the socket option IPV6_V6ONLY
should be off by default, see
https://tools.ietf.org/html/rfc3493#page-22 (chapter 5.3).
However this does not seem to be the case on all systems.
For instance on any Windows OS, the option is on by default.
Therefore clear this option in order to allow
an server socket which can support IPv6 and IPv4 at the same time.
I use a custom TaskQueue, with variable number of workers, adding workers on demand is an easy task when new connection arrive (in enqueue function) however i need another funtion to be called even (or better) went no new connections arrives to reduce workers count. I only added a new virtual method in TaskQueue class to allow custom class to adjust workers size over time. Even if this methods is called frequenlty custom class can keep a "last_update" counter to check if need to adjust worker count or any other internal task. Without this function i need an external thread to make this adjust task.
* Fixed error:
ULONG_MAX is defined in the limits.h header file. Putting #include <climits>
```
httplib.h: In function ‘bool httplib::detail::read_content_chunked(httplib::Stream&, httplib::ContentReceiver)’:
httplib.h:1921:22: error: ‘ULONG_MAX’ was not declared in this scope
if (chunk_len == ULONG_MAX) { return false; }
^~~~~~~~~
httplib.h:1921:22: note: suggested alternative: ‘_SC_ULONG_MAX’
if (chunk_len == ULONG_MAX) { return false; }
^~~~~~~~~
_SC_ULONG_MAX
```
* Move #include <climits> to after #include <cassert>
We cannot trivially support such large chunks, and the maximum value
std::strtoul can parse accurately is ULONG_MAX-1. Error out early if the
length is longer than that.
detail::read_content_chunked was using std::stoul to parse the
hexadecimal chunk lengths for "Transfer-Encoding: chunked" requests.
This throws an exception if the string does not begin with any valid
digits. read_content_chunked is not called in the context of a try block
so this caused the process to terminate.
Rather than use exceptions, I opted for std::stroul, which is similar to
std::stoul but does not throw exceptions. Since malformed user input is
not particularly exceptional, and some projects are compiled without
exception support, this approach seems both more portable and more
correct.
* SSLServer: add constructor to pass ssl-certificates and key from memory
* SSLClient: add constructor to pass ssl-certificates and key from memory
* add TestCase for passing certificates from memory to SSLClient/SSLServer
Previously, calling set_content always resulted in 's' being copied. With this change, there will still be the same number of copies made (1) when doing `set_content(my_value, ...)`, but there will be no copies if a caller elects to do `set_content(std::move(value), ...)` or `set_content(some_function_that_returns_a_temporary(), ...)` instead.
The regex that parses header lines potentially causes an unlimited
amount of backtracking, which can cause an exception in the libc++ regex
engine.
The exception that occurs looks like this and is identical to the
message of the exception fixed in
https://github.com/yhirose/cpp-httplib/pull/280:
libc++abi.dylib: terminating with uncaught exception of type
std::__1::regex_error: The complexity of an attempted match
against a regular expression exceeded a pre-set level.
This commit eliminates the problematic backtracking.
for /f "usebackq tokens=*" %%i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -property installationPath`) do set VSDIR=%%i
call "%VSDIR%\VC\Auxiliary\Build\vcvarsall.bat" x86 || exit /b 1
cd test && make test_split EXTRA_CXXFLAGS="-fno-exceptions -DCPPHTTPLIB_NO_EXCEPTIONS" && make test_openssl_parallel EXTRA_CXXFLAGS="-fno-exceptions -DCPPHTTPLIB_NO_EXCEPTIONS"
This document describes the streaming extensions for cpp-httplib, providing an iterator-style API for handling HTTP responses incrementally with **true socket-level streaming**.
> **Important Notes**:
>
> - **No Keep-Alive**: Each `stream::Get()` call uses a dedicated connection that is closed after the response is fully read. For connection reuse, use `Client::Get()`.
> - **Single iteration only**: The `next()` method can only iterate through the body once.
> - **Result is not thread-safe**: While `stream::Get()` can be called from multiple threads simultaneously, the returned `stream::Result` must be used from a single thread only.
## Overview
The streaming API allows you to process HTTP response bodies chunk by chunk using an iterator-style pattern. Data is read directly from the network socket, enabling low-memory processing of large responses. This is particularly useful for:
| SSE with auto-reconnect | `SSEClient` (see [README-sse.md](README-sse.md)) |
| LLM streaming (JSON Lines) | `stream::Get()` |
| Large file download | `stream::Get()` or `open_stream()` |
| Reverse proxy | `open_stream()` |
| Small responses with Keep-Alive | `Client::Get()` |
## API Reference
### Low-Level API: `StreamHandle`
The `StreamHandle` struct provides direct control over streaming responses. It takes ownership of the socket connection and reads data directly from the network.
> **Note:** When using `open_stream()`, the connection is dedicated to streaming and **Keep-Alive is not supported**. For Keep-Alive connections, use `client.Get()` instead.
A simple, blocking WebSocket implementation for C++11.
> [!IMPORTANT]
> This is a blocking I/O WebSocket implementation using a thread-per-connection model (plus one heartbeat thread per connection). It is intended for small- to mid-scale workloads; handling large numbers of simultaneous WebSocket connections is outside the design target of this library. If you need high-concurrency WebSocket support with non-blocking/async I/O (e.g., thousands of simultaneous connections), this is not the one that you want.
> [!NOTE]
> WebSocket extensions (`permessage-deflate` and others defined by RFC 6455) are **not supported**. If a client proposes an extension via `Sec-WebSocket-Extensions`, the server silently declines it — the negotiated connection always runs without extensions.
## Features
- **RFC 6455 compliant**: Full WebSocket protocol support (extensions are not implemented)
- **Server and Client**: Both sides included
- **SSL/TLS support**: `wss://` scheme for secure connections
- **Text and Binary**: Both message types supported
Timeout=3,// Read timeout elapsed; connection still open
};
```
Returned by `read()`. Since `Fail` is `0`, the result works naturally in boolean contexts — `while (ws.read(msg))` continues until the connection closes. When you need to distinguish text from binary, check the return value directly.
`Timeout` is only returned for a read timeout you set yourself with `set_read_timeout()`. It means the timeout elapsed on a message boundary: nothing was consumed and the connection is still open, so you can send on it and read again. The compile-time defaults (`CPPHTTPLIB_WEBSOCKET_SERVER_READ_TIMEOUT_SECOND`, 300 seconds on the server; a client waits forever) are a backstop against a peer that has gone quiet, not a request for control: when one of them elapses, `read()` returns `Fail` and closes the connection, so code that never calls `set_read_timeout()` can keep using `while (ws.read(msg))`.
**`msg` is left untouched on `Timeout`.** Because `Timeout` is non-zero, `while (ws.read(msg))` keeps looping — with the *previous* message still in `msg`. Once you set a read timeout, test the result instead:
The `SubProtocolSelector` receives the list of subprotocols proposed by the client (from the `Sec-WebSocket-Protocol` header) and returns the selected one. Return an empty string to decline all proposed subprotocols. A returned value that the client did not propose is ignored.
### WebSocket (Server-side)
Passed to the handler registered with `Server::WebSocket()`. The handler runs in a dedicated thread per connection.
```cpp
// Read next message (blocks until received, returns Fail/Text/Binary)
The check above runs after the handshake, so the client sees a successful upgrade followed by a close frame. To refuse the upgrade itself with an HTTP status, use a pre-routing or pre-request handler. Both run before the `101 Switching Protocols` response, and `req.matched_route` is available in the pre-request handler:
The server can negotiate a subprotocol with the client using `Sec-WebSocket-Protocol`. This is required for protocols like GraphQL over WebSocket (`graphql-ws`) and MQTT.
```cpp
// Server: register a handler with a subprotocol selector
By default the heartbeat only sends pings — it does not enforce that pongs come back. To detect a silently dropped connection faster, enable the max-missed-pongs check. Once `max_missed_pongs` consecutive pings go unanswered, the heartbeat thread closes the connection with `CloseStatus::GoingAway` and the reason `"pong timeout"`.
```cpp
ws.set_websocket_max_missed_pongs(2);// close after 2 consecutive unacked pings
```
The server side has the same `set_websocket_max_missed_pongs()`.
With the default ping interval of 30 seconds, `max_missed_pongs = 2` detects a dead peer within ~60 seconds. The counter is reset every time a Pong frame is received, so the mechanism only works when your code is actively calling `read()` — exactly the pattern a normal WebSocket client already uses.
**The default is `0`**, which means "never close the connection because of missing pongs." Pings are still sent on the heartbeat interval, but their responses are not checked. On the server side a dead connection still does not linger: while a handler is inside `read()`, `CPPHTTPLIB_WEBSOCKET_SERVER_READ_TIMEOUT_SECOND` (default **300 seconds = 5 minutes**) acts as a backstop. A client has no such backstop — it waits forever unless you set a read timeout — so there `max_missed_pongs` is what notices an unresponsive peer at all. On either side it is also the knob for noticing one *faster* than the 5-minute fallback.
## Threading Model
WebSocket connections share the same thread pool as HTTP requests. Each WebSocket connection occupies one thread for its entire lifetime.
The default thread pool uses dynamic scaling: it maintains a base thread count of `CPPHTTPLIB_THREAD_POOL_COUNT` (8 or `std::thread::hardware_concurrency() - 1`, whichever is greater) and can scale up to 4x that count under load (`CPPHTTPLIB_THREAD_POOL_MAX_COUNT`). When all base threads are busy, temporary threads are spawned automatically up to the maximum. These dynamic threads exit after an idle timeout (`CPPHTTPLIB_THREAD_POOL_IDLE_TIMEOUT`, default 3 seconds).
This dynamic scaling helps accommodate WebSocket connections alongside HTTP requests. However, if you expect many simultaneous WebSocket connections, you should configure the thread pool accordingly:
Choose sizes that account for both your expected HTTP load and the maximum number of simultaneous WebSocket connections.
### Calling from Multiple Threads
A single `WebSocket` (server-side) or `WebSocketClient` handle is shared by three potential callers: the thread running your handler (or holding the client), the heartbeat thread, and, if your code does its own thing, a separate thread calling `send()`/`close()` while another thread is blocked in `read()`.
**Supported**: calling `read()` from one thread while calling `send()`/`close()` from another. This is the common pattern for a client that reads incoming messages in a loop on one thread and sends from elsewhere (e.g. a UI thread). A message that is in flight when `close()` is called still arrives intact; `close()` sends the Close frame and returns, leaving the connection's read side to the thread that owns it, so it does not block waiting for the peer's Close reply in that case. The heartbeat thread's automatic pings use the same `send()` path internally, so they are safe to run concurrently with your `read()` loop too — for `wss://` this requires every TLS call on a connection to be serialized internally, which cpp-httplib does for you.
**Not supported**: calling `read()` from two threads at the same time on the same handle. The calls are serialized rather than left to corrupt each other, but which thread receives which message is unspecified, so there is nothing useful to build on it.
## Protocol
The implementation follows [RFC 6455](https://tools.ietf.org/html/rfc6455):
- Handshake via HTTP Upgrade with `Sec-WebSocket-Key` / `Sec-WebSocket-Accept`
- Subprotocol negotiation via `Sec-WebSocket-Protocol`
- Frame masking (client-to-server)
- Control frames: Close, Ping, Pong
- Message fragmentation and reassembly
- Close handshake with status codes
## Browser Test
Run the echo server example and open `http://localhost:8080` in a browser:
title:"C01. Get the Response Body / Save to a File"
order:1
status:"draft"
---
## Get it as a string
```cpp
httplib::Clientcli("http://localhost:8080");
autores=cli.Get("/hello");
if(res&&res->status==200){
std::cout<<res->body<<std::endl;
}
```
`res->body` is a `std::string`, ready to use as-is. The entire response is loaded into memory.
> **Warning:** If you fetch a large file with `res->body`, it all goes into memory. For large downloads, use a `ContentReceiver` as shown below.
## Save to a file
```cpp
httplib::Clientcli("http://localhost:8080");
std::ofstreamofs("output.bin",std::ios::binary);
if(!ofs){
std::cerr<<"Failed to open file"<<std::endl;
return1;
}
autores=cli.Get("/large-file",
[&](constchar*data,size_tlen){
ofs.write(data,len);
returnstatic_cast<bool>(ofs);
});
```
With a `ContentReceiver`, data arrives in chunks. You can write each chunk straight to disk without buffering the whole body in memory — perfect for large file downloads.
Return `false` from the callback to abort the download. In the example above, if writing to `ofs` fails, the download stops automatically.
> **Detail:** Want to check response headers like Content-Length before downloading? Combine a `ResponseHandler` with a `ContentReceiver`.
>
> ```cpp
> auto res = cli.Get("/large-file",
> [](const httplib::Response &res) {
> auto len = res.get_header_value("Content-Length");
> std::cout << "Size: " << len << std::endl;
> return true; // return false to skip the download
> },
> [&](const char *data, size_t len) {
> ofs.write(data, len);
> return static_cast<bool>(ofs);
> });
> ```
>
> The `ResponseHandler` is called after headers arrive but before the body. Return `false` to skip the download entirely.
> To show download progress, see [C11. Use the progress callback](../c11-progress-callback).
cpp-httplib doesn't include a JSON parser. Use a library like [nlohmann/json](https://github.com/nlohmann/json) to build and parse JSON. The examples here use `nlohmann/json`.
Pass the JSON string as the second argument to `Post()` and the Content-Type as the third. The same pattern works with `Put()` and `Patch()`.
> **Warning:** If you omit the Content-Type (the third argument), the server may not recognize the body as JSON. Always specify `"application/json"`.
## Receive a JSON response
```cpp
autores=cli.Get("/api/users/1");
if(res&&res->status==200){
autoj=nlohmann::json::parse(res->body);
std::cout<<j["name"]<<std::endl;
}
```
`res->body` is a `std::string`, so you can pass it straight to your JSON library.
> **Note:** Servers sometimes return HTML on errors. Check the status code before parsing to be safe. Some APIs also require an `Accept: application/json` header. If you're calling a JSON API repeatedly, [C03. Set default headers](../c03-default-headers) can save you some boilerplate.
> For how to receive and return JSON on the server side, see [S02. Receive JSON requests and return JSON responses](../s02-json-api).
When you want the same headers on every request, use `set_default_headers()`. Once set, they're attached automatically to every request sent from that client.
## Basic usage
```cpp
httplib::Clientcli("https://api.example.com");
cli.set_default_headers({
{"Accept","application/json"},
{"User-Agent","my-app/1.0"},
});
autores=cli.Get("/users");
```
Register the headers you need on every API call — like `Accept` or `User-Agent` — in one place. No need to repeat them on each request.
## Send a Bearer token on every request
```cpp
httplib::Clientcli("https://api.example.com");
cli.set_default_headers({
{"Authorization","Bearer "+token},
{"Accept","application/json"},
});
autores1=cli.Get("/me");
autores2=cli.Get("/projects");
```
Set the auth token once, and every subsequent request carries it. Handy when you're writing an API client that hits multiple endpoints.
> **Note:** `set_default_headers()` **replaces** the existing default headers. Even if you only want to add one, pass the full set again.
## Combine with per-request headers
You can still pass extra headers on individual requests, even with defaults set.
```cpp
httplib::Headersheaders={
{"X-Request-ID","abc-123"},
};
autores=cli.Get("/users",headers);
```
Per-request headers are **added** on top of the defaults. Both are sent to the server.
> For details on Bearer token auth, see [C06. Call an API with a Bearer token](../c06-bearer-token).
By default, cpp-httplib does not follow HTTP redirects (3xx). If the server returns `302 Found`, you'll get it as a response with status code 302 — nothing more.
To follow redirects automatically, call `set_follow_location(true)`.
## Follow redirects
```cpp
httplib::Clientcli("http://example.com");
cli.set_follow_location(true);
autores=cli.Get("/old-path");
if(res&&res->status==200){
std::cout<<res->body<<std::endl;
}
```
With `set_follow_location(true)`, the client reads the `Location` header and reissues the request to the new URL automatically. The final response ends up in `res`.
## Redirects from HTTP to HTTPS
```cpp
httplib::Clientcli("http://example.com");
cli.set_follow_location(true);
autores=cli.Get("/");
```
Many sites redirect HTTP traffic to HTTPS. With `set_follow_location(true)` on, this case is handled transparently — the client follows redirects even when the scheme or host changes.
> **Warning:** To follow redirects to HTTPS, you need to build cpp-httplib with OpenSSL (or another TLS backend). Without TLS support, redirects to HTTPS will fail.
> **Note:** Following redirects adds to the total request time. See [C12. Set timeouts](../c12-timeouts) for timeout configuration.
For endpoints that require Basic authentication, pass the username and password to `set_basic_auth()`. cpp-httplib builds the `Authorization: Basic ...` header for you.
## Basic usage
```cpp
httplib::Clientcli("https://api.example.com");
cli.set_basic_auth("alice","s3cret");
autores=cli.Get("/private");
if(res&&res->status==200){
std::cout<<res->body<<std::endl;
}
```
Set it once, and every request from that client carries the credentials. No need to build the header each time.
## Per-request usage
If you want credentials on only one specific request, pass headers directly.
`make_basic_authentication_header()` builds the Base64-encoded header for you.
> **Warning:** Basic authentication **encodes** credentials in Base64 — it does not encrypt them. Always use it over HTTPS. Over plain HTTP, your password travels the network in the clear.
## Digest authentication
For the more secure Digest authentication scheme, use `set_digest_auth()`. This is only available when cpp-httplib is built with OpenSSL (or another TLS backend).
```cpp
cli.set_digest_auth("alice","s3cret");
```
> To call an API with a Bearer token, see [C06. Call an API with a Bearer token](../c06-bearer-token).
For Bearer token authentication — common in OAuth 2.0 and modern Web APIs — use `set_bearer_token_auth()`. Pass the token and cpp-httplib builds the `Authorization: Bearer <token>` header for you.
Set it once and every subsequent request carries the token. This is the go-to pattern for token-based APIs like GitHub, Slack, or your own OAuth service.
## Per-request usage
When you want the token on only one request — or need a different token per request — pass it via headers.
When you want to send a file the same way an HTML `<input type="file">` does, use multipart form data (`multipart/form-data`). cpp-httplib offers two APIs — `UploadFormDataItems` and `FormDataProviderItems` — and you pick between them based on **file size**.
## Send a small file
Read the file into memory first, then send it. For small files, this is the simplest path.
Each `UploadFormData` entry is `{name, content, filename, content_type}`. For plain text fields, leave `filename` and `content_type` empty.
## Stream a large file
To avoid loading the whole file into memory, use `make_file_provider()`. It reads the file in chunks as it sends — so even huge files won't blow up your memory footprint.
The arguments to `make_file_provider()` are `(form name, file path, file name, content type)`. Leave the file name empty to use the file path as-is.
> **Note:** You can mix `UploadFormDataItems` and `FormDataProviderItems` in the same request. A clean split is: text fields in `UploadFormDataItems`, files in `FormDataProviderItems`.
> To show upload progress, see [C11. Use the progress callback](../c11-progress-callback).
Sometimes you want to send a file's contents as the request body directly — no multipart wrapping. This is common for S3-compatible APIs or endpoints that take raw image data. For this, use `make_file_body()`.
`make_file_body()` returns a pair of file size and a `ContentProvider`. Pass them to `Post()` or `Put()` and the file contents flow straight into the request body.
The `ContentProvider` reads the file in chunks, so even huge files never sit fully in memory.
## When the file can't be opened
If the file can't be opened, `make_file_body()` returns `size` as `0` and `provider` as an empty function object. Sending that would produce garbage — always check `size` first.
> **Warning:** `make_file_body()` needs to fix the Content-Length up front, so it reads the file size ahead of time. If the file size might change mid-upload, this API isn't the right fit.
> To send the file as multipart form data instead, see [C07. Upload a file as multipart form data](../c07-multipart-upload).
When you don't know the body size up front — for data generated on the fly or piped from another stream — use `ContentProviderWithoutLength`. The client sends the body with HTTP chunked transfer encoding.
## Basic usage
```cpp
httplib::Clientcli("http://localhost:8080");
autores=cli.Post("/stream",
[&](size_toffset,httplib::DataSink&sink){
std::stringchunk=produce_next_chunk();
if(chunk.empty()){
sink.done();// done sending
returntrue;
}
returnsink.write(chunk.data(),chunk.size());
},
"application/octet-stream");
```
The lambda's job is just: produce the next chunk and send it with `sink.write()`. When there's no more data, call `sink.done()` and you're finished.
## When the size is known
If you **do** know the total size ahead of time, use the `ContentProvider` overload (taking `size_t offset, size_t length, DataSink &sink`) and pass the total size as well.
With a known size, the request carries a Content-Length header — so the server can show progress. Prefer this form when you can.
> **Detail:** `sink.write()` returns a `bool` indicating whether the write succeeded. If it returns `false`, the connection is gone — return `false` from the lambda to stop.
> If you're just sending a file, `make_file_body()` is easier. See [C08. POST a file as raw binary](../c08-post-file-body).
To receive a response body chunk by chunk, use a `ContentReceiver`. It's the obvious choice for large files, but it's equally handy for NDJSON (newline-delimited JSON) or log streams where you want to start processing data as it arrives.
## Process each chunk
```cpp
httplib::Clientcli("http://localhost:8080");
autores=cli.Get("/logs/stream",
[](constchar*data,size_tlen){
std::cout.write(data,len);
std::cout.flush();
returntrue;// return false to stop receiving
});
```
Data arrives in the lambda in the order it's received from the server. Return `false` from the callback to stop the download partway through.
## Parse NDJSON line by line
Here's a buffered approach for processing newline-delimited JSON one line at a time.
Accumulate into a buffer, then pull out and parse one line each time you see a newline. This is the standard pattern for consuming a streaming API in real time.
> **Warning:** When you pass a `ContentReceiver`, `res->body` stays **empty**. Store or process the body inside the callback yourself.
> To track download progress, combine this with [C11. Use the progress callback](../c11-progress-callback).
> For Server-Sent Events (SSE), see [E04. Receive SSE on the client](../e04-sse-client).
To display download or upload progress, pass a `DownloadProgress` or `UploadProgress` callback. Both take two arguments: `(current, total)`.
## Download progress
```cpp
httplib::Clientcli("http://localhost:8080");
autores=cli.Get("/large-file",
[](size_tcurrent,size_ttotal){
autopercent=(total>0)?(current*100/total):0;
std::cout<<"\rDownloading: "<<percent<<"% ("
<<current<<"/"<<total<<")"<<std::flush;
returntrue;// return false to abort
});
std::cout<<std::endl;
```
The callback fires each time data arrives. `total` comes from the Content-Length header — if the server doesn't send one, it may be `0`. In that case, you can't compute a percentage, so just display bytes received.
## Upload progress
Uploads work the same way. Pass an `UploadProgress` as the last argument to `Post()` or `Put()`.
Return `false` from the callback to abort the transfer. This is how you wire up a "Cancel" button in a UI — flip a flag, and the next progress tick stops the transfer.
```cpp
std::atomic<bool>cancelled{false};
autores=cli.Get("/large-file",
[&](size_tcurrent,size_ttotal){
return!cancelled.load();
});
```
> **Note:** `ContentReceiver` and the progress callback can be used together. When you want to stream to a file and show progress at the same time, pass both.
> For a concrete example of saving to a file, see [C01. Get the response body / save to a file](../c01-get-response-body).
The client has three kinds of timeouts, each set independently.
| Kind | API | Default | Meaning |
| --- | --- | --- | --- |
| Connection | `set_connection_timeout` | 300s | Time to wait for the TCP connection to establish |
| Read | `set_read_timeout` | 300s | Time to wait for a single `recv` when receiving the response |
| Write | `set_write_timeout` | 5s | Time to wait for a single `send` when sending the request |
## Basic usage
```cpp
httplib::Clientcli("http://localhost:8080");
cli.set_connection_timeout(5,0);// 5 seconds
cli.set_read_timeout(10,0);// 10 seconds
cli.set_write_timeout(10,0);// 10 seconds
autores=cli.Get("/api/data");
```
Pass seconds and microseconds as two arguments. If you don't need the sub-second part, you can omit the second argument.
## Use `std::chrono`
There's also an overload that takes a `std::chrono` duration directly. It's easier to read — recommended.
```cpp
usingnamespacestd::chrono_literals;
cli.set_connection_timeout(5s);
cli.set_read_timeout(10s);
cli.set_write_timeout(500ms);
```
## Watch out for the long 300s default
Connection and read timeouts default to **300 seconds (5 minutes)**. If the server hangs, you'll be waiting five minutes by default. Shorter values are usually a better idea.
```cpp
cli.set_connection_timeout(3s);
cli.set_read_timeout(10s);
```
> **Warning:** The read timeout covers a single receive call — not the whole request. If data keeps trickling in during a large download, the request can take half an hour without ever hitting the timeout. To cap the total request time, use [C13. Set an overall timeout](../c13-max-timeout).
> For WebSocket client timeouts, see [W06. Set Timeouts](../w06-websocket-timeouts).
The three timeouts from [C12. Set timeouts](../c12-timeouts) all apply to a single `send` or `recv` call. To cap the total time a request can take, use `set_max_timeout()`.
## Basic usage
```cpp
httplib::Clientcli("http://localhost:8080");
cli.set_max_timeout(5000);// 5 seconds (in milliseconds)
autores=cli.Get("/slow-endpoint");
```
The value is in milliseconds. Connection, send, and receive together — the whole request is aborted if it exceeds the limit.
## Use `std::chrono`
There's also an overload that takes a `std::chrono` duration.
```cpp
usingnamespacestd::chrono_literals;
cli.set_max_timeout(5s);
```
## When to use which
`set_read_timeout` fires when no data arrives for a while. If data keeps trickling in bit by bit, it will never fire. An endpoint that sends one byte per second can make `set_read_timeout` useless no matter how short you set it.
`set_max_timeout` caps elapsed time, so it handles those cases cleanly. It's great for calls to external APIs or anywhere you don't want users waiting forever.
```cpp
cli.set_connection_timeout(3s);
cli.set_read_timeout(10s);
cli.set_max_timeout(30s);// abort if the whole request takes over 30s
```
> **Note:** `set_max_timeout()` works alongside the regular timeouts. Short stalls get caught by `set_read_timeout`; long-running requests get capped by `set_max_timeout`. Use both for a safety net.
title:"C14. Understand Connection Reuse and Keep-Alive"
order:14
status:"draft"
---
When you send multiple requests through the same `httplib::Client` instance, the TCP connection is reused automatically. HTTP/1.1 Keep-Alive does the work for you — you don't pay the TCP and TLS handshake cost on every call.
## Connections are reused automatically
```cpp
httplib::Clientcli("https://api.example.com");
autores1=cli.Get("/users/1");
autores2=cli.Get("/users/2");// reuses the same connection
autores3=cli.Get("/users/3");// reuses the same connection
```
No special config required. Just hold on to `cli` — internally, the socket stays open across calls. The effect is especially noticeable over HTTPS, where the TLS handshake is expensive.
## Disable Keep-Alive explicitly
To force a fresh connection every time, call `set_keep_alive(false)`. Mostly useful for testing.
```cpp
cli.set_keep_alive(false);
```
For normal use, leave it on (the default).
## Don't create a `Client` per request
If you create a `Client` inside a loop and let it fall out of scope each iteration, you lose the reuse benefit. Create the instance outside the loop.
```cpp
// Bad: a new connection every iteration
for(autoid:ids){
httplib::Clientcli("https://api.example.com");
cli.Get("/users/"+id);
}
// Good: the connection is reused
httplib::Clientcli("https://api.example.com");
for(autoid:ids){
cli.Get("/users/"+id);
}
```
## Concurrent requests
If you want to send requests in parallel from multiple threads, give each thread its own `Client` instance. A single `Client` uses a single TCP connection, so firing concurrent requests at the same instance ends up serializing them anyway.
> **Note:** If the server closes the connection after its Keep-Alive timeout, cpp-httplib reconnects and retries transparently. You don't need to handle this in application code.
With `set_compress(true)`, the body of POST or PUT requests gets gzipped before sending. The server needs to handle compressed bodies too.
## Decompress the response
```cpp
httplib::Clientcli("https://api.example.com");
cli.set_decompress(true);// on by default
autores=cli.Get("/api/data");
std::cout<<res->body<<std::endl;
```
With `set_decompress(true)`, the client automatically decompresses responses that arrive with `Content-Encoding: gzip` or similar. `res->body` contains the decompressed data.
It's on by default, so normally you don't need to do anything. Set it to `false` only if you want the raw compressed bytes.
> **Warning:** If you build without `CPPHTTPLIB_ZLIB_SUPPORT`, calling `set_compress()` or `set_decompress()` does nothing. If compression isn't working, check the macro definition first.
To route traffic through a corporate network or a specific path, send requests via an HTTP proxy. Just pass the proxy host and port to `set_proxy()`.
## Basic usage
```cpp
httplib::Clientcli("https://api.example.com");
cli.set_proxy("proxy.internal",8080);
autores=cli.Get("/users");
```
The request goes through the proxy. For HTTPS, the client uses the CONNECT method to tunnel through — no extra setup required.
## Proxy authentication
If the proxy itself requires authentication, use `set_proxy_basic_auth()` or `set_proxy_bearer_token_auth()`.
```cpp
cli.set_proxy("proxy.internal",8080);
cli.set_proxy_basic_auth("user","password");
```
```cpp
cli.set_proxy_bearer_token_auth("token");
```
If cpp-httplib is built with OpenSSL (or another TLS backend), you can also use Digest authentication for the proxy.
```cpp
cli.set_proxy_digest_auth("user","password");
```
## Combine with end-server authentication
Proxy authentication is separate from authenticating to the end server ([C05. Use Basic authentication](../c05-basic-auth), [C06. Call an API with a Bearer token](../c06-bearer-token)). When both are needed, set both.
- a hostname suffix (e.g. `example.com`) — matches `example.com` itself and any subdomain (`foo.example.com`). A leading dot is permitted but informational; both forms are equivalent.
- a single IP literal (e.g. `192.168.1.1`, `::1`)
- a CIDR block (e.g. `10.0.0.0/8`, `fe80::/10`)
Hostname matching is case-insensitive and uses a dot-boundary rule, so an entry of `example.com` does **not** match `evilexample.com`. IP comparisons are normalized through `inet_pton`, so `127.0.0.1` cannot be bypassed via alternate string forms (e.g. `127.000.000.001`). When an entry matches, the `Proxy-Authorization` header is suppressed as well.
Malformed entries are silently dropped. Port-specific entries such as `example.com:8080` are not supported (cpp-httplib's other host-keyed APIs are also keyed on hostname only).
## Read proxy settings from the environment
cpp-httplib doesn't touch `HTTP_PROXY` / `HTTPS_PROXY` / `NO_PROXY` on its own — the config API is always explicit, the same way `set_ca_cert_path()` is. If you'd like that behavior, read the variables in your application and feed them to `set_proxy()` and `set_no_proxy()`.
```cpp
if(constchar*v=std::getenv("no_proxy")){
std::vector<std::string>patterns;
std::stringstreamss(v);
for(std::stringitem;std::getline(ss,item,',');){
if(!item.empty()){patterns.push_back(item);}
}
cli.set_no_proxy(patterns);
}
```
If you also read `HTTP_PROXY` yourself, honor the lowercase `http_proxy` only. The uppercase form is poisoned in CGI/FastCGI environments by the `Proxy:` request header ([CVE-2016-5385 / "httpoxy"](https://httpoxy.org/)). `HTTPS_PROXY` and `NO_PROXY` are safe in either case because their names don't begin with `HTTP_`.
`cli.Get()`, `cli.Post()`, and friends return a `Result`. When the request fails — can't reach the server, times out, etc. — the result is "falsy". To get the specific reason, use `Result::error()`.
Use `if (res)` to check success. On failure, `res.error()` returns a `httplib::Error` enum value. Pass it to `to_string()` to get a human-readable description.
## Common errors
| Value | Meaning |
| --- | --- |
| `Error::Connection` | Couldn't connect to the server |
When an HTTPS request fails, `res.error()` returns values like `Error::SSLConnection` or `Error::SSLServerVerification`. Sometimes that's not enough to pinpoint the cause. That's where `Result::ssl_error()` and `Result::ssl_backend_error()` help.
`ssl_error()` returns the error code from the SSL library (e.g., OpenSSL's `SSL_get_error()`). `ssl_backend_error()` gives you the backend's more detailed error value — for OpenSSL, that's `ERR_get_error()`.
## Format OpenSSL errors as strings
When you have a value from `ssl_backend_error()`, pass it to OpenSSL's `ERR_error_string()` to get a readable message.
The callback you pass to `set_logger()` fires once for each completed request. You get both the request and the response as arguments — so you can log the method, path, status, headers, body, or whatever else you need.
## Catch errors only
When a network-layer error happens (like `Error::Connection`), `set_logger()` is **not** called — there's no response to log. For those cases, use `set_error_logger()`.
> **Note:** The log callbacks run synchronously on the same thread as the request. Heavy work inside them slows the request down — push it to a background queue if you need to do anything expensive.
Server-Sent Events (SSE) is a simple protocol for pushing events one-way from server to client. The connection stays open, and the server can send data whenever it wants. It's lighter than WebSocket and fits entirely within HTTP — a nice combination.
cpp-httplib doesn't have a dedicated SSE server API, but you can implement one with `set_chunked_content_provider()` and `text/event-stream`.
When the client disconnects, call `sink.done()` to stop. Details in [S16. Detect client disconnection](../s16-disconnect).
## Heartbeats via comment lines
Lines starting with `:` are SSE comments — clients ignore them, but they **keep the connection alive**. Handy for preventing proxies and load balancers from closing idle connections.
```cpp
// heartbeat every 30 seconds
if(tick_count%30==0){
std::stringping=": ping\n\n";
sink.write(ping.data(),ping.size());
}
```
## Relationship with the thread pool
SSE connections stay open, so each client holds a worker thread. For lots of concurrent connections, enable dynamic scaling on the thread pool.
```cpp
svr.new_task_queue=[]{
returnnewhttplib::ThreadPool(8,128);
};
```
See [S21. Configure the thread pool](../s21-thread-pool).
> **Note:** When `data:` contains newlines, split it into multiple `data:` lines — one per line. This is how the SSE spec requires multiline data to be transmitted.
> For event names, see [E02. Use named events in SSE](../e02-sse-event-names). For the client side, see [E04. Receive SSE on the client](../e04-sse-client).
SSE lets you send multiple kinds of events over the same stream. Give each one a name with the `event:` field, and the client can dispatch to a different handler per type. Great for things like "new message", "user joined", "user left" in a chat app.
A message is `event:` → `data:` → blank line. If you omit `event:`, the client treats it as a default `"message"` event.
## Attach IDs for reconnect
When you include an `id:` field, the client automatically sends it back as `Last-Event-ID` on reconnect, telling the server "here's how far I got."
```cpp
autosend_event=[](httplib::DataSink&sink,
conststd::string&event,
conststd::string&data,
conststd::string&id){
std::stringmsg="id: "+id+"\n"
+"event: "+event+"\n"
+"data: "+data+"\n\n";
sink.write(msg.data(),msg.size());
};
send_event(sink,"message","Hello!","42");
```
The ID format is up to you. Monotonic counters or UUIDs both work — just pick something unique and orderable on the server side. See [E03. Handle SSE reconnection](../e03-sse-reconnect) for details.
## JSON payloads in data
For structured data, the usual move is to put JSON in `data:`.
```cpp
nlohmann::jsonpayload={
{"user","alice"},
{"text","Hello!"},
};
send_event(sink,"message",payload.dump(),"42");
```
On the client, parse the incoming `data` as JSON to get the original object back.
## Data with newlines
If the data value contains newlines, split it across multiple `data:` lines.
```cpp
std::stringmsg="data: line1\n"
"data: line2\n"
"data: line3\n\n";
sink.write(msg.data(),msg.size());
```
On the client side, these come back as a single `data` string with newlines.
> **Note:** Using `event:` makes client-side dispatch cleaner, but it also helps in the browser DevTools — events are easier to filter by type. That matters more than you'd expect while debugging.
SSE connections drop for all sorts of network reasons. Clients automatically try to reconnect, so it's a good idea to make your server resume from where it left off.
## Read `Last-Event-ID`
When the client reconnects, it sends the ID of the last event it received in the `Last-Event-ID` header. The server reads that and picks up from the next one.
On the first connect, `Last-Event-ID` is empty, so start from `0`. On reconnect, resume from the next ID. Event history is the server's responsibility — you need to keep recent events around somewhere.
## Set the reconnect interval
Sending a `retry:` field tells the client how long to wait before reconnecting, in milliseconds.
```cpp
std::stringmsg="retry: 5000\n\n";// reconnect after 5 seconds
sink.write(msg.data(),msg.size());
```
Usually you send this once at the start. During peak load or maintenance windows, a longer retry interval helps reduce reconnect storms.
## Buffer recent events
To support reconnection, keep a rolling buffer of recent events on the server.
When a client reconnects, call `since(last_id)` to send any events it missed.
## How much to keep
The buffer size is a tradeoff between memory and how far back a client can resume. It depends on the use case:
- Real-time chat: a few minutes to half an hour
- Notifications: the last N items
- Trading data: persist to a database and pull from there
> **Warning:** `Last-Event-ID` is a client-provided value — don't trust it blindly. If you read it as a number, validate the range. If it's a string, sanitize it.
cpp-httplib ships a dedicated `sse::SSEClient` class. It handles auto-reconnect, per-event-name dispatch, and `Last-Event-ID` tracking for you — so receiving SSE is painless.
Build an `SSEClient` with a `Client` and a path, register a callback with `on_message()`, and call `start()`. The event loop kicks in and automatically reconnects if the connection drops.
## Dispatch by event name
When the server sends events with an `event:` field, register a handler per name via `on_event()`.
```cpp
sse.on_event("message",[](constauto&msg){
std::cout<<"chat: "<<msg.data<<std::endl;
});
sse.on_event("join",[](constauto&msg){
std::cout<<msg.data<<" joined"<<std::endl;
});
sse.on_event("leave",[](constauto&msg){
std::cout<<msg.data<<" left"<<std::endl;
});
```
`on_message()` serves as a generic fallback for unnamed events (the default `message` type).
Hook into connection open and error events. Even when the error handler fires, `SSEClient` keeps trying to reconnect in the background.
## Run asynchronously
If you don't want to block the main thread, use `start_async()`.
```cpp
sse.start_async();
// main thread continues to do other things
do_other_work();
// when you're done, stop it
sse.stop();
```
`start_async()` spawns a background thread to run the event loop. Use `stop()` to shut it down cleanly.
## Configure reconnection
You can tune the reconnect interval and maximum retries.
```cpp
sse.set_reconnect_interval(5000);// 5 seconds
sse.set_max_reconnect_attempts(10);// up to 10 (0 = unlimited)
```
If the server sends a `retry:` field, that takes precedence.
## Automatic Last-Event-ID
`SSEClient` tracks the `id` of each received event internally and sends it back as `Last-Event-ID` on reconnect. As long as the server sends events with `id:`, this all works automatically.
> **Note:** `SSEClient::start()` blocks, which is fine for a one-off command-line tool. For GUI apps or embedded in a server, the `start_async()` + `stop()` pair is the usual pattern.
> For the server side, see [E01. Implement an SSE server](../e01-sse-server).
A collection of recipes that answer "How do I...?" questions. Each recipe is self-contained — read only what you need. For an introduction to the basics, see the [Tour](../tour/).
## Client
### Basics
- [C01. Get the response body / save to a file](c01-get-response-body)
- [C02. Send and receive JSON](c02-json)
- [C03. Set default headers](c03-default-headers)
- [C04. Follow redirects](c04-follow-location)
### Authentication
- [C05. Use Basic authentication](c05-basic-auth)
- [C06. Call an API with a Bearer token](c06-bearer-token)
### File Upload
- [C07. Upload a file as multipart form data](c07-multipart-upload)
- [C08. POST a file as raw binary](c08-post-file-body)
- [C09. Send the body with chunked transfer](c09-chunked-upload)
### Streaming & Progress
- [C10. Receive a response as a stream](c10-stream-response)
- [C11. Use the progress callback](c11-progress-callback)
### Connection & Performance
- [C12. Set timeouts](c12-timeouts)
- [C13. Set an overall timeout](c13-max-timeout)
- [C14. Understand connection reuse and Keep-Alive behavior](c14-keep-alive)
- [C15. Enable compression](c15-compression)
- [C16. Send requests through a proxy](c16-proxy)
### Error Handling & Debugging
- [C17. Handle error codes](c17-error-codes)
- [C18. Handle SSL errors](c18-ssl-errors)
- [C19. Set up client logging](c19-client-logger)
## Server
### Basics
- [S01. Register GET / POST / PUT / DELETE handlers](s01-handlers)
- [S02. Receive JSON requests and return JSON responses](s02-json-api)
- [S03. Use path parameters](s03-path-params)
- [S04. Set up a static file server](s04-static-files)
### Streaming & Files
- [S05. Stream a large file in the response](s05-stream-response)
- [S06. Return a file download response](s06-download-response)
- [S07. Receive multipart data as a stream](s07-multipart-reader)
- [S08. Return a compressed response](s08-compress-response)
### Handler Chain
- [S09. Add pre-processing to all routes](s09-pre-routing)
- [S10. Add response headers with a post-routing handler](s10-post-routing)
- [S11. Authenticate per route with a pre-request handler](s11-pre-request)
- [S12. Pass data between handlers with `res.user_data`](s12-user-data)
title:"S01. Register GET / POST / PUT / DELETE Handlers"
order:20
status:"draft"
---
With `httplib::Server`, you register a handler per HTTP method. Just pass a pattern and a lambda to `Get()`, `Post()`, `Put()`, or `Delete()`. For methods outside the built-in set, such as WebDAV's `PROPFIND`, use `CustomRoute()`.
Handlers take `(const Request&, Response&)`. Use `res.set_content()` to set the body and Content-Type, and `res.status` for the status code. `listen()` starts the server and blocks.
To add a response header, use `res.set_header("Name", "Value")`.
> **Note:** `listen()` is a blocking call. To run it on a different thread, wrap it in `std::thread`. If you need non-blocking startup, see [S18. Control startup order with `listen_after_bind`](../s18-listen-after-bind).
> To use path parameters like `/users/:id`, see [S03. Use path parameters](../s03-path-params).
> For methods outside the built-in set, such as WebDAV's `PROPFIND`, see [S23. Handle custom HTTP methods](../s23-custom-methods).
title:"S02. Receive a JSON Request and Return a JSON Response"
order:21
status:"draft"
---
cpp-httplib doesn't include a JSON parser. On the server side, combine it with something like [nlohmann/json](https://github.com/nlohmann/json). The examples below use `nlohmann/json`.
`req.body` is a plain `std::string`, so you pass it straight to your JSON library. For the response, `dump()` to a string and set the Content-Type to `application/json`.
> **Note:** A large JSON body ends up entirely in `req.body`, which means it all sits in memory. For huge payloads, consider streaming reception — see [S07. Receive multipart data as a stream](../s07-multipart-reader).
> For the client side, see [C02. Send and receive JSON](../c02-json).
A request to `/users/42` fills `req.path_params["id"]` with `"42"`. `path_params` is a `std::unordered_map<std::string, std::string>`, so use `at()` to read it.
To serve static files like HTML, CSS, and images, use `set_mount_point()`. Just map a URL path to a local directory, and the whole directory becomes accessible.
## Basic usage
```cpp
httplib::Serversvr;
svr.set_mount_point("/","./public");
svr.listen("0.0.0.0",8080);
```
`./public/index.html` is now reachable at `http://localhost:8080/index.html`, and `./public/css/style.css` at `http://localhost:8080/css/style.css`. The directory layout maps directly to URLs.
## Multiple mount points
You can register more than one mount point.
```cpp
svr.set_mount_point("/","./public");
svr.set_mount_point("/assets","./dist/assets");
svr.set_mount_point("/uploads","./var/uploads");
```
You can even mount multiple directories at the same path — they're searched in registration order, and the first hit wins.
## Combine with API handlers
Static files and API handlers coexist happily. Handlers registered with `Get()` and friends take priority; the mount points are searched only when nothing matches.
```cpp
svr.Get("/api/users",[](constauto&req,auto&res){
res.set_content("[]","application/json");
});
svr.set_mount_point("/","./public");
```
This gives you an SPA-friendly setup: `/api/*` hits the handlers, everything else is served from `./public/`.
## Add MIME types
cpp-httplib ships with a built-in extension-to-Content-Type map, but you can add your own.
> **Warning:** The static file server methods are **not thread-safe**. Don't call them after `listen()` — configure everything before starting the server.
> For download-style responses, see [S06. Return a file download response](../s06-download-response).
When the response is a huge file or data generated on the fly, loading the whole thing into memory isn't realistic. Use `Response::set_content_provider()` to produce data in chunks as you send it.
The lambda is called repeatedly with `offset` and `length`. Read just that range and write it to `sink`. Only a small chunk sits in memory at any given time.
## Just send a file
If you only want to serve a file, `set_file_content()` is far simpler.
It streams internally, so even huge files are safe. Omit the Content-Type and it's guessed from the extension.
## When the size is unknown — chunked transfer
For data generated on the fly, where you don't know the total size up front, use `set_chunked_content_provider()`. It's sent with HTTP chunked transfer encoding.
> **Note:** The provider lambda is called multiple times. Watch out for the lifetime of captured variables — wrap them in a `std::shared_ptr` if needed.
> To serve the file as a download, see [S06. Return a file download response](../s06-download-response).
To force a browser to show a **download dialog** instead of rendering inline, send a `Content-Disposition` header. There's no special cpp-httplib API for this — it's just a header.
> **Note:** Some browsers will trigger a download based on Content-Type alone, even without `Content-Disposition`. Conversely, setting `inline` tries to render the content in the browser when possible.
A naive upload handler puts the whole request into `req.body`, which blows up memory for large files. Use `HandlerWithContentReader` to receive the body chunk by chunk.
Only a small chunk sits in memory at any moment, so gigabyte-scale files are no problem.
## Count the parts yourself
There is a cap on the number of parts, `CPPHTTPLIB_MULTIPART_FORM_DATA_FILE_MAX_COUNT` (1024 by default), but it only applies to the buffered path, where every part is accumulated into `req.form`. The `ContentReader` keeps nothing on the library side, so the cap does not apply here.
If you want an upper bound, count the parts yourself and return `false` from the header callback. The parser stops right there.
When `content_reader` returns `false`, set the response status yourself. The rest of the body is left unread and the connection is closed, so a client that is still sending sees the connection drop.
> **Warning:** When you use `HandlerWithContentReader`, `req.body` stays **empty**. Handle the body yourself inside the callbacks.
> For the client side of multipart uploads, see [C07. Upload a file as multipart form data](../c07-multipart-upload).
cpp-httplib automatically compresses response bodies when the client indicates support via `Accept-Encoding`. The handler doesn't need to do anything special. Supported encodings are gzip, Brotli, and Zstd.
## Build-time setup
To enable compression, define the relevant macros before including `httplib.h`:
```cpp
#define CPPHTTPLIB_ZLIB_SUPPORT // gzip
#define CPPHTTPLIB_BROTLI_SUPPORT // brotli
#define CPPHTTPLIB_ZSTD_SUPPORT // zstd
#include<httplib.h>
```
You'll also need to link `zlib`, `brotli`, and `zstd` respectively. Enable only what you need.
That's it. If the client sent `Accept-Encoding: gzip`, cpp-httplib compresses the response with gzip automatically. `Content-Encoding: gzip` and `Vary: Accept-Encoding` are added for you.
## Encoding priority
When the client accepts multiple encodings, cpp-httplib picks in this order (among those enabled at build time): Brotli → Zstd → gzip. Your code doesn't need to care — you always get the most efficient option available.
## Streaming responses are compressed too
Streaming responses via `set_chunked_content_provider()` get the same automatic compression.
The lower bound defaults to 1400 bytes. A response that already fits in a single 1500-byte MTU is not delivered any faster for being smaller, and a file of a few bytes comes back larger than it went in, because gzip's header and trailer outweigh what deflate saves.
The upper bound defaults to 4MB and exists for a different reason: the file is compressed on every request, and the compressed bytes stay in memory until the response has been written, so the peak cost scales with the number of requests in flight. It bounds what a single request can cost, and says nothing about how well large files compress, so raising it is reasonable when the files are known and the traffic is not.
Either bound takes `0` to turn it off, and each has a compile-time default (`CPPHTTPLIB_STATIC_FILE_COMPRESSION_MIN_LENGTH`, `CPPHTTPLIB_STATIC_FILE_COMPRESSION_MAX_LENGTH`).
A compressed response keeps its `Content-Length`, so `HEAD` reports the same size a `GET` would. Two details to know: Range requests are answered from the uncompressed representation, and the `ETag` carries the coding it belongs to, as in `W/"...-gzip"`.
Content providers registered with `set_content_provider()` are not covered. Running one through a compressor holds each write back until the internal buffer fills, which stalls providers that build their body a piece at a time. To compress a generated body, use `set_chunked_content_provider()`.
> **Note:** The size range covers static files only. A body passed to `set_content()` is compressed whenever the client accepts it and the MIME type is compressible, however small it is, so a response of a few bytes ends up larger than it started. Decide in the handler if you want to avoid that.
> For the client-side counterpart, see [C15. Enable compression](../c15-compression).
If auth fails, return `Handled` to respond with 401 immediately. If it passes, return `Unhandled` and let routing take over.
## For per-route auth
If you want different auth rules per route rather than a single global check, `set_pre_request_handler()` is a better fit. See [S11. Authenticate per route with a pre-request handler](../s11-pre-request).
> **Note:** If all you want is to modify the response, `set_post_routing_handler()` is the right tool. See [S10. Add response headers with a post-routing handler](../s10-post-routing).
title:"S10. Add Response Headers with a Post-Routing Handler"
order:29
status:"draft"
---
Sometimes you want to add shared headers to the response after the handler has run — CORS headers, security headers, a request ID, and so on. That's what `set_post_routing_handler()` is for.
The post-routing handler runs **after the route handler, before the response is sent**. From here you can call `res.set_header()` or `res.headers.erase()` to add or remove headers across every response in one place.
No matter which handler produced the response, the same headers get attached.
> **Note:** The post-routing handler also runs for responses that didn't match any route and for responses from error handlers. That's exactly what you want when you need certain headers on every response, guaranteed.
title:"S11. Authenticate Per Route with a Pre-Request Handler"
order:30
status:"draft"
---
The `set_pre_routing_handler()` from [S09. Add pre-processing to all routes](../s09-pre-routing) runs **before routing**, so it has no idea which route matched. When you want per-route behavior, `set_pre_request_handler()` is what you need.
## Pre-routing vs. pre-request
| Hook | When it runs | Route info | Request body |
| --- | --- | --- | --- |
| `set_pre_routing_handler` | Before routing | Not available | Not read yet |
| `set_pre_request_handler` | After routing, right before the route handler | Available via `req.matched_route` | Not read yet |
In a pre-request handler, `req.matched_route` holds the **pattern string** that matched. You can vary behavior based on the route definition itself.
Because the body has not been read when the pre-request handler runs, you can reject a request — for example on a failed auth check — without consuming a (potentially large) request body. Note that this also means `req.body` and form fields parsed from the body are not available here; inspect headers, the path, query parameters, or `req.matched_route` instead.
`matched_route` is the pattern **before** path parameters are expanded (e.g. `/admin/users/:id`). You compare against the route definition, not the actual request path, so IDs or names don't throw you off.
The pre-request handler also runs for routes registered with `svr.WebSocket()`. It is called before the `101 Switching Protocols` response, so returning `Handled` sends your HTTP response (such as a 403) and the connection is never upgraded.
## Return values
Same as pre-routing — return `HandlerResponse`.
-`Unhandled`: continue (the route handler runs)
-`Handled`: we're done, skip the route handler
## Passing auth info to the route handler
To pass decoded user info into the route handler, use `res.user_data`. See [S12. Pass data between handlers with `res.user_data`](../s12-user-data).
title:"S12. Pass Data Between Handlers with res.user_data"
order:31
status:"draft"
---
Say your pre-request handler decodes an auth token and you want the route handler to use the result. That "data handoff between handlers" is what `res.user_data` is for — it holds values of arbitrary types.
`user_data.set()` stores a value of any type, and `user_data.get<T>()` retrieves it. If you give the wrong type you get `nullptr` back — so be careful.
## Typical value types
Strings, numbers, structs, `std::shared_ptr` — anything copyable or movable works.
The usual flow is: set it in `set_pre_routing_handler()` or `set_pre_request_handler()`, read it in the route handler. Pre-request runs after routing, so you can combine it with `req.matched_route` to set values only for specific routes.
## A gotcha
`user_data` lives on `Response`, not `Request`. That's because handlers get `Response&` (mutable) but only `const Request&`. It looks odd at first, but it makes sense once you think of it as "the mutable context shared between handlers."
> **Warning:** `user_data.get<T>()` returns `nullptr` when the type doesn't match. Use the exact same type on set and get. Storing as `AuthUser` and fetching as `const AuthUser` won't work.
The error handler runs right before an error response is sent — any time `res.status` is 4xx or 5xx. Replace the body with `res.set_content()` and every error response uses the same template.
Now every error comes back in a consistent JSON shape.
> **Note:** `set_error_handler()` also fires for 500 responses caused by exceptions thrown from a route handler. To get at the exception itself, combine it with `set_exception_handler()`. See [S14. Catch exceptions](../s14-exception-handler).
When a route handler throws, cpp-httplib keeps the server running and responds with 500. By default, though, very little of the error information reaches the client. `set_exception_handler()` lets you intercept exceptions and build your own response.
The handler receives a `std::exception_ptr`. The idiomatic move is to rethrow it with `std::rethrow_exception()` and catch by type. You can vary status code and message based on the exception type.
## Branch on custom exception types
If you throw your own exception types, you can map them to 400 or 404 responses.
```cpp
structNotFound:std::runtime_error{
usingstd::runtime_error::runtime_error;
};
structBadRequest:std::runtime_error{
usingstd::runtime_error::runtime_error;
};
svr.set_exception_handler(
[](constauto&req,auto&res,std::exception_ptrep){
try{
std::rethrow_exception(ep);
}catch(constNotFound&e){
res.status=404;
res.set_content(e.what(),"text/plain");
}catch(constBadRequest&e){
res.status=400;
res.set_content(e.what(),"text/plain");
}catch(conststd::exception&e){
res.status=500;
res.set_content("internal error","text/plain");
}
});
```
Now throwing `NotFound("user not found")` inside a handler is enough to return 404. No per-handler try/catch needed.
## Relationship with set_error_handler
`set_exception_handler()` runs the moment the exception is thrown. After that, if `res.status` is 4xx or 5xx, `set_error_handler()` also runs. The order is `exception_handler` → `error_handler`. Think of their roles as:
- **Exception handler**: interpret the exception, set the status and message
- **Error handler**: see the status and wrap it in the shared template
> **Note:** Without an exception handler, cpp-httplib returns a default 500 response and the exception details never make it to logs. Always set one for anything you want to debug.
To log the requests the server receives and the responses it returns, use `Server::set_logger()`. The callback fires once per completed request, making it the foundation for access logs and metrics collection.
For more on `user_data`, see [S12. Pass data between handlers with `res.user_data`](../s12-user-data).
> **Note:** The logger runs synchronously on the same thread as request processing. Heavy work inside it hurts throughput — push it to a queue and process asynchronously if you need anything expensive.
title:"S16. Detect When the Client Has Disconnected"
order:35
status:"draft"
---
During a long-running response, the client might close the connection. There's no point continuing to do work no one's waiting for. In cpp-httplib, check `req.is_connection_closed()`.
When you detect a disconnect, call `sink.done()` to stop the provider from being called again.
## How often should you check?
The call itself is cheap, but calling it in a tight inner loop doesn't add much value. Check at **boundaries where interrupting is safe** — after producing a chunk, after a database query, etc.
> **Warning:** `is_connection_closed()` is not guaranteed to reflect reality instantly. Because of how TCP works, sometimes you only notice the disconnect when you try to send. Don't expect pixel-perfect real-time detection — think of it as "we'll notice eventually."
Standing up a test server often hits port conflicts. With `bind_to_any_port()`, you let the OS pick a free port and then read back which one it gave you.
## Basic usage
```cpp
httplib::Serversvr;
svr.Get("/",[](constauto&req,auto&res){
res.set_content("hello","text/plain");
});
intport=svr.bind_to_any_port("0.0.0.0");
std::cout<<"listening on port "<<port<<std::endl;
svr.listen_after_bind();
```
`bind_to_any_port()` is equivalent to passing `0` as the port — the OS assigns a free one. The return value is the port actually used.
After that, call `listen_after_bind()` to start accepting. You can't combine bind and listen into a single call here, so you work in two steps.
## Useful in tests
This pattern is great for tests that spin up a server and hit it.
`bind_to_port()` reserves the port; `listen_after_bind()` actually starts accepting. Splitting them gives you a window between the two steps.
## Common use cases
**Privilege drop**: Binding to a port under 1024 requires root. Bind as root, drop to a normal user, and all subsequent request handling runs with reduced privileges.
```cpp
svr.bind_to_port("0.0.0.0",80);
drop_privileges();
svr.listen_after_bind();
```
**Startup notification**: Tell the parent process or systemd "I'm ready" before starting to accept connections.
**Test synchronization**: In tests, you can reliably catch "the moment the server is bound" and start the client after that.
## Check the return values
`bind_to_port()` returns `false` on failure, for example when you don't have permission to bind to the port. Always check it.
```cpp
if(!svr.bind_to_port("0.0.0.0",8080)){
std::cerr<<"bind failed"<<std::endl;
return1;
}
```
`listen_after_bind()` blocks until the server stops and returns `true` on a clean shutdown.
## Detect a port that's already in use
With the default settings, you can actually bind to a port another server is already using. That's because cpp-httplib sets `SO_REUSEPORT` (Linux, macOS) or `SO_REUSEADDR` (Windows) on the server socket. A restarted server can bind again right away. The flip side is that a second server on the same port starts without an error, and connections get split between the two.
To make `bind_to_port()` fail on a port in use, replace the socket options with `set_socket_options()`.
`set_socket_options()` replaces the defaults entirely. Setting `SO_REUSEADDR` on Linux and macOS keeps the "restarted server can bind again right away" behavior.
> **Note:** `SO_REUSEADDR` alone isn't enough on Windows. Two sockets that both set it can bind to the same port, so use `SO_EXCLUSIVEADDRUSE` instead.
> **Note:** To auto-pick a free port, see [S17. Bind to any available port](../s17-bind-any-port). Under the hood, that's just `bind_to_any_port()` + `listen_after_bind()`.
To stop the server, call `Server::stop()`. It's safe to call even while requests are in flight, so you can wire it to SIGINT or SIGTERM for a graceful shutdown.
`listen()` blocks, so the typical pattern is: run the server on a background thread and call `stop()` from the main thread. After `stop()`, `listen()` returns and you can `join()`.
## Shut down on a signal
Here's how to stop the server on SIGINT (Ctrl+C) or SIGTERM.
`stop()` is thread-safe and signal-safe — you can call it from a signal handler. Even when `listen()` is running on the main thread, the signal pulls it out cleanly.
## What happens to in-flight requests
When you call `stop()`, new connections are refused, but requests already being processed are **allowed to finish**. Once all workers drain, `listen()` returns. That's what makes it graceful.
> **Warning:** There's a wait between calling `stop()` and `listen()` returning — it's the time in-flight requests take to finish. To enforce a timeout, you'll need to add your own shutdown timer in application code.
`httplib::Server` enables HTTP/1.1 Keep-Alive automatically. From the client's perspective, connections are reused — so they don't pay the TCP handshake cost on every request. When you need to tune the behavior, there are two setters.
## What you can configure
| API | Default | Meaning |
| --- | --- | --- |
| `set_keep_alive_max_count` | 100 | Max requests served over a single connection |
| `set_keep_alive_timeout` | 5s | How long an idle connection is kept before closing |
## Basic usage
```cpp
httplib::Serversvr;
svr.set_keep_alive_max_count(20);
svr.set_keep_alive_timeout(10);// 10 seconds
svr.listen("0.0.0.0",8080);
```
`set_keep_alive_timeout()` also has a `std::chrono` overload.
```cpp
usingnamespacestd::chrono_literals;
svr.set_keep_alive_timeout(10s);
```
## Tuning ideas
**Too many idle connections eating resources**
Shorten the timeout so idle connections drop and release their worker threads.
```cpp
svr.set_keep_alive_timeout(2s);
```
**API is hammered and you want max reuse**
Raising the per-connection request cap improves benchmark numbers.
```cpp
svr.set_keep_alive_max_count(1000);
```
**Never reuse connections**
Set `set_keep_alive_max_count(1)` and every request gets its own connection. Mostly only useful for debugging or compatibility testing.
## Relationship with the thread pool
A Keep-Alive connection holds a worker thread for its entire lifetime. If `connections × concurrent requests` exceeds the thread pool size, new requests wait. For thread counts, see [S21. Configure the thread pool](../s21-thread-pool).
> **Note:** For the client side, see [C14. Understand connection reuse and Keep-Alive behavior](../c14-keep-alive). Even when the server closes the connection on timeout, the client reconnects automatically.
cpp-httplib serves requests from a thread pool. By default, the base thread count is the greater of `std::thread::hardware_concurrency() - 1` and `8`, and it can scale up dynamically to 4× that. To set thread counts explicitly, provide your own factory via `new_task_queue`.
The factory is a lambda returning a `TaskQueue*`. Pass `base_threads` and `max_threads` to `ThreadPool` and the pool scales between them based on load. Idle threads exit after a timeout (3 seconds by default).
## Also cap the queue
The pending queue can eat memory if it grows unchecked. You can cap it too.
```cpp
svr.new_task_queue=[]{
returnnewhttplib::ThreadPool(
/*base_threads=*/12,
/*max_threads=*/0,// disable dynamic scaling
/*max_queued_requests=*/18);
};
```
`max_threads=0` disables dynamic scaling — you get a fixed `base_threads`. Requests that don't fit in `max_queued_requests` are rejected.
## Use your own thread pool
You can plug in a fully custom thread pool by subclassing `TaskQueue` and returning it from the factory.
Handy when you already have a thread pool in your project and want to keep thread management unified.
## Compile-time tuning
You can set the defaults with macros if you want compile-time configuration.
```cpp
#define CPPHTTPLIB_THREAD_POOL_COUNT 16 // base thread count
#define CPPHTTPLIB_THREAD_POOL_MAX_COUNT 128 // max thread count
#define CPPHTTPLIB_THREAD_POOL_IDLE_TIMEOUT 5 // seconds before idle threads exit
#include<httplib.h>
```
> **Note:** A WebSocket connection holds a worker thread for its entire lifetime. For lots of simultaneous WebSocket connections, enable dynamic scaling (e.g. `ThreadPool(8, 64)`).
When you want to talk only to other processes on the same host, a Unix domain socket is a nice fit. It avoids TCP overhead and uses filesystem permissions for access control. Local IPC and services sitting behind a reverse proxy are classic use cases.
## Server side
```cpp
httplib::Serversvr;
svr.set_address_family(AF_UNIX);
svr.Get("/",[](constauto&,auto&res){
res.set_content("hello from unix socket","text/plain");
});
svr.listen("/tmp/httplib.sock",80);
```
Call `set_address_family(AF_UNIX)` first, then pass the socket file path as the first argument to `listen()`. The port number is unused but required by the signature — pass any value.
## Client side
```cpp
httplib::Clientcli("/tmp/httplib.sock");
cli.set_address_family(AF_UNIX);
autores=cli.Get("/");
if(res){
std::cout<<res->body<<std::endl;
}
```
Pass the socket file path to the `Client` constructor and call `set_address_family(AF_UNIX)`. Everything else works like a normal HTTP request.
## When to use it
- **Behind a reverse proxy**: An nginx-to-backend setup over a Unix socket is faster than TCP and sidesteps port management
- **Local-only APIs**: IPC between tools that shouldn't be reachable from outside
- **In-container IPC**: Process-to-process communication within the same pod or container
- **Dev environments**: No more worrying about port conflicts
## Clean up the socket file
A Unix domain socket creates a real file in the filesystem. It doesn't get removed on shutdown, so delete it before starting if needed.
```cpp
std::remove("/tmp/httplib.sock");
svr.listen("/tmp/httplib.sock",80);
```
## Permissions
You control who can connect via the socket file's permissions.
```cpp
svr.listen("/tmp/httplib.sock",80);
// from another process or thread
chmod("/tmp/httplib.sock",0660);// owner and group only
```
> **Warning:** Some Windows versions support AF_UNIX, but the implementation and behavior differ by platform. Test thoroughly before running cross-platform in production.
The server rejects HTTP methods it does not know with `400 Bad Request`. To accept an extension method, such as the WebDAV methods of RFC 4918 (`PROPFIND`, `PROPPATCH`, `MKCOL` and friends) or UPnP's `SUBSCRIBE`, register a handler with `CustomRoute()`. Registering the handler is what makes the server accept the method.
Patterns work the same way as they do for `Get()`. Regular expressions and path parameters are both available.
## Advertise your methods with OPTIONS
A WebDAV client asks the server about its capabilities with `OPTIONS` before doing anything else. cpp-httplib generates neither the `DAV:` header nor `Allow`, so return them yourself. Forget this and clients will turn you away even though your `PROPFIND` works.
- The method name has to be a valid HTTP method token (RFC 9110), and it must be registered before you call `listen()`
-`GET`, `HEAD`, `POST`, `PUT`, `DELETE`, `CONNECT`, `OPTIONS`, `TRACE`, `PATCH` and `PRI` cannot be registered here. Use the dedicated methods for those
- A rejected registration makes `is_valid()` return `false` and `listen()` fail, so the server never starts holding a handler that would never run
- Static file serving and WebSocket upgrades stay `GET`/`HEAD` only
> **Note:** cpp-httplib takes you as far as routing the method. If you want to call it WebDAV, generating the `207 Multi-Status` XML, interpreting the `Depth` header and managing locks are all yours to implement. The protocol itself lives outside the library.
> For the basics of registering handlers, see [S01. Register GET / POST / PUT / DELETE handlers](../s01-handlers).
title:"T01. Choosing Between OpenSSL, mbedTLS, and wolfSSL"
order:43
status:"draft"
---
cpp-httplib doesn't ship its own TLS implementation — it uses one of three backends that you pick at build time via a macro.
| Backend | Macro | Character |
| --- | --- | --- |
| OpenSSL | `CPPHTTPLIB_OPENSSL_SUPPORT` | Most widely used, richest feature set |
| mbedTLS | `CPPHTTPLIB_MBEDTLS_SUPPORT` | Lightweight, aimed at embedded |
| wolfSSL | `CPPHTTPLIB_WOLFSSL_SUPPORT` | Embedded-friendly, commercial support available |
## Build-time selection
Define the macro for your chosen backend before including `httplib.h`:
```cpp
#define CPPHTTPLIB_OPENSSL_SUPPORT
#include<httplib.h>
```
You'll also need to link against the backend's libraries (`libssl`, `libcrypto`, `libmbedtls`, `libwolfssl`, etc.).
## Which to pick
**When in doubt, OpenSSL**
It has the most features and the best documentation. For normal server use or Linux desktop apps, start here — you probably won't need anything else.
**To shrink binary size or target embedded**
mbedTLS or wolfSSL are a better fit. They're far more compact than OpenSSL and run on memory-constrained devices.
**When you need commercial support**
wolfSSL offers commercial licensing and support. If you're shipping in a product, it's worth considering.
## Supporting multiple backends
The usual approach is to treat each backend as a build variant and recompile the same source with different macros. cpp-httplib smooths over most of the API differences, but the backends are not 100% identical — always test.
## APIs that work across all backends
Certificate verification control, standing up an SSLServer, reading the peer certificate — these all share the same API across backends:
- [T02. Control SSL certificate verification](../t02-cert-verification)
- [T03. Start an SSL/TLS server](../t03-ssl-server)
- [T05. Access the peer certificate on the server](../t05-peer-cert)
> **Note:** On macOS with an OpenSSL-family backend, cpp-httplib automatically loads root certificates from the system keychain (via `CPPHTTPLIB_USE_CERTS_FROM_MACOSX_KEYCHAIN`, on by default). To disable this, define `CPPHTTPLIB_DISABLE_MACOSX_AUTOMATIC_ROOT_CERTIFICATES`.
By default, an HTTPS client verifies the server certificate — it uses the OS root certificate store to check the chain and the hostname. Here are the APIs for changing that behavior.
## Specify a custom CA certificate
When connecting to a server whose certificate is signed by an internal CA, use `set_ca_cert_path()`.
The first argument is the CA certificate file; the second is an optional CA directory. With the OpenSSL backend, you can also pass an `X509_STORE*` directly via `set_ca_cert_store()`.
That's all it takes to disable chain verification.
> **Warning:** Disabling certificate verification removes protection against man-in-the-middle attacks. **Never do this in production.** If you find yourself needing it outside of dev/test, pause and make sure you're not doing something wrong.
## Disable hostname verification only
There's an in-between option: verify the certificate chain, but skip the hostname check. Useful when you need to reach a server whose cert CN/SAN doesn't match the request's hostname.
```cpp
cli.enable_server_hostname_verification(false);
```
The certificate itself is still validated, so this is safer than fully disabling verification — but still not recommended in production.
## Use the OS cert store as-is
On most Linux distributions, root certificates live in a single file like `/etc/ssl/certs/ca-certificates.crt`. cpp-httplib reads the OS default store at startup, so for most servers you don't need to configure anything.
> The same APIs work on the mbedTLS and wolfSSL backends. For choosing between backends, see [T01. Choosing between OpenSSL, mbedTLS, and wolfSSL](../t01-tls-backends).
> For details on diagnosing failures, see [C18. Handle SSL errors](../c18-ssl-errors).
> For TLS configuration on a WebSocket client (`wss://`), see [W05. Configure TLS for wss:// Connections](../w05-websocket-tls).
To stand up an HTTPS server, use `httplib::SSLServer` instead of `httplib::Server`. Pass a certificate and private key to the constructor, and you get back something that works exactly like `Server`.
## Basic usage
```cpp
#define CPPHTTPLIB_OPENSSL_SUPPORT
#include<httplib.h>
intmain(){
httplib::SSLServersvr("cert.pem","key.pem");
svr.Get("/",[](constauto&req,auto&res){
res.set_content("hello over TLS","text/plain");
});
svr.listen("0.0.0.0",443);
}
```
Pass the server certificate (PEM format) and private key file paths to the constructor. That's all you need for a TLS-enabled server. Registering handlers and calling `listen()` work the same as with `Server`.
## Password-protected private keys
The fifth argument is the private key password.
```cpp
httplib::SSLServersvr("cert.pem","key.pem",
nullptr,nullptr,"password");
```
The third and fourth arguments are for client certificate verification (mTLS, see [T04. Configure mTLS](../t04-mtls)). For now, pass `nullptr`.
## Load PEM data from memory
When you want to load certs from memory instead of files, use the `PemMemory` struct.
```cpp
httplib::SSLServer::PemMemorypem{};
pem.cert_pem=cert_data.data();
pem.cert_pem_len=cert_data.size();
pem.key_pem=key_data.data();
pem.key_pem_len=key_data.size();
httplib::SSLServersvr(pem);
```
Handy when you pull certificates from environment variables or a secrets manager.
## Rotate certificates
Before a certificate expires, you may want to swap it out without restarting the server. That's what `update_certs_pem()` is for.
```cpp
svr.update_certs_pem(new_cert_pem,new_key_pem);
```
Existing connections keep using the old cert; new connections use the new one.
## Generating a test certificate
For a throwaway self-signed cert, use the `openssl` CLI.
In production, use certificates from Let's Encrypt or your internal CA.
> **Warning:** Binding an HTTPS server to port 443 requires root. For a safe way to do that, see the privilege-drop pattern in [S18. Control startup order with `listen_after_bind`](../s18-listen-after-bind).
> For mutual TLS (client certificates), see [T04. Configure mTLS](../t04-mtls).
Regular TLS verifies the server certificate only. **mTLS** (mutual TLS) adds the other direction: the client presents a certificate too, and the server verifies it. It's common for zero-trust API-to-API traffic and internal system authentication.
## Server side
Pass the CA used to verify client certificates as the third (and fourth) argument to `SSLServer`.
```cpp
httplib::SSLServersvr(
"server-cert.pem",// server certificate
"server-key.pem",// server private key
"client-ca.pem",// CA that signs valid client certs
With this, any connection whose client certificate isn't signed by `client-ca.pem` is rejected at the handshake. By the time a handler runs, the client is already authenticated.
## Configure with in-memory PEM
```cpp
httplib::SSLServer::PemMemorypem{};
pem.cert_pem=server_cert.data();
pem.cert_pem_len=server_cert.size();
pem.key_pem=server_key.data();
pem.key_pem_len=server_key.size();
pem.client_ca_pem=client_ca.data();
pem.client_ca_pem_len=client_ca.size();
httplib::SSLServersvr(pem);
```
This is the clean way when you load certificates from environment variables or a secrets manager.
## Client side
On the client side, pass the client certificate and key to `SSLClient`.
```cpp
httplib::SSLClientcli("api.example.com",443,
"client-cert.pem",
"client-key.pem");
autores=cli.Get("/");
```
Note you're using `SSLClient` directly, not `Client`. If the private key has a password, pass it as the fifth argument.
The client side has the same `PemMemory` struct too, letting you set the client certificate from PEM in memory.
```cpp
httplib::SSLClient::PemMemorypem{};
pem.cert_pem=client_cert.data();
pem.cert_pem_len=client_cert.size();
pem.key_pem=client_key.data();
pem.key_pem_len=client_key.size();
httplib::SSLClientcli("api.example.com",443,pem);
autores=cli.Get("/");
```
> For mTLS with a WebSocket client (`wss://`), see [W05. Configure TLS for wss:// Connections](../w05-websocket-tls).
## Read client info from a handler
To see which client connected from inside a handler, use `req.peer_cert()`. Details in [T05. Access the peer certificate on the server](../t05-peer-cert).
## Use cases
- **Microservice-to-microservice calls**: Issue a cert per service, use the cert as identity
- **IoT device management**: Burn a cert into each device and use it to gate API access
- **An alternative to internal VPN**: Put cert-based auth in front of public endpoints so internal resources can be reached safely
> **Note:** Issuing and revoking client certificates is more operational work than password-based auth. You'll need either an internal PKI setup or an automated flow using ACME-family tools.
Combined with a pre-request handler, you can keep all authorization logic in one place. See [S11. Authenticate per route with a pre-request handler](../s11-pre-request).
## SNI (Server Name Indication)
cpp-httplib handles SNI automatically. If one server hosts multiple domains, SNI is used under the hood — but normally handlers don't need to care.
> **Warning:** `req.peer_cert()` only returns a meaningful value when mTLS is enabled and the client actually presented a certificate. For plain TLS, you get an empty `PeerCert`. Always do the `bool` check before using it.
> To set up mTLS, see [T04. Configure mTLS](../t04-mtls).
title:"W01. Implement a WebSocket Echo Server and Client"
order:52
status:"draft"
---
WebSocket is a protocol for **two-way** messaging between client and server. cpp-httplib provides APIs for both sides. Let's start with the simplest example: an echo server.
Register a WebSocket handler with `svr.WebSocket()`. By the time the handler runs, the WebSocket handshake is already complete. Inside the loop, just `ws.read()` and `ws.send()` to get a working echo.
The `read()` return value is a `ReadResult` enum:
-`ReadResult::Text`: received a text message
-`ReadResult::Binary`: received a binary message
-`ReadResult::Fail`: error, or connection closed
-`ReadResult::Timeout`: a read timeout you set with `set_read_timeout()` elapsed with nothing received; the connection is still open. The compile-time default timeout closes the connection and is reported as `Fail` instead — see [W06. Set Timeouts](../w06-websocket-timeouts)
The `std::string` overload sends as **text**; the `const char*` + size overload sends as **binary**. A bit subtle, but once you know it, it's intuitive. See [W04. Send and receive binary frames](../w04-websocket-binary) for details.
## Thread pool implications
A WebSocket handler holds its worker thread for the entire life of the connection — one connection per thread. For many concurrent clients, configure a dynamic thread pool.
```cpp
svr.new_task_queue=[]{
returnnewhttplib::ThreadPool(8,128);
};
```
See [S21. Configure the thread pool](../s21-thread-pool).
> **Note:** To run WebSocket over HTTPS, use `httplib::SSLServer` instead of `httplib::Server` — the same `WebSocket()` handler just works. On the client side, use a `wss://` URL. For CA and client certificate configuration, see [W05. Configure TLS for wss:// Connections](../w05-websocket-tls).
WebSocket connections stay open for a long time, and proxies or load balancers will sometimes drop them for being "idle." To prevent that, you periodically send Ping frames to keep the connection alive. cpp-httplib can do this for you automatically.
## Server side
```cpp
svr.set_websocket_ping_interval(30);// ping every 30 seconds
svr.WebSocket("/chat",[](constauto&req,auto&ws){
// ...
});
```
Just pass the interval in seconds. Every WebSocket connection this server accepts will be pinged on that interval.
The default interval is set by the build-time macro `CPPHTTPLIB_WEBSOCKET_PING_INTERVAL_SECOND`. Usually you won't need to change it, but adjust downward if you're dealing with an aggressive proxy.
## What about Pong?
The WebSocket protocol requires that Ping frames are answered with Pong frames. cpp-httplib responds to Pings automatically — you don't need to think about it in application code.
## Picking an interval
| Environment | Suggested |
| --- | --- |
| Normal internet | 30–60s |
| Strict proxies (e.g. AWS ALB) | 15–30s |
| Mobile networks | 60s+ (too short drains battery) |
Too short wastes bandwidth; too long and connections get dropped. As a rule of thumb, target about **half the idle timeout** of whatever's between you and the client.
> **Warning:** A very short ping interval spawns background work per connection and increases CPU usage. For servers with many connections, keep the interval modest.
## Detecting an unresponsive peer
Sending pings alone doesn't tell you anything if the peer just silently dies — the TCP socket might still look open while the process on the other end is long gone. To catch that, enable the max-missed-pongs check: if N consecutive pings go unanswered, the connection is closed.
```cpp
cli.set_websocket_max_missed_pongs(2);// close after 2 consecutive unacked pings
```
The server side has the same `set_websocket_max_missed_pongs()`.
With a 30-second ping interval and `max_missed_pongs = 2`, a dead peer is detected within roughly 60 seconds and the connection is closed with `CloseStatus::GoingAway` and the reason `"pong timeout"`.
The counter is reset whenever `read()` consumes an incoming Pong frame, so this only works if your code is actively calling `read()` in a loop — which is what a normal WebSocket client does anyway.
### Why the default is 0
`max_missed_pongs` defaults to `0`, which means "never close the connection because of missing pongs." Pings are still sent on the heartbeat interval, but their responses aren't checked. If you want unresponsive-peer detection, set it explicitly to `1` or higher.
On the server side, even with `0`, a dead connection won't linger forever: while a handler is inside `read()`, `CPPHTTPLIB_WEBSOCKET_SERVER_READ_TIMEOUT_SECOND` (default **300 seconds = 5 minutes**) acts as a backstop. A client has no backstop of its own — it waits forever unless you set a read timeout — so there `max_missed_pongs` is what notices an unresponsive peer at all. On either side, it is also how you notice one **faster** than that 5-minute fallback.
> For handling a closed connection, see [W03. Handle connection close](../w03-websocket-close).
A WebSocket ends when either side closes it explicitly, or when the network drops. Handle close cleanly, and your cleanup and reconnect logic stays tidy.
## Detect a closed connection
When `ws.read()` returns `ReadResult::Fail`, the connection is gone — either cleanly or with an error. Break out of the loop and the handler will finish.
> **Note:** On a sudden network drop, `read()` returns `Fail` with no chance to call `close()`. Put your cleanup at the end of the handler, and both paths — clean close and abrupt disconnect — end up in the same place.
WebSocket has two frame types: text and binary. JSON and plain text go in text frames; images and raw protocol bytes go in binary. In cpp-httplib, `send()` picks the right type via overload.
## How to pick a frame type
```cpp
ws.send(std::string("Hello"));// text
ws.send("Hello",5);// binary
ws.send(binary_data,binary_data_size);// binary
```
The `std::string` overload sends as **text**. The `const char*` + size overload sends as **binary**. A bit subtle, but once you know it, it sticks.
If you have a `std::string` and want to send it as binary, pass `.data()` and `.size()` explicitly.
```cpp
std::stringraw=build_binary_payload();
ws.send(raw.data(),raw.size());// binary frame
```
## Detect frame type on receive
The return value of `ws.read()` tells you whether the received frame was text or binary.
// read timeout elapsed; the connection is still open
break;
}
```
Binary frames still come back in a `std::string`, but treat its contents as raw bytes — use `msg.data()` and `msg.size()`.
## When binary is the right call
- **Images, video, audio**: No Base64 overhead
- **Custom protocols**: protobuf, MessagePack, or any structured binary format
- **Game networking**: When latency matters
- **Sensor data streams**: Push numeric arrays directly
## Ping is binary-ish, but hidden
WebSocket Ping/Pong frames are close cousins of binary frames at the opcode level, but cpp-httplib handles them automatically — you don't touch them. See [W02. Set a WebSocket heartbeat](../w02-websocket-ping).
You can mix text and binary in the same connection. A common pattern: JSON for control messages, binary for the actual data — you get efficient handling of metadata and payload both.
> **Note:** WebSocket frames don't have an infinite size limit. For very large data, chunk it in your application code. cpp-httplib can handle a big frame in one shot, but it does load it all into memory at once.
Client-side TLS configuration for `wss://` (WebSocket over TLS) connections uses almost the same API as `SSLClient`. `ws::WebSocketClient` handles both `ws://` and `wss://` through the same class, so there's no separate class to switch to the way `SSLClient` requires.
Use `set_ca_cert_path()` to point at your own CA certificate. The signature matches `SSLClient`: the first argument is the CA certificate file, the second is an optional CA directory.
To disable certificate verification entirely, use `enable_server_certificate_verification(false)`. For details on that behavior, see [T02. Control SSL Certificate Verification](../t02-cert-verification).
## Presenting a client certificate (mTLS)
`ws::WebSocketClient` has a constructor overload that takes a `PemMemory` struct, letting `wss://` connections present a client certificate.
Passing `PemMemory` to a `ws://` (non-TLS) URL is silently ignored. There's no constructor that reads the cert files directly, so unlike `SSLClient` you always load the PEM into memory yourself before passing it in.
For the full mTLS picture, including server-side setup and use cases, see [T04. Configure mTLS](../t04-mtls).
Set the connection and write timeouts before calling `connect()`. The read timeout can be changed at any time — setting it on an open connection takes effect on the next `read()`.
## Use `std::chrono`
Just like `Client`, there's an overload that takes a `std::chrono` duration directly.
```cpp
usingnamespacestd::chrono_literals;
ws.set_connection_timeout(5s);
ws.set_read_timeout(30s);
ws.set_write_timeout(10s);
```
## What the read timeout means
`set_read_timeout()` applies to a single `read()` call. If no message arrives within that time, `read()` returns `ReadResult::Timeout`: **the connection is still open** and nothing was consumed, so you can send on it and read again. That is what separates it from `ReadResult::Fail`, which means the connection is gone.
This is what lets one thread own a connection in both directions:
```cpp
usingnamespacestd::chrono_literals;
ws.set_read_timeout(100ms);
std::stringmsg;
while(ws.is_open()){
autor=ws.read(msg);
if(r==httplib::ws::Timeout){
flush_outgoing(ws);// nothing arrived — send whatever is queued
continue;
}
if(r==httplib::ws::Fail){break;}
handle(msg);
}
```
Without a read timeout, `read()` blocks until a message arrives, so the thread holding the connection never gets to its writes.
Two things to know about `Timeout`:
- It leaves `msg` untouched, and it is non-zero. So `while (ws.read(msg))` is not usable once a read timeout is set — the loop would keep running with the *previous* message still in `msg`.
- It is only reported on a message boundary. If the timeout elapses partway through a fragmented message, that message cannot be resumed and `read()` returns `Fail`.
For connections where long idle periods are normal — waiting on notifications, for example — either leave the read timeout unset, or treat `Timeout` as the no-op it is and keep looping.
## On the server side
A handler's `ws::WebSocket` has `set_read_timeout()` too, and the pattern above is how a handler relays between connections instead of parking in `read()`.
The server default is 300s (`CPPHTTPLIB_WEBSOCKET_SERVER_READ_TIMEOUT_SECOND`) rather than "forever": it is a backstop that reclaims a worker from a peer that has gone silent, since a WebSocket handler holds its worker for the life of the connection.
Because it is a backstop rather than something the handler asked for, it does not surface as `Timeout`. When it elapses, `read()` returns `Fail` and closes the connection, so a handler written as `while (ws.read(msg))` ends the way it always has. Only a timeout the handler set itself with `set_read_timeout()` comes back as `Timeout`.
> Unresponsive-peer detection via Ping/Pong is a separate mechanism. See [W02. Set a WebSocket Heartbeat](../w02-websocket-ping) for details.
## How this differs from `Client`
For `Client`'s timeout configuration, see [C12. Set Timeouts](../c12-timeouts). The behavior and API are nearly identical, but `WebSocketClient` has no equivalent to `set_max_timeout()` for capping the whole request — once connected, the connection stays open for as long as you keep calling `read()`.
[cpp-httplib](https://github.com/yhirose/cpp-httplib) is an HTTP/HTTPS library for C++. Just copy a single header file, [`httplib.h`](https://github.com/yhirose/cpp-httplib/raw/refs/tags/latest/httplib.h), and you're ready to go.
When you need a quick HTTP server or client in C++, you want something that just works. That's exactly why I built cpp-httplib. You can start writing both servers and clients in just a few lines of code.
The API uses a lambda-based design that feels natural. It runs anywhere you have a C++11 or later compiler. Windows, macOS, Linux — use whatever environment you already have.
HTTPS works too. Just link OpenSSL or mbedTLS, and both server and client gain TLS support. Content-Encoding (gzip, Brotli, etc.), file uploads, and other features you actually need in real-world development are all included. WebSocket is also supported.
Under the hood, it uses blocking I/O with a thread pool. It's not built for handling massive numbers of simultaneous connections. But for API servers, embedded HTTP in tools, mock servers for testing, and many other use cases, it delivers solid performance.
"Solve today's problem, today." That's the kind of simplicity cpp-httplib aims for.
## Documentation
- [A Tour of cpp-httplib](tour/) — A step-by-step tutorial covering the basics. Start here if you're new
- [Building a Desktop LLM App](llm-app/) — A hands-on guide to building a desktop app with llama.cpp, step by step
## Stay Tuned
- [Cookbook](cookbook/) — A collection of recipes organized by topic. Jump to whatever you need
Let's incrementally build a text translation REST API server using llama.cpp as the inference engine. By the end, a request like this will return a translation result.
```bash
curl -X POST http://localhost:8080/translate \
-H "Content-Type: application/json"\
-d '{"text": "The weather is nice today. Shall we go for a walk?", "target_lang": "ja"}'
```
```json
{
"translation":"今日はいい天気ですね。散歩に行きましょうか？"
}
```
The "Translation API" is just one example. By swapping out the prompt, you can adapt this to any LLM application you like, such as summarization, code generation, or a chatbot.
Here's the full list of APIs the server will provide.
| Method | Path | Description | Chapter |
| -------- | ---- | ---- | -- |
| `GET` | `/health` | Returns server status | 1 |
| `POST` | `/translate` | Translates text and returns JSON | 2 |
| `POST` | `/translate/stream` | SSE streaming on a per-token basis | 3 |
| `GET` | `/models` | Model list (available / downloaded / selected) | 4 |
| `POST` | `/models/select` | Select a model (automatically downloads if not yet downloaded) | 4 |
In this chapter, let's set up the project environment. We'll fetch the dependency libraries, create the directory structure, configure the build settings, and grab the model file, so that we're ready to start writing code in the next chapter.
| [webview/webview](https://github.com/webview/webview) | Desktop WebView (used in Chapter 6) |
cpp-httplib, nlohmann/json, and cpp-llamalib are header-only libraries. You could just download a single header file with `curl` and `#include` it, but in this book we use CMake's `FetchContent` to fetch them automatically. Declare them in `CMakeLists.txt`, and `cmake -B build` downloads and builds everything for you. webview is used in Chapter 6, so you don't need to worry about it for now.
## 1.2 Directory Structure
The final structure will look like this.
```ascii
translate-app/
├── CMakeLists.txt
├── models/
│ └── (GGUF files)
└── src/
└── main.cpp
```
We don't include library source code in the project. CMake's `FetchContent` fetches them automatically at build time, so all you need is your own code.
Let's create the project directory and initialize a git repository.
```bash
mkdir translate-app &&cd translate-app
mkdir src models
git init
```
## 1.3 Obtaining the GGUF Model File
You need a model file for LLM inference. GGUF is the model format used by llama.cpp, and you can find many models on Hugging Face.
Let's start by trying a small model. The quantized version of Google's Gemma 2 2B (~1.6 GB) is a good starting point. It's lightweight but supports multiple languages and works well for translation tasks.
In Chapter 4, we'll add the ability to download models from within the app using cpp-httplib's client functionality.
## 1.4 CMakeLists.txt
Create a `CMakeLists.txt` in the project root. By declaring dependencies with `FetchContent`, CMake will automatically download and build them for you.
`FetchContent_Declare` tells CMake where to find each library, and `FetchContent_MakeAvailable` fetches and builds them. The first `cmake -B build` will take some time because it downloads all libraries and builds llama.cpp, but subsequent runs will use the cache.
Just link with `target_link_libraries`, and each library's CMake configuration sets up include paths and build settings for you.
## 1.5 Creating the Skeleton Code
We'll use this skeleton code as a base and add functionality chapter by chapter.
<!-- data-file="main.cpp" -->
```cpp
// src/main.cpp
#include<httplib.h>
#include<nlohmann/json.hpp>
#include<csignal>
#include<iostream>
usingjson=nlohmann::json;
httplib::Serversvr;
// Graceful shutdown on `Ctrl+C`
voidsignal_handler(intsig){
if(sig==SIGINT||sig==SIGTERM){
std::cout<<"\nReceived signal, shutting down gracefully...\n";
// Allow the server to be stopped with `Ctrl+C` (`SIGINT`) or `kill` (`SIGTERM`)
signal(SIGINT,signal_handler);
signal(SIGTERM,signal_handler);
// Start the server
std::cout<<"Listening on http://127.0.0.1:8080"<<std::endl;
svr.listen("127.0.0.1",8080);
}
```
## 1.6 Building and Verifying
Build the project, start the server, and verify that requests work with curl.
```bash
cmake -B build
cmake --build build -j
./build/translate-server
```
From another terminal, try it with curl.
```bash
curl http://localhost:8080/health
# => {"status":"ok"}
```
If you see JSON come back, the setup is complete.
## Next Chapter
Now that the environment is set up, in the next chapter we'll implement the translation REST API on top of this skeleton. We'll run inference with llama.cpp and expose it as an HTTP endpoint with cpp-httplib.
**Next:** [Integrating llama.cpp to Build a REST API](../ch02-rest-api)
title:"2. Integrating llama.cpp to Build a REST API"
order:2
---
In the skeleton from Chapter 1, `/translate` simply returned `"TODO"`. In this chapter we integrate llama.cpp inference and turn it into an API that actually returns translation results.
Calling the llama.cpp API directly makes the code quite long, so we use a thin wrapper library called [cpp-llamalib](https://github.com/yhirose/cpp-llamalib). It lets you load a model and run inference in just a few lines, keeping the focus on cpp-httplib.
## 2.1 Initializing the LLM
Simply pass the path to a model file to `llamalib::Llama`, and model loading, context creation, and sampler configuration are all taken care of. If you downloaded a different model in Chapter 1, adjust the path accordingly.
`llm.chat()` can throw exceptions during inference (for example, when the context length is exceeded). By catching them with `try/catch` and returning the error as JSON, we prevent the server from crashing.
## 2.3 Complete Code
Here is the finished code with all the changes so far.
// Allow the server to be stopped with `Ctrl+C` (`SIGINT`) or `kill` (`SIGTERM`)
signal(SIGINT,signal_handler);
signal(SIGTERM,signal_handler);
// Start the server (blocks until `stop()` is called)
std::cout<<"Listening on http://127.0.0.1:8080"<<std::endl;
svr.listen("127.0.0.1",8080);
}
```
</details>
## 2.4 Testing It Out
Rebuild and start the server, then verify that it now returns actual translation results.
```bash
cmake --build build -j
./build/translate-server
```
```bash
curl -X POST http://localhost:8080/translate \
-H "Content-Type: application/json"\
-d '{"text": "I had a great time visiting Tokyo last spring. The cherry blossoms were beautiful.", "target_lang": "ja"}'
# => {"translation":"去年の春に東京を訪れた。桜が綺麗だった。"}
```
In Chapter 1 the response was `"TODO"`, but now you get an actual translation back.
## Next Chapter
The REST API we built in this chapter waits for the entire translation to complete before sending the response, so for long texts the user has to wait with no indication of progress.
In the next chapter, we use SSE (Server-Sent Events) to stream tokens back in real time as they are generated.
**Next:** [Adding Token Streaming with SSE](../ch03-sse-streaming)
The `/translate` endpoint from Chapter 2 returned the entire translation at once after completion. This is fine for short sentences, but for longer text the user has to wait several seconds with nothing displayed.
In this chapter, we add a `/translate/stream` endpoint that uses SSE (Server-Sent Events) to return tokens in real time as they are generated. This is the same approach used by the ChatGPT and Claude APIs.
## 3.1 What is SSE?
SSE is a way to send HTTP responses as a stream. When a client sends a request, the server keeps the connection open and gradually returns events. The format is simple text.
```text
data: "去年の"
data: "春に"
data: "東京を"
data: [DONE]
```
Each line starts with `data:` and events are separated by blank lines. The Content-Type is `text/event-stream`. Tokens are sent as escaped JSON strings, so they appear enclosed in double quotes (we implement this in Section 3.3).
## 3.2 Streaming with cpp-httplib
In cpp-httplib, you can use `set_chunked_content_provider` to send responses incrementally. Each time you write to `sink.os` inside the callback, data is sent to the client.
```cpp
res.set_chunked_content_provider(
"text/event-stream",
[](size_toffset,httplib::DataSink&sink){
sink.os<<"data: hello\n\n";
sink.done();
returntrue;
});
```
Calling `sink.done()` ends the stream. If the client disconnects mid-stream, writing to `sink.os` will fail and `sink.os.fail()` will return `true`. You can use this to detect disconnection and abort unnecessary inference.
## 3.3 The `/translate/stream` Handler
JSON parsing and validation are the same as the `/translate` endpoint from Chapter 2. The only difference is how the response is returned. We combine the streaming callback of `llm.chat()` with `set_chunked_content_provider`.
- When you pass a callback to `llm.chat()`, it is called each time a token is generated. If the callback returns `false`, generation is aborted
- After writing to `sink.os`, you can check whether the client is still connected with `sink.os.good()`. If the client has disconnected, it returns `false` to stop inference
- Each token is escaped as a JSON string using `json(token).dump()` before sending. This is safe even for tokens containing newlines or quotes
- The first three arguments of `dump(-1, ' ', false, ...)` are the defaults. What matters is the fourth argument, `json::error_handler_t::replace`. Since the LLM returns tokens at the subword level, multi-byte characters (such as Japanese) can be split mid-character across tokens. Passing an incomplete UTF-8 byte sequence directly to `dump()` would throw an exception, so `replace` safely substitutes them. The browser reassembles the bytes on its end, so everything displays correctly
- The entire lambda is wrapped in `try/catch`. `llm.chat()` can throw exceptions for reasons such as exceeding the context window. If an exception goes uncaught inside the lambda, the server will crash, so we return the error as an SSE event instead
-`data: [DONE]` follows the OpenAI API convention to signal the end of the stream to the client
## 3.4 Complete Code
Here is the complete code with the `/translate/stream` endpoint added to the code from Chapter 2.
// Allow the server to be stopped with `Ctrl+C` (`SIGINT`) or `kill` (`SIGTERM`)
signal(SIGINT,signal_handler);
signal(SIGTERM,signal_handler);
// Start the server (blocks until `stop()` is called)
std::cout<<"Listening on http://127.0.0.1:8080"<<std::endl;
svr.listen("127.0.0.1",8080);
}
```
</details>
## 3.5 Testing It Out
Build and start the server.
```bash
cmake --build build -j
./build/translate-server
```
Using curl's `-N` option to disable buffering, you can see tokens displayed in real time as they arrive.
```bash
curl -N -X POST http://localhost:8080/translate/stream \
-H "Content-Type: application/json"\
-d '{"text": "I had a great time visiting Tokyo last spring. The cherry blossoms were beautiful.", "target_lang": "ja"}'
```
```text
data: "去年の"
data: "春に"
data: "東京を"
data: "訪れた"
data: "。"
data: "桜が"
data: "綺麗だった"
data: "。"
data: [DONE]
```
You should see tokens streaming in one by one. The `/translate` endpoint from Chapter 2 continues to work as well.
## Next Chapter
The server's translation functionality is now complete. In the next chapter, we use cpp-httplib's client functionality to add the ability to fetch and manage models from Hugging Face.
**Next:** [Adding Model Download and Management](../ch04-model-management)
By the end of Chapter 3, the server's translation functionality was fully in place. However, the only model file available is the one we manually downloaded in Chapter 1. In this chapter, we'll use cpp-httplib's **client functionality** to enable downloading and switching Hugging Face models from within the app.
Once complete, you'll be able to manage models with requests like these:
# Select a different model (automatically downloads if not yet available)
curl -N -X POST http://localhost:8080/models/select \
-H "Content-Type: application/json"\
-d '{"model": "gemma-2-9b-it"}'
```
```text
data: {"status":"downloading","progress":0}
data: {"status":"downloading","progress":12}
...
data: {"status":"downloading","progress":100}
data: {"status":"loading"}
data: {"status":"ready"}
```
## 4.1 httplib::Client Basics
So far we've only used `httplib::Server`, but cpp-httplib also provides client functionality. Since Hugging Face uses HTTPS, we need a TLS-capable client.
```cpp
#include<httplib.h>
// Including the URL scheme automatically uses SSLClient
httplib::Clientcli("https://huggingface.co");
// Automatically follow redirects (Hugging Face redirects to a CDN)
cli.set_follow_location(true);
autores=cli.Get("/api/models");
if(res&&res->status==200){
std::cout<<res->body<<std::endl;
}
```
To use HTTPS, you need to enable OpenSSL at build time. Add the following to your `CMakeLists.txt`:
Defining `CPPHTTPLIB_OPENSSL_SUPPORT` enables `httplib::Client("https://...")` to make TLS connections. On macOS, you also need to link the CoreFoundation and Security frameworks to access the system certificate store. See Section 4.8 for the complete `CMakeLists.txt`.
## 4.2 Defining the Model List
Let's define the list of models that the app can handle. Here are four models we've verified for translation tasks.
Up through Chapter 3, we stored models in the `models/` directory within the project. However, when managing multiple models, a dedicated app directory makes more sense. On macOS/Linux we use `~/.translate-app/models/`, and on Windows we use `%APPDATA%\translate-app\models\`.
If the environment variable isn't set, it falls back to the current directory. The app creates this directory at startup (`create_directories` won't error even if it already exists).
## 4.4 Rewriting Model Initialization
We rewrite the model initialization at the beginning of `main()`. In Chapter 1 we hardcoded the path, but from here on we support model switching. We track the currently loaded filename in `selected_model` and load the first entry in `MODELS` at startup. The `GET /models` and `POST /models/select` handlers reference and update this variable.
Since cpp-httplib runs handlers concurrently on a thread pool, reassigning `llm` while another thread is calling `llm.chat()` would crash. We add a `std::mutex` to protect against this.
```cpp
intmain(){
automodels_dir=get_models_dir();
std::filesystem::create_directories(models_dir);
std::stringselected_model=MODELS[0].filename;
autopath=models_dir/selected_model;
// Automatically download the default model if not yet present
std::cerr<<"\nFailed to download model."<<std::endl;
return1;
}
std::cout<<std::endl;
}
autollm=llamalib::Llama{path};
std::mutexllm_mutex;// Protect access during model switching
// ...
}
```
This ensures that users don't need to manually download models with curl on first launch. It uses the `download_model` function from Section 4.6 and displays progress on the console.
## 4.5 The `GET /models` Handler
This returns the model list with information about whether each model has been downloaded and whether it's currently selected.
GGUF models are several gigabytes, so we can't load the entire file into memory. By passing callbacks to `httplib::Client::Get`, we can receive data chunk by chunk.
```cpp
// content_receiver: callback that receives data chunks
// progress: download progress callback
cli.Get(url,
[&](constchar*data,size_tlen){// content_receiver
ofs.write(data,len);
returntrue;// returning false aborts the download
},
[&](size_tcurrent,size_ttotal){// progress
intpct=total?(int)(current*100/total):0;
std::cout<<pct<<"%"<<std::endl;
returntrue;// returning false aborts the download
});
```
Let's use this to create a function that downloads models from Hugging Face.
```cpp
#include<filesystem>
#include<fstream>
// Download a model and report progress via progress_cb.
// If progress_cb returns false, the download is aborted.
- We send SSE events directly from the `download_model` progress callback. This is an application of `set_chunked_content_provider` + `sink.os` from Chapter 3
- Since the callback returns `sink.os.good()`, the download stops if the client disconnects. The cancel button we add in Chapter 5 uses this
- When we update `selected_model`, it's reflected in the `selected` flag of `GET /models`
- The `llm` reassignment is protected by `llm_mutex`. The `/translate` and `/translate/stream` handlers also lock the same mutex, so inference can't run during a model switch (see the complete code)
## 4.8 Complete Code
Here is the complete code with model management added to the Chapter 3 code.
// Allow the server to be stopped with `Ctrl+C` (`SIGINT`) or `kill` (`SIGTERM`)
signal(SIGINT,signal_handler);
signal(SIGTERM,signal_handler);
std::cout<<"Listening on http://127.0.0.1:8080"<<std::endl;
svr.listen("127.0.0.1",8080);
}
```
</details>
## 4.9 Testing
Since we added OpenSSL configuration to CMakeLists.txt, we need to re-run CMake before building.
```bash
cmake -B build
cmake --build build -j
./build/translate-server
```
### Checking the Model List
```bash
curl http://localhost:8080/models
```
The gemma-2-2b-it model downloaded in Chapter 1 should show `downloaded: true` and `selected: true`.
### Switching to a Different Model
```bash
curl -N -X POST http://localhost:8080/models/select \
-H "Content-Type: application/json"\
-d '{"model": "gemma-2-9b-it"}'
```
Download progress streams via SSE, and `"ready"` appears when it's done.
### Comparing Translations Across Models
Let's translate the same sentence with different models.
```bash
# Translate with gemma-2-9b-it (the model we just switched to)
curl -X POST http://localhost:8080/translate \
-H "Content-Type: application/json"\
-d '{"text": "The quick brown fox jumps over the lazy dog.", "target_lang": "ja"}'
# Switch back to gemma-2-2b-it
curl -N -X POST http://localhost:8080/models/select \
-H "Content-Type: application/json"\
-d '{"model": "gemma-2-2b-it"}'
# Translate the same sentence
curl -X POST http://localhost:8080/translate \
-H "Content-Type: application/json"\
-d '{"text": "The quick brown fox jumps over the lazy dog.", "target_lang": "ja"}'
```
Translation results vary depending on the model, even with the same code and the same prompt. Since cpp-llamalib automatically applies the appropriate chat template for each model, no code changes are needed.
## Next Chapter
The server's main features are now complete: REST API, SSE streaming, and model download and switching. In the next chapter, we'll add static file serving and build a Web UI you can use from a browser.
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
yhirose
