Merge branch develop into claude/perf-contiguous-parser

Resolve the conflict in docs/mkdocs/docs/features/performance.md: keep
the description of the FILE* bulk scan from #6030 and the stream timing
from this branch.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-11-07 10:20:09 +01:00
commit c1104d20b8
8 files changed
+582 -11

No files matched your search

@@ -5,7 +5,9 @@
```
When defined to `1`, parsing from a `#!cpp FILE*` reads every character with `std::fgetc`, as in version 3.12.0 and
earlier, instead of locking the file once and reading it with the unlocked stdio functions.
earlier, instead of locking the file once and reading it with the unlocked stdio functions. This also turns off the
scan of the `FILE`'s buffer in bulk (see below), which [`JSON_DISABLE_STREAM_BULK_SCAN`](json_disable_stream_bulk_scan.md)
does not affect.
It does not change any result: values, exceptions, and the position of the file after parsing are the same either way.
Only the speed and the locking of the file differ (see the notes on locking below).
@@ -37,9 +39,17 @@ The default value is `0`: the file is locked once and read with the unlocked std
| Microsoft (MSVC, clang-cl) | `_lock_file`, `_fgetc_nolock`, `_unlock_file` |
| others (e.g., musl, MinGW, newlib) | `std::fgetc`, as before |
This makes parsing from a `FILE*` 3 to 4 times as fast with the Apple C library and 1.1 to 1.3 times as fast with
glibc, whose locking is cheaper. The binary formats ([`from_cbor`](../basic_json/from_cbor.md),
[`from_msgpack`](../basic_json/from_msgpack.md), ...) read from a `FILE*` the same way.
With glibc and the Apple C library, the parser also scans the characters that the `FILE` already holds in its
buffer in bulk, as it scans the get area of a stream (see
[`JSON_DISABLE_STREAM_BULK_SCAN`](json_disable_stream_bulk_scan.md)): it reads the same fields of the `FILE` that
the C library's own `getc_unlocked` reads, and it consumes exactly the characters that `getc_unlocked` would have
returned, so the file is left where it was left before.
Locking the file once makes parsing from a `FILE*` 3 to 4 times as fast with the Apple C library and 1.1 to 1.3
times as fast with glibc, whose locking is cheaper. Scanning the buffer in bulk makes `parse()` another 1.7 to 2.6
times as fast (and `accept()` 2.9 to 4.4 times), about as fast as parsing from a `std::ifstream`. The binary
formats ([`from_cbor`](../basic_json/from_cbor.md), [`from_msgpack`](../basic_json/from_msgpack.md), ...) read
from a `FILE*` the same way.
!!! note "Locking"
+1 -1
View File
@@ -77,7 +77,7 @@ them faster than others:
|--------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
| strings, string views, string literals, contiguous containers, pointer or iterator ranges over contiguous memory | the whole input is scanned in bulk: runs of string characters, numbers, and indentation are recognized several bytes at a time |
| `#!cpp std::istream` (`#!cpp std::ifstream`, `#!cpp std::istringstream`, ...) | the characters the stream buffer holds are scanned in bulk like a string; the stream buffer is asked for more when they are used up |
| `#!cpp FILE*` | one character at a time, with the file locked once for the whole parse |
| `#!cpp FILE*` | with glibc and the Apple C library, the characters the `FILE` holds in its buffer are scanned in bulk like a stream's; elsewhere one character at a time; the file is locked once for the whole parse |
| other iterator ranges (e.g., `#!cpp std::list<char>`, `#!cpp std::istreambuf_iterator<char>`), wide strings | one character at a time |
Input in one block of memory is read even faster with the default options (no
+2 -2
View File
@@ -27,8 +27,8 @@ A `#!cpp std::istream` (including `#!cpp std::ifstream`) gets the same bulk scan
characters its stream buffer already holds (the *get area*) and asks the stream buffer for more only when they are used
up. Nothing is read ahead, so the stream is left exactly where it would be after reading one character at a time, and
[`operator>>`](../api/operator_gtgt.md) can still read several values from one stream. A stream buffer without a get
area is read one character at a time. A `#!cpp FILE*` is still read one character at a time, but `file_input_adapter`
locks the file once per parse instead of once per character. See
area is read one character at a time. A `#!cpp FILE*` is locked once per parse instead of once per character, and with
glibc and the Apple C library, the characters in its buffer are scanned in bulk as a stream's get area is. See
[input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. Parsing from a
stream takes about 1.15 to 1.45 times as long as parsing an equivalent `#!cpp std::string`, which `parse()` reads with
a parser of its own (see there).