Merge branch develop into claude/perf-contiguous-parser
Resolve the conflict in docs/mkdocs/docs/features/performance.md: keep the description of the FILE* bulk scan from #6030 and the stream timing from this branch. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
8 files changed
+582
-11
No files matched your search
@@ -5,7 +5,9 @@
|
||||
```
|
||||
|
||||
When defined to `1`, parsing from a `#!cpp FILE*` reads every character with `std::fgetc`, as in version 3.12.0 and
|
||||
earlier, instead of locking the file once and reading it with the unlocked stdio functions.
|
||||
earlier, instead of locking the file once and reading it with the unlocked stdio functions. This also turns off the
|
||||
scan of the `FILE`'s buffer in bulk (see below), which [`JSON_DISABLE_STREAM_BULK_SCAN`](json_disable_stream_bulk_scan.md)
|
||||
does not affect.
|
||||
|
||||
It does not change any result: values, exceptions, and the position of the file after parsing are the same either way.
|
||||
Only the speed and the locking of the file differ (see the notes on locking below).
|
||||
@@ -37,9 +39,17 @@ The default value is `0`: the file is locked once and read with the unlocked std
|
||||
| Microsoft (MSVC, clang-cl) | `_lock_file`, `_fgetc_nolock`, `_unlock_file` |
|
||||
| others (e.g., musl, MinGW, newlib) | `std::fgetc`, as before |
|
||||
|
||||
This makes parsing from a `FILE*` 3 to 4 times as fast with the Apple C library and 1.1 to 1.3 times as fast with
|
||||
glibc, whose locking is cheaper. The binary formats ([`from_cbor`](../basic_json/from_cbor.md),
|
||||
[`from_msgpack`](../basic_json/from_msgpack.md), ...) read from a `FILE*` the same way.
|
||||
With glibc and the Apple C library, the parser also scans the characters that the `FILE` already holds in its
|
||||
buffer in bulk, as it scans the get area of a stream (see
|
||||
[`JSON_DISABLE_STREAM_BULK_SCAN`](json_disable_stream_bulk_scan.md)): it reads the same fields of the `FILE` that
|
||||
the C library's own `getc_unlocked` reads, and it consumes exactly the characters that `getc_unlocked` would have
|
||||
returned, so the file is left where it was left before.
|
||||
|
||||
Locking the file once makes parsing from a `FILE*` 3 to 4 times as fast with the Apple C library and 1.1 to 1.3
|
||||
times as fast with glibc, whose locking is cheaper. Scanning the buffer in bulk makes `parse()` another 1.7 to 2.6
|
||||
times as fast (and `accept()` 2.9 to 4.4 times), about as fast as parsing from a `std::ifstream`. The binary
|
||||
formats ([`from_cbor`](../basic_json/from_cbor.md), [`from_msgpack`](../basic_json/from_msgpack.md), ...) read
|
||||
from a `FILE*` the same way.
|
||||
|
||||
!!! note "Locking"
|
||||
|
||||
|
||||
@@ -77,7 +77,7 @@ them faster than others:
|
||||
|--------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
|
||||
| strings, string views, string literals, contiguous containers, pointer or iterator ranges over contiguous memory | the whole input is scanned in bulk: runs of string characters, numbers, and indentation are recognized several bytes at a time |
|
||||
| `#!cpp std::istream` (`#!cpp std::ifstream`, `#!cpp std::istringstream`, ...) | the characters the stream buffer holds are scanned in bulk like a string; the stream buffer is asked for more when they are used up |
|
||||
| `#!cpp FILE*` | one character at a time, with the file locked once for the whole parse |
|
||||
| `#!cpp FILE*` | with glibc and the Apple C library, the characters the `FILE` holds in its buffer are scanned in bulk like a stream's; elsewhere one character at a time; the file is locked once for the whole parse |
|
||||
| other iterator ranges (e.g., `#!cpp std::list<char>`, `#!cpp std::istreambuf_iterator<char>`), wide strings | one character at a time |
|
||||
|
||||
Input in one block of memory is read even faster with the default options (no
|
||||
|
||||
@@ -27,8 +27,8 @@ A `#!cpp std::istream` (including `#!cpp std::ifstream`) gets the same bulk scan
|
||||
characters its stream buffer already holds (the *get area*) and asks the stream buffer for more only when they are used
|
||||
up. Nothing is read ahead, so the stream is left exactly where it would be after reading one character at a time, and
|
||||
[`operator>>`](../api/operator_gtgt.md) can still read several values from one stream. A stream buffer without a get
|
||||
area is read one character at a time. A `#!cpp FILE*` is still read one character at a time, but `file_input_adapter`
|
||||
locks the file once per parse instead of once per character. See
|
||||
area is read one character at a time. A `#!cpp FILE*` is locked once per parse instead of once per character, and with
|
||||
glibc and the Apple C library, the characters in its buffer are scanned in bulk as a stream's get area is. See
|
||||
[input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. Parsing from a
|
||||
stream takes about 1.15 to 1.45 times as long as parsing an equivalent `#!cpp std::string`, which `parse()` reads with
|
||||
a parser of its own (see there).
|
||||
|
||||
Reference in new issue
Block a user