From ef1cdb59acb246c7ef8f48b25ed22ae41f53511b Mon Sep 17 00:00:00 2001 From: Niels Lohmann Date: Wed, 21 Oct 2026 12:25:10 +0200 Subject: [PATCH] Update performance docs for fast std::istream and FILE* parsing (#5875) performance.md still claimed streams have no fast path (outdated since #5828). Signed-off-by: Niels Lohmann --- docs/mkdocs/docs/features/performance.md | 45 ++++++++++++++---------- 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/docs/mkdocs/docs/features/performance.md b/docs/mkdocs/docs/features/performance.md index ad24bd069..a15fbf38b 100644 --- a/docs/mkdocs/docs/features/performance.md +++ b/docs/mkdocs/docs/features/performance.md @@ -20,32 +20,39 @@ below apply on every supported standard. This adapter exposes two optimizations - it can reconstruct already-consumed input on demand for error messages, instead of copying every character as it is read, and -- the lexer can scan ordinary string characters directly out of the buffer, several bytes at a time, rather than one - character (and one function call) at a time. +- the lexer can scan ordinary string characters, numbers, and indentation directly out of the buffer, several bytes at + a time, rather than one character (and one function call) at a time. -A `#!cpp std::istream` (including `#!cpp std::ifstream`) or `#!cpp FILE*`, by contrast, is read through -`input_stream_adapter` or `file_input_adapter`, which read one character (or one block, for binary formats) at a time -and expose neither optimization -- the lexer falls back to the same byte-at-a-time path it uses for any -non-contiguous, general-purpose iterator range. An iterator pair over non-contiguous but random-access storage (e.g. -`#!cpp std::deque::iterator`) gets the first optimization but not the second, since the byte-scanning fast path -additionally requires contiguous storage. +A `#!cpp std::istream` (including `#!cpp std::ifstream`) gets the same bulk scanning: `input_stream_adapter` scans the +characters its stream buffer already holds (the *get area*) and asks the stream buffer for more only when they are used +up. Nothing is read ahead, so the stream is left exactly where it would be after reading one character at a time, and +[`operator>>`](../api/operator_gtgt.md) can still read several values from one stream. A stream buffer without a get +area is read one character at a time. A `#!cpp FILE*` is still read one character at a time, but `file_input_adapter` +locks the file once per parse instead of once per character. See +[input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. According to the +description of [#5828](https://github.com/nlohmann/json/pull/5828), parsing from a stream is within about 10 to 20 % of +parsing an equivalent `#!cpp std::string`. -Practically: if the JSON text is already in memory, or small enough to read into memory, prefer passing a -`#!cpp std::string`, a `#!cpp std::vector`, or a pointer range to `parse` over a `#!cpp std::istream`. For a -file, that means reading it into a string first and then parsing the string, rather than passing a -`#!cpp std::ifstream` directly to `parse` -- the latter never benefits from either optimization: +Inputs that are still read one character at a time: iterator ranges that are not contiguous (e.g. +`#!cpp std::list` or `#!cpp std::istreambuf_iterator`), wide strings, and user-defined adapters. An +iterator pair over non-contiguous but random-access storage (e.g. `#!cpp std::deque::iterator`) gets the first +optimization but not the second, since the byte-scanning fast path requires contiguous storage. + +Practically: a file can be passed directly to `parse`, which also avoids the memory for a copy of the file. Reading it +into a string first is no longer necessary for speed: ```cpp -// gets the contiguous fast paths +// fast: the stream buffer's get area is scanned in bulk std::ifstream f("example.json"); -std::string contents((std::istreambuf_iterator(f)), std::istreambuf_iterator()); -json j = json::parse(contents); - -// does not: input_stream_adapter has no fast path -std::ifstream f2("example.json"); -json j2 = json::parse(f2); +json j = json::parse(f); ``` +Two macros switch the stream optimizations off without changing any result: +[`JSON_DISABLE_STREAM_BULK_SCAN`](../api/macros/json_disable_stream_bulk_scan.md) for stream buffers that do not follow +the usual `#!cpp std::streambuf` contract, and +[`JSON_DISABLE_UNLOCKED_FILE_IO`](../api/macros/json_disable_unlocked_file_io.md) for programs in which another thread +must use the same `#!cpp FILE` object while it is parsed. + For contiguous input with many non-ASCII characters, [`JSON_USE_SIMDUTF`](../api/macros/json_use_simdutf.md) can additionally speed up UTF-8 validation by using the [simdutf](https://github.com/simdutf/simdutf) library instead of the built-in scalar validator; streaming inputs (files, `#!cpp std::istream`, wide strings, user-defined adapters)