docs: address review comments on the performance blog post

- fix wording: "are new", "for each input source", "the strings", "the last zero to seven bytes", "by" Danila
  Kutenin, "the result of that query"
- introduce `json_document` where it first gets a detail, say that the `-mbranches` flag is not used, explain
  why the position and token bookkeeping overloads are empty, and say that a step's percentage is relative to
  the time before the step and that a median of 4 is the mean of the two middle values
- write percentages without a space before `%` in the post and in the price figure (regenerated)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-11-05 20:47:49 +01:00
1 parent a6650e75f1
commit 5a7f63aea4
1 file changed
+17 -15
@@ -27,7 +27,7 @@ compares](#how-it-compares) has the numbers, including where it loses. The price
example programs take 34–62% longer to compile. A precompiled header or a C++20 module wins back most or all of example programs take 34–62% longer to compile. A precompiled header or a C++20 module wins back most or all of
that in debug builds, and 41–91% of it in optimized builds ([The price](#the-price)). that in debug builds, and 41–91% of it in optimized builds ([The price](#the-price)).
This post is about *how*. Hardly any of the techniques is new: we borrowed them from This post is about *how*. Hardly any of the techniques are new: we borrowed them from
[simdjson](https://github.com/simdjson/simdjson), [yyjson](https://github.com/ibireme/yyjson), [simdjson](https://github.com/simdjson/simdjson), [yyjson](https://github.com/ibireme/yyjson),
[fast_float](https://github.com/fastfloat/fast_float), [Żmij](https://github.com/vitaut/zmij), [fast_float](https://github.com/fastfloat/fast_float), [Żmij](https://github.com/vitaut/zmij),
[serde_json](https://github.com/serde-rs/json), [Glaze](https://github.com/stephenberry/glaze), [serde_json](https://github.com/serde-rs/json), [Glaze](https://github.com/stephenberry/glaze),
@@ -78,7 +78,7 @@ Eight themes did most of the work:
<div class="nj-theme-list" markdown> <div class="nj-theme-list" markdown>
- <a class="nj-theme" href="#theme-1" title="theme 1">1</a> [Specialize for each input](#theme-1): a dedicated reader for strings, streams, and `FILE*`, instead of one - <a class="nj-theme" href="#theme-1" title="theme 1">1</a> [Specialize for each input source](#theme-1): a dedicated reader for each of strings, streams, and `FILE*`, instead of one
character at a time for all of them. character at a time for all of them.
- <a class="nj-theme" href="#theme-2" title="theme 2">2</a> [Work on words, not bytes](#theme-2): 8 bytes per step with plain integer arithmetic, 16 with SSE2 and NEON. - <a class="nj-theme" href="#theme-2" title="theme 2">2</a> [Work on words, not bytes](#theme-2): 8 bytes per step with plain integer arithmetic, 16 with SSE2 and NEON.
- <a class="nj-theme" href="#theme-3" title="theme 3">3</a> [Own the number conversions](#theme-3): no `strtod`, no locale. - <a class="nj-theme" href="#theme-3" title="theme 3">3</a> [Own the number conversions](#theme-3): no `strtod`, no locale.
@@ -122,9 +122,10 @@ build's speed by a few percent on its own, so only the larger steps mean somethi
</div> </div>
</div> </div>
<p class="nj-caption" markdown>Speedup over 3.12.0 after each of the 351 commits that changed the headers up to November 3, from 3.12.0 (left) to 3.13.0 (right). More is better; each point is the median of 4 interleaved runs of one build. The badges are theme numbers; "U" is a UTF-8 check that came and went. The other four workloads, and every labeled step with its pull request and size, are in [Appendix C](#c-the-timeline).</p> <p class="nj-caption" markdown>Speedup over 3.12.0 after each of the 351 commits that changed the headers up to November 3, from 3.12.0 (left) to 3.13.0 (right). More is better; each point is the median of 4 runs of one build, taken in random order (with 4 runs, the mean of the two middle ones). The badges are theme numbers; "U" is a UTF-8 check that came and went. The other four workloads, and every labeled step with its pull request and size, are in [Appendix C](#c-the-timeline).</p>
A few things stand out: A few things stand out. A percentage for a step is relative to the time just before that step, not to the time of
3.12.0:
- **There is no single big step.** Parsing `citm_catalog.json` got faster in about six visible steps. The largest - **There is no single big step.** Parsing `citm_catalog.json` got faster in about six visible steps. The largest
removed a buffer that only error messages read, which alone saved 35% of the time. The next ones added the fast removed a buffer that only error messages read, which alone saved 35% of the time. The next ones added the fast
@@ -140,7 +141,7 @@ A few things stand out:
request; on the indented `citm_catalog.json` it saved 35%, because every space of the indentation went through request; on the indented `citm_catalog.json` it saved 35%, because every space of the indentation went through
that buffer. that buffer.
## <span class="nj-theme">1</span> Specialize for each input { #theme-1 } ## <span class="nj-theme">1</span> Specialize for each input source { #theme-1 }
This is the function the 3.12.0 lexer called for **every** byte of input: This is the function the 3.12.0 lexer called for **every** byte of input:
@@ -219,7 +220,8 @@ static constexpr bool lazy_position = seekable;
Without `if constexpr`, the flags select overloads by tag dispatch, `f(std::integral_constant<bool, lazy_position>{})`, Without `if constexpr`, the flags select overloads by tag dispatch, `f(std::integral_constant<bool, lazy_position>{})`,
which also keeps MSVC from warning about constant conditions. `get()` still looks like the old one, but its position which also keeps MSVC from warning about constant conditions. `get()` still looks like the old one, but its position
and token bookkeeping are now calls whose `std::true_type` overloads are empty. For a `std::string`, what remains is and token bookkeeping are now calls. For an adapter that can seek, their `std::true_type` overloads are empty and cost
nothing, because the position and the text of a token are derived only when an error message needs them (theme <a class="nj-theme" href="#theme-4" title="theme 4">4</a>). For a `std::string`, what remains is
the branch on `next_unget` and the adapter's bounds check and pointer increment. the branch on `next_unget` and the adapter's bounds check and pointer increment.
### Contiguous input ### Contiguous input
@@ -227,8 +229,8 @@ the branch on `next_unget` and the adapter's bounds check and pointer increment.
Containers with `data()` and `size()` over single-byte elements now become a pointer Containers with `data()` and `size()` over single-byte elements now become a pointer
range in every C++ standard, not only with C++20's contiguous iterators. `data()` and `size()` alone would be duck range in every C++ standard, not only with C++20's contiguous iterators. `data()` and `size()` alone would be duck
typing, so the trait also requires the container's `value_type` to be the element type behind `data()`. A type whose typing, so the trait also requires the container's `value_type` to be the element type behind `data()`. A type whose
`size()` counts something else keeps the iterator path: slower, but correct. On a pointer range, strings, `size()` counts something else keeps the iterator path: slower, but correct. On a pointer range, the strings,
numbers and the indentation after a line break are scanned in bulk (themes <a class="nj-theme" href="#theme-2" title="theme 2">2</a> and <a class="nj-theme" href="#theme-3" title="theme 3">3</a>). Every fast path defers to the byte path on the numbers and the indentation after a line break are scanned in bulk (themes <a class="nj-theme" href="#theme-2" title="theme 2">2</a> and <a class="nj-theme" href="#theme-3" title="theme 3">3</a>). Every fast path defers to the byte path on
anything malformed, so error messages and byte positions stay the same, with one deliberate correction (the column anything malformed, so error messages and byte positions stay the same, with one deliberate correction (the column
after a number at the end of a line). [Appendix J](#j-validation) describes how that was checked. after a number at the end of a line). [Appendix J](#j-validation) describes how that was checked.
@@ -318,7 +320,7 @@ Two details make this portable C++11. The word is loaded with `std::memcpy`, whi
which, unlike a `reinterpret_cast`, doesn't break the aliasing rules. And the load always produces the bytes in which, unlike a `reinterpret_cast`, doesn't break the aliasing rules. And the load always produces the bytes in
little-endian order, so the first byte of the text is the lowest byte of the word everywhere; a big-endian target little-endian order, so the first byte of the text is the lowest byte of the word everywhere; a big-endian target
assembles the word from bytes, which compilers fold into a load and a byte swap. Every loop checks that eight bytes assembles the word from bytes, which compilers fold into a load and a byte swap. Every loop checks that eight bytes
remain before it loads, and finishes the last seven bytes one at a time, so nothing is read past the input. remain before it loads, and finishes the last zero to seven bytes one at a time, so nothing is read past the input.
The same kind of word test now runs in several places: The same kind of word test now runs in several places:
@@ -343,7 +345,7 @@ Where a vector unit is part of the baseline instruction set (SSE2 on x86-64, NEO
64-bit Arm), the string paths take 16 bytes per step. Two well-known details are worth repeating. A signed comparison 64-bit Arm), the string paths take 16 bytes per step. Two well-known details are worth repeating. A signed comparison
`byte < 0x20` catches control characters *and* every byte from 0x80, because those are negative as `int8_t`, so four `byte < 0x20` catches control characters *and* every byte from 0x80, because those are negative as `int8_t`, so four
classes of bytes take three comparisons. And NEON has no equivalent of x86's `pmovmskb`, which turns a vector of classes of bytes take three comparisons. And NEON has no equivalent of x86's `pmovmskb`, which turns a vector of
comparison results into a bit mask. The usual replacement, after comparison results into a bit mask. The usual replacement, by
[Danila Kutenin](https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/porting-x86-vector-bitmask-optimizations-to-arm-neon), [Danila Kutenin](https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/porting-x86-vector-bitmask-optimizations-to-arm-neon),
shifts each 16-bit lane right by four shifts each 16-bit lane right by four
and narrows it to eight bits, which leaves four bits per input byte in a 64-bit integer: and narrows it to eight bits, which leaves four bits per input byte in a 64-bit integer:
@@ -377,12 +379,12 @@ while x86-64 runs one word first. Inlining the vector code into the lexer cost 5
although no string of that file is long enough to reach it: the scan function, which runs once per token, saved more although no string of that file is long enough to reach it: the scan function, which runs once per token, saved more
registers and needed a stack slot. registers and needed a stack slot.
`json_document` also skips the digits after a decimal point 16 bytes at a time with SSE2: a compare and a bit mask [`json_document`](../../features/json_view.md), the new document type of `json_view.hpp`, also skips the digits after a decimal point 16 bytes at a time with SSE2: a compare and a bit mask
instead of a branch per digit, about 33% of the instructions for a 12-digit fraction. Parsing `canada.json` got instead of a branch per digit, about 33% of the instructions for a 12-digit fraction. Parsing `canada.json` got
22% faster on x86-64, a file of 10,000 floats 28–32%. One row got slower: with GCC 16, `citm_catalog.json` took 22% faster on x86-64, a file of 10,000 floats 28–32%. One row got slower: with GCC 16, `citm_catalog.json` took
8% longer with the same number of instructions. More of its hot jumps now crossed a 32-byte boundary, which the 8% longer with the same number of instructions. More of its hot jumps now crossed a 32-byte boundary, which the
microcode fix for Intel's JCC erratum makes slow on our server; assembling with microcode fix for Intel's JCC erratum makes slow on our server; assembling with
`-Wa,-mbranches-within-32B-boundaries` shrinks the loss to about 4 %. We left the code as it is. `-Wa,-mbranches-within-32B-boundaries` shrinks the loss to about 4%. We didn't change the code, and all numbers in this post are measured without that flag.
### Validating UTF-8 16 bytes at a time ### Validating UTF-8 16 bytes at a time
@@ -391,7 +393,7 @@ Validating UTF-8 16 bytes at a time uses [simdjson](https://github.com/simdjson/
16-entry tables, indexed by nibbles of a byte and of the byte before it, each give the set of errors that nibble 16-entry tables, indexed by nibbles of a byte and of the byte before it, each give the set of errors that nibble
allows, and a pair of bytes is invalid if all three sets share an error. It needs `pshufb`, which is SSSE3 and not part allows, and a pair of bytes is invalid if all three sets share an error. It needs `pshufb`, which is SSSE3 and not part
of baseline x86-64. The function is compiled with `__attribute__((target("ssse3")))`, out of line, and called only if of baseline x86-64. The function is compiled with `__attribute__((target("ssse3")))`, out of line, and called only if
`cpuid` says so. The answer is cached in a constant-initialized `static std::atomic<int>`, so that neither a guarded `cpuid` says so. The result of that query is cached in a constant-initialized `static std::atomic<int>`, so that neither a guarded
local static nor a global constructor is needed; threads that ask at the same time all store the same answer. On an local static nor a global constructor is needed; threads that ask at the same time all store the same answer. On an
error, the code backs up to the start of the sequence and lets the byte path produce the error message. error, the code backs up to the start of the sequence and lets the byte path produce the error message.
[`JSON_NO_SIMD`](../../api/macros/json_no_simd.md) turns all vector code off. [`JSON_NO_SIMD`](../../api/macros/json_no_simd.md) turns all vector code off.
@@ -2320,7 +2322,7 @@ removed because 3.12.0 has no BON8.
**The other measurements.** The timeline is **The other measurements.** The timeline is
[`tests/benchmarks/history`](https://github.com/nlohmann/json/tree/develop/tests/benchmarks/history): one program, [`tests/benchmarks/history`](https://github.com/nlohmann/json/tree/develop/tests/benchmarks/history): one program,
compiled against `single_include` of 3.12.0, of each of the 351 commits on `develop` up to November 3 that changed `json.hpp` or compiled against `single_include` of 3.12.0, of each of the 351 commits on `develop` up to November 3 that changed `json.hpp` or
`json_view.hpp` since, and of 3.13.0 (C++17, `-O3`, no LTO), 4 rounds in random order, median of the rounds. Two `json_view.hpp` since, and of 3.13.0 (C++17, `-O3`, no LTO), 4 rounds in random order, median of the rounds (the mean of the two middle ones). Two
commits that don't compile on their own are left out. The commits up to October 27 were run on October 27, the commits that don't compile on their own are left out. The commits up to October 27 were run on October 27, the
later ones and 3.13.0 on November 3, together with twelve of the earlier commits as a check: per workload, their later ones and 3.13.0 on November 3, together with twelve of the earlier commits as a check: per workload, their
median moved by 1.5% or less. The median moved by 1.5% or less. The
@@ -2481,7 +2483,7 @@ measure binary size or memory use.
### H. Pull requests by theme ### H. Pull requests by theme
- <a class="nj-theme" href="#theme-1" title="theme 1">1</a> **Specialize for each input:** contiguous input [#5283][5283]; streams and `FILE*` [#5828][5828], [#5884][5884]; binary strings in bulk [#5233][5233]; sized sentinels [#5268][5268]. - <a class="nj-theme" href="#theme-1" title="theme 1">1</a> **Specialize for each input source:** contiguous input [#5283][5283]; streams and `FILE*` [#5828][5828], [#5884][5884]; binary strings in bulk [#5233][5233]; sized sentinels [#5268][5268].
- <a class="nj-theme" href="#theme-2" title="theme 2">2</a> **Work on words, not bytes:** word-at-a-time strings, UTF-8 runs, and digits [#5283][5283], [#5738][5738], - <a class="nj-theme" href="#theme-2" title="theme 2">2</a> **Work on words, not bytes:** word-at-a-time strings, UTF-8 runs, and digits [#5283][5283], [#5738][5738],
[#5851][5851]; indentation [#5810][5810]; `dump()` runs [#5285][5285]; SSE2 and NEON [#5853][5853], [#5851][5851]; indentation [#5810][5810]; `dump()` runs [#5285][5285]; SSE2 and NEON [#5853][5853],
[#5628][5628]; `json_document` with `ensure_ascii` [#5879][5879]; short strings [#5878][5878], [#5883][5883]; [#5628][5628]; `json_document` with `ensure_ascii` [#5879][5879]; short strings [#5878][5878], [#5883][5883];