docs: address review comments on the performance blog post
- fix wording: "are new", "for each input source", "the strings", "the last zero to seven bytes", "by" Danila Kutenin, "the result of that query" - introduce `json_document` where it first gets a detail, say that the `-mbranches` flag is not used, explain why the position and token bookkeeping overloads are empty, and say that a step's percentage is relative to the time before the step and that a median of 4 is the mean of the two middle values - write percentages without a space before `%` in the post and in the price figure (regenerated) Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
1 file changed
+17
-15
@@ -27,7 +27,7 @@ compares](#how-it-compares) has the numbers, including where it loses. The price
|
|||||||
example programs take 34–62% longer to compile. A precompiled header or a C++20 module wins back most or all of
|
example programs take 34–62% longer to compile. A precompiled header or a C++20 module wins back most or all of
|
||||||
that in debug builds, and 41–91% of it in optimized builds ([The price](#the-price)).
|
that in debug builds, and 41–91% of it in optimized builds ([The price](#the-price)).
|
||||||
|
|
||||||
This post is about *how*. Hardly any of the techniques is new: we borrowed them from
|
This post is about *how*. Hardly any of the techniques are new: we borrowed them from
|
||||||
[simdjson](https://github.com/simdjson/simdjson), [yyjson](https://github.com/ibireme/yyjson),
|
[simdjson](https://github.com/simdjson/simdjson), [yyjson](https://github.com/ibireme/yyjson),
|
||||||
[fast_float](https://github.com/fastfloat/fast_float), [Żmij](https://github.com/vitaut/zmij),
|
[fast_float](https://github.com/fastfloat/fast_float), [Żmij](https://github.com/vitaut/zmij),
|
||||||
[serde_json](https://github.com/serde-rs/json), [Glaze](https://github.com/stephenberry/glaze),
|
[serde_json](https://github.com/serde-rs/json), [Glaze](https://github.com/stephenberry/glaze),
|
||||||
@@ -78,7 +78,7 @@ Eight themes did most of the work:
|
|||||||
|
|
||||||
<div class="nj-theme-list" markdown>
|
<div class="nj-theme-list" markdown>
|
||||||
|
|
||||||
- <a class="nj-theme" href="#theme-1" title="theme 1">1</a> [Specialize for each input](#theme-1): a dedicated reader for strings, streams, and `FILE*`, instead of one
|
- <a class="nj-theme" href="#theme-1" title="theme 1">1</a> [Specialize for each input source](#theme-1): a dedicated reader for each of strings, streams, and `FILE*`, instead of one
|
||||||
character at a time for all of them.
|
character at a time for all of them.
|
||||||
- <a class="nj-theme" href="#theme-2" title="theme 2">2</a> [Work on words, not bytes](#theme-2): 8 bytes per step with plain integer arithmetic, 16 with SSE2 and NEON.
|
- <a class="nj-theme" href="#theme-2" title="theme 2">2</a> [Work on words, not bytes](#theme-2): 8 bytes per step with plain integer arithmetic, 16 with SSE2 and NEON.
|
||||||
- <a class="nj-theme" href="#theme-3" title="theme 3">3</a> [Own the number conversions](#theme-3): no `strtod`, no locale.
|
- <a class="nj-theme" href="#theme-3" title="theme 3">3</a> [Own the number conversions](#theme-3): no `strtod`, no locale.
|
||||||
@@ -122,9 +122,10 @@ build's speed by a few percent on its own, so only the larger steps mean somethi
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<p class="nj-caption" markdown>Speedup over 3.12.0 after each of the 351 commits that changed the headers up to November 3, from 3.12.0 (left) to 3.13.0 (right). More is better; each point is the median of 4 interleaved runs of one build. The badges are theme numbers; "U" is a UTF-8 check that came and went. The other four workloads, and every labeled step with its pull request and size, are in [Appendix C](#c-the-timeline).</p>
|
<p class="nj-caption" markdown>Speedup over 3.12.0 after each of the 351 commits that changed the headers up to November 3, from 3.12.0 (left) to 3.13.0 (right). More is better; each point is the median of 4 runs of one build, taken in random order (with 4 runs, the mean of the two middle ones). The badges are theme numbers; "U" is a UTF-8 check that came and went. The other four workloads, and every labeled step with its pull request and size, are in [Appendix C](#c-the-timeline).</p>
|
||||||
|
|
||||||
A few things stand out:
|
A few things stand out. A percentage for a step is relative to the time just before that step, not to the time of
|
||||||
|
3.12.0:
|
||||||
|
|
||||||
- **There is no single big step.** Parsing `citm_catalog.json` got faster in about six visible steps. The largest
|
- **There is no single big step.** Parsing `citm_catalog.json` got faster in about six visible steps. The largest
|
||||||
removed a buffer that only error messages read, which alone saved 35% of the time. The next ones added the fast
|
removed a buffer that only error messages read, which alone saved 35% of the time. The next ones added the fast
|
||||||
@@ -140,7 +141,7 @@ A few things stand out:
|
|||||||
request; on the indented `citm_catalog.json` it saved 35%, because every space of the indentation went through
|
request; on the indented `citm_catalog.json` it saved 35%, because every space of the indentation went through
|
||||||
that buffer.
|
that buffer.
|
||||||
|
|
||||||
## <span class="nj-theme">1</span> Specialize for each input { #theme-1 }
|
## <span class="nj-theme">1</span> Specialize for each input source { #theme-1 }
|
||||||
|
|
||||||
This is the function the 3.12.0 lexer called for **every** byte of input:
|
This is the function the 3.12.0 lexer called for **every** byte of input:
|
||||||
|
|
||||||
@@ -219,7 +220,8 @@ static constexpr bool lazy_position = seekable;
|
|||||||
|
|
||||||
Without `if constexpr`, the flags select overloads by tag dispatch, `f(std::integral_constant<bool, lazy_position>{})`,
|
Without `if constexpr`, the flags select overloads by tag dispatch, `f(std::integral_constant<bool, lazy_position>{})`,
|
||||||
which also keeps MSVC from warning about constant conditions. `get()` still looks like the old one, but its position
|
which also keeps MSVC from warning about constant conditions. `get()` still looks like the old one, but its position
|
||||||
and token bookkeeping are now calls whose `std::true_type` overloads are empty. For a `std::string`, what remains is
|
and token bookkeeping are now calls. For an adapter that can seek, their `std::true_type` overloads are empty and cost
|
||||||
|
nothing, because the position and the text of a token are derived only when an error message needs them (theme <a class="nj-theme" href="#theme-4" title="theme 4">4</a>). For a `std::string`, what remains is
|
||||||
the branch on `next_unget` and the adapter's bounds check and pointer increment.
|
the branch on `next_unget` and the adapter's bounds check and pointer increment.
|
||||||
|
|
||||||
### Contiguous input
|
### Contiguous input
|
||||||
@@ -227,8 +229,8 @@ the branch on `next_unget` and the adapter's bounds check and pointer increment.
|
|||||||
Containers with `data()` and `size()` over single-byte elements now become a pointer
|
Containers with `data()` and `size()` over single-byte elements now become a pointer
|
||||||
range in every C++ standard, not only with C++20's contiguous iterators. `data()` and `size()` alone would be duck
|
range in every C++ standard, not only with C++20's contiguous iterators. `data()` and `size()` alone would be duck
|
||||||
typing, so the trait also requires the container's `value_type` to be the element type behind `data()`. A type whose
|
typing, so the trait also requires the container's `value_type` to be the element type behind `data()`. A type whose
|
||||||
`size()` counts something else keeps the iterator path: slower, but correct. On a pointer range, strings,
|
`size()` counts something else keeps the iterator path: slower, but correct. On a pointer range, the strings,
|
||||||
numbers and the indentation after a line break are scanned in bulk (themes <a class="nj-theme" href="#theme-2" title="theme 2">2</a> and <a class="nj-theme" href="#theme-3" title="theme 3">3</a>). Every fast path defers to the byte path on
|
the numbers and the indentation after a line break are scanned in bulk (themes <a class="nj-theme" href="#theme-2" title="theme 2">2</a> and <a class="nj-theme" href="#theme-3" title="theme 3">3</a>). Every fast path defers to the byte path on
|
||||||
anything malformed, so error messages and byte positions stay the same, with one deliberate correction (the column
|
anything malformed, so error messages and byte positions stay the same, with one deliberate correction (the column
|
||||||
after a number at the end of a line). [Appendix J](#j-validation) describes how that was checked.
|
after a number at the end of a line). [Appendix J](#j-validation) describes how that was checked.
|
||||||
|
|
||||||
@@ -318,7 +320,7 @@ Two details make this portable C++11. The word is loaded with `std::memcpy`, whi
|
|||||||
which, unlike a `reinterpret_cast`, doesn't break the aliasing rules. And the load always produces the bytes in
|
which, unlike a `reinterpret_cast`, doesn't break the aliasing rules. And the load always produces the bytes in
|
||||||
little-endian order, so the first byte of the text is the lowest byte of the word everywhere; a big-endian target
|
little-endian order, so the first byte of the text is the lowest byte of the word everywhere; a big-endian target
|
||||||
assembles the word from bytes, which compilers fold into a load and a byte swap. Every loop checks that eight bytes
|
assembles the word from bytes, which compilers fold into a load and a byte swap. Every loop checks that eight bytes
|
||||||
remain before it loads, and finishes the last seven bytes one at a time, so nothing is read past the input.
|
remain before it loads, and finishes the last zero to seven bytes one at a time, so nothing is read past the input.
|
||||||
|
|
||||||
The same kind of word test now runs in several places:
|
The same kind of word test now runs in several places:
|
||||||
|
|
||||||
@@ -343,7 +345,7 @@ Where a vector unit is part of the baseline instruction set (SSE2 on x86-64, NEO
|
|||||||
64-bit Arm), the string paths take 16 bytes per step. Two well-known details are worth repeating. A signed comparison
|
64-bit Arm), the string paths take 16 bytes per step. Two well-known details are worth repeating. A signed comparison
|
||||||
`byte < 0x20` catches control characters *and* every byte from 0x80, because those are negative as `int8_t`, so four
|
`byte < 0x20` catches control characters *and* every byte from 0x80, because those are negative as `int8_t`, so four
|
||||||
classes of bytes take three comparisons. And NEON has no equivalent of x86's `pmovmskb`, which turns a vector of
|
classes of bytes take three comparisons. And NEON has no equivalent of x86's `pmovmskb`, which turns a vector of
|
||||||
comparison results into a bit mask. The usual replacement, after
|
comparison results into a bit mask. The usual replacement, by
|
||||||
[Danila Kutenin](https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/porting-x86-vector-bitmask-optimizations-to-arm-neon),
|
[Danila Kutenin](https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/porting-x86-vector-bitmask-optimizations-to-arm-neon),
|
||||||
shifts each 16-bit lane right by four
|
shifts each 16-bit lane right by four
|
||||||
and narrows it to eight bits, which leaves four bits per input byte in a 64-bit integer:
|
and narrows it to eight bits, which leaves four bits per input byte in a 64-bit integer:
|
||||||
@@ -377,12 +379,12 @@ while x86-64 runs one word first. Inlining the vector code into the lexer cost 5
|
|||||||
although no string of that file is long enough to reach it: the scan function, which runs once per token, saved more
|
although no string of that file is long enough to reach it: the scan function, which runs once per token, saved more
|
||||||
registers and needed a stack slot.
|
registers and needed a stack slot.
|
||||||
|
|
||||||
`json_document` also skips the digits after a decimal point 16 bytes at a time with SSE2: a compare and a bit mask
|
[`json_document`](../../features/json_view.md), the new document type of `json_view.hpp`, also skips the digits after a decimal point 16 bytes at a time with SSE2: a compare and a bit mask
|
||||||
instead of a branch per digit, about 33% of the instructions for a 12-digit fraction. Parsing `canada.json` got
|
instead of a branch per digit, about 33% of the instructions for a 12-digit fraction. Parsing `canada.json` got
|
||||||
22% faster on x86-64, a file of 10,000 floats 28–32%. One row got slower: with GCC 16, `citm_catalog.json` took
|
22% faster on x86-64, a file of 10,000 floats 28–32%. One row got slower: with GCC 16, `citm_catalog.json` took
|
||||||
8% longer with the same number of instructions. More of its hot jumps now crossed a 32-byte boundary, which the
|
8% longer with the same number of instructions. More of its hot jumps now crossed a 32-byte boundary, which the
|
||||||
microcode fix for Intel's JCC erratum makes slow on our server; assembling with
|
microcode fix for Intel's JCC erratum makes slow on our server; assembling with
|
||||||
`-Wa,-mbranches-within-32B-boundaries` shrinks the loss to about 4 %. We left the code as it is.
|
`-Wa,-mbranches-within-32B-boundaries` shrinks the loss to about 4%. We didn't change the code, and all numbers in this post are measured without that flag.
|
||||||
|
|
||||||
### Validating UTF-8 16 bytes at a time
|
### Validating UTF-8 16 bytes at a time
|
||||||
|
|
||||||
@@ -391,7 +393,7 @@ Validating UTF-8 16 bytes at a time uses [simdjson](https://github.com/simdjson/
|
|||||||
16-entry tables, indexed by nibbles of a byte and of the byte before it, each give the set of errors that nibble
|
16-entry tables, indexed by nibbles of a byte and of the byte before it, each give the set of errors that nibble
|
||||||
allows, and a pair of bytes is invalid if all three sets share an error. It needs `pshufb`, which is SSSE3 and not part
|
allows, and a pair of bytes is invalid if all three sets share an error. It needs `pshufb`, which is SSSE3 and not part
|
||||||
of baseline x86-64. The function is compiled with `__attribute__((target("ssse3")))`, out of line, and called only if
|
of baseline x86-64. The function is compiled with `__attribute__((target("ssse3")))`, out of line, and called only if
|
||||||
`cpuid` says so. The answer is cached in a constant-initialized `static std::atomic<int>`, so that neither a guarded
|
`cpuid` says so. The result of that query is cached in a constant-initialized `static std::atomic<int>`, so that neither a guarded
|
||||||
local static nor a global constructor is needed; threads that ask at the same time all store the same answer. On an
|
local static nor a global constructor is needed; threads that ask at the same time all store the same answer. On an
|
||||||
error, the code backs up to the start of the sequence and lets the byte path produce the error message.
|
error, the code backs up to the start of the sequence and lets the byte path produce the error message.
|
||||||
[`JSON_NO_SIMD`](../../api/macros/json_no_simd.md) turns all vector code off.
|
[`JSON_NO_SIMD`](../../api/macros/json_no_simd.md) turns all vector code off.
|
||||||
@@ -2320,7 +2322,7 @@ removed because 3.12.0 has no BON8.
|
|||||||
**The other measurements.** The timeline is
|
**The other measurements.** The timeline is
|
||||||
[`tests/benchmarks/history`](https://github.com/nlohmann/json/tree/develop/tests/benchmarks/history): one program,
|
[`tests/benchmarks/history`](https://github.com/nlohmann/json/tree/develop/tests/benchmarks/history): one program,
|
||||||
compiled against `single_include` of 3.12.0, of each of the 351 commits on `develop` up to November 3 that changed `json.hpp` or
|
compiled against `single_include` of 3.12.0, of each of the 351 commits on `develop` up to November 3 that changed `json.hpp` or
|
||||||
`json_view.hpp` since, and of 3.13.0 (C++17, `-O3`, no LTO), 4 rounds in random order, median of the rounds. Two
|
`json_view.hpp` since, and of 3.13.0 (C++17, `-O3`, no LTO), 4 rounds in random order, median of the rounds (the mean of the two middle ones). Two
|
||||||
commits that don't compile on their own are left out. The commits up to October 27 were run on October 27, the
|
commits that don't compile on their own are left out. The commits up to October 27 were run on October 27, the
|
||||||
later ones and 3.13.0 on November 3, together with twelve of the earlier commits as a check: per workload, their
|
later ones and 3.13.0 on November 3, together with twelve of the earlier commits as a check: per workload, their
|
||||||
median moved by 1.5% or less. The
|
median moved by 1.5% or less. The
|
||||||
@@ -2481,7 +2483,7 @@ measure binary size or memory use.
|
|||||||
|
|
||||||
### H. Pull requests by theme
|
### H. Pull requests by theme
|
||||||
|
|
||||||
- <a class="nj-theme" href="#theme-1" title="theme 1">1</a> **Specialize for each input:** contiguous input [#5283][5283]; streams and `FILE*` [#5828][5828], [#5884][5884]; binary strings in bulk [#5233][5233]; sized sentinels [#5268][5268].
|
- <a class="nj-theme" href="#theme-1" title="theme 1">1</a> **Specialize for each input source:** contiguous input [#5283][5283]; streams and `FILE*` [#5828][5828], [#5884][5884]; binary strings in bulk [#5233][5233]; sized sentinels [#5268][5268].
|
||||||
- <a class="nj-theme" href="#theme-2" title="theme 2">2</a> **Work on words, not bytes:** word-at-a-time strings, UTF-8 runs, and digits [#5283][5283], [#5738][5738],
|
- <a class="nj-theme" href="#theme-2" title="theme 2">2</a> **Work on words, not bytes:** word-at-a-time strings, UTF-8 runs, and digits [#5283][5283], [#5738][5738],
|
||||||
[#5851][5851]; indentation [#5810][5810]; `dump()` runs [#5285][5285]; SSE2 and NEON [#5853][5853],
|
[#5851][5851]; indentation [#5810][5810]; `dump()` runs [#5285][5285]; SSE2 and NEON [#5853][5853],
|
||||||
[#5628][5628]; `json_document` with `ensure_ascii` [#5879][5879]; short strings [#5878][5878], [#5883][5883];
|
[#5628][5628]; `json_document` with `ensure_ascii` [#5879][5879]; short strings [#5878][5878], [#5883][5883];
|
||||||
|
|||||||
Reference in new issue
Block a user