Scan json_view strings with SIMD and index large objects (#5628)

* Scan json_view strings with SIMD and index large objects

Speed up json_view's parser with SIMD scanning and a hash table
for large objects.

Long runs of string bytes are scanned 16 bytes at a time with NEON
(AArch64, GCC and Clang) and SSE2 (x86-64), both baseline
instruction sets. Keys keep 16 table checks before the vector
loop, because their lengths repeat from record to record; string
values get 8, because their lengths vary more. Non-ASCII text is
validated 16 bytes at a time with simdjson's "lookup4" check
(Keiser and Lemire, 2021), with NEON on AArch64 and, on x86-64,
with SSSE3. SSSE3 is not part of baseline x86-64, so the check is
compiled for SSSE3 with a function attribute and used only where
CPUID reports it, which all x86-64 CPUs since about 2011 do; the
answer is cached in a statically initialized atomic, so there is
no guard of a local static and no global constructor. The same
input is accepted either way. JSON_VIEW_NO_SIMD selects the
portable code.

On x86-64, string runs are now checked vector-first: one SSE2
compare from the first byte finds the end of most keys and short
values, instead of a branch per byte for the first 8-16 bytes.
AArch64 keeps the byte-wise steps, where a NEON mask costs more and
the branches predict well. Entering an object or array no longer
stalls: open() stores the parent's frame field by field instead of
building it on the stack and reading it back with wider loads,
which waited for the narrower stores to retire.

Objects with 128 members or more get an open-addressing hash table
built when the object closes, so operator[], at(), find(),
contains(), count(), value(), and JSON pointers take constant time
on average in such objects; of duplicate keys, the first is kept,
as for the linear search. The idea comes from Boost.JSON.

simdjson is credited in simd.hpp's SPDX block, the README, and
license.md.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix GCC module build and MSVC C4127 in the json_view SIMD code

GCC ignores the target attribute in modules, so the SSSE3 dispatch is
disabled for the module interface (the check stays portable, SSE2 is kept).
Make the 8-vs-16 byte unrolling condition in scan_string_run a
preprocessor/template split to avoid a constant condition (C4127).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Remove an accidentally committed GCC module cache

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Bound the probe length of the large-object hash index

The key hash is not seeded, so keys chosen to collide made building the table quadratic (20,000 colliding keys took 470 ms to parse). A key may now sit at most 64 slots from its home slot; if a key would sit further away, the table is dropped and the object is searched linearly. Lookups stop after the same distance.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Release the spare capacity of the hash index in shrink_to_fit

shrink_to_fit() now trims the tables of large objects like the node array and the decoded strings, and the list of large objects is released as soon as the tables are built.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Use the portable string scan on non-x86 targets that define __SSE2__

WebAssembly with -msse2 defines __SSE2__, but has no <cpuid.h> or x86 intrinsics headers.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Document the hash index number in node::extra

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the last of duplicate keys in the object hash index

Lookups in objects with 128 members or more now return the last member of
a repeated key, like the linear search of smaller objects and like
materialize() and basic_json::parse(). build_object_index let the first
occurrence win, so the same text gave different results depending on the
size of the object.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the first of duplicate keys in the object hash index again

Lookups in objects with 128 members or more return the first member of a
repeated key again, like the linear search of smaller objects. This undoes
the code change of a69542046; its test now expects the first member from
lookups (with and without a table) and the last value from materialize()
and basic_json::parse().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy 22 findings in the json_view headers

Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast
would trip -Wcast-align=strict), use auto for a cast initialiser, add
parentheses to a mixed expression, and name bugprone-std-namespace-modification
in the NOLINTs of the tuple_size/tuple_element specialisations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy 22 findings in unit-json_view.cpp and unit-json_view_builder.cpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Avoid GCC useless-cast and strict-overflow warnings in unit-json_view.cpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Index strings with size_t in the colliding-keys test (MSVC C4244 on Win32)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make json_view.hpp pass include-what-you-use

ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Move the dump, comparison, and large-object tests to unit-json_view_dump.cpp

The dump and comparison tests moved to their own file in the dump pull
request; the hash index tests of this branch join them there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Follow the renames from the review of the view builder

The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Rename slots in the object index for Qt compatibility

Qt defines slots as a macro, so a Qt header included before
json_view.hpp broke the declarations of the local variable (see #5865).
The test variable is renamed as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the review of the json_view object index and SIMD scan

From gregmarr's review of #5628:

- Docs and comments say that the hash index is one of the object's keys.
- object_index.hpp: conditional_static_cast for the hash, "slot" for the
  table position, no flag for a degenerate table, and a static_assert
  that the smallest indexed object is larger than the probe window.
- scan.hpp unrolls with NLOHMANN_VIEW_REPEAT_FIRST_8/SECOND_8.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Encode JSON_VIEW_NO_SIMD and JSON_VIEW_USE_SSSE3 in the ABI namespace

Both macros select between definitions of the same inline functions, so
translation units that disagree on them violated the ODR. Like the other
ABI-affecting macros, they now add a tag to the inline namespace (_vns and
_vssse3), and the docs replace the "Define consistently" warning with the
usual ABI compatibility note. From gregmarr's review of #5628.

nlohmann_json.natvis covers twice as many tags now, so it lists four times
as many namespaces (14.7 MB instead of 3.4 MB).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not include type_traits.hpp directly in the object index

json.hpp, which object_index.hpp already includes, provides it; included
directly, the json_view amalgamation (which only treats nlohmann/json.hpp
as external) copied the json internals into json_view.hpp a second time,
so the fuzzers, which include both single headers, failed with
redefinitions (make_void, integer_sequence, priority_tag, ...).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Compare keys with key_equals() in the object hash index

As the linear lookups do since the second review round of #5622.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json_document/read.md

@@ -51,6 +51,11 @@ own on the next, since ownership is decided freshly each time. An input that is
 [`parse()`](parse.md#notes)) is written into the document's previous owned copy, reusing its memory; a moved-in
 `#!cpp std::string` replaces it.
 
+Reusing a document matters most for large inputs: the operating system provides the memory of a fresh node index one
+page at a time, and every page costs a page fault the first time it is written. On x86-64 Linux (4 KiB pages), parsing
+a 55 MB document into a reused document took about 40 % less time than parsing it into a fresh one. Programs that parse
+many documents of similar size should therefore keep one document and call `read()`.
+
 An integer argument that is not a `#!cpp bool` where the flags are expected, such as `#!cpp read(ptr, len)`, does not
 compile; see [`parse`](parse.md#notes).
 

docs/mkdocs/docs/api/basic_json_view/at.md

@@ -77,6 +77,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
 1. Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after
    another, in document order, stopping at the first match. Each comparison first checks the
    key's length -- already known from the index, without reading the key bytes -- before comparing its content.
+   Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
+   constant time on average.
 2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
    index (unlike `BasicJsonType`'s array, which is random-access).
 3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at

docs/mkdocs/docs/api/basic_json_view/contains.md

@@ -35,6 +35,8 @@ No-throw guarantee: this function never throws exceptions.
 1. Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after
    another, in document order, stopping at the first match. Each comparison first checks the
    key's length -- already known from the index, without reading the key bytes -- before comparing its content.
+   Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
+   constant time on average.
 2. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
    that level or the index into the array -- as for [`operator[]`](operator[].md#complexity) and
    [`at`](at.md#complexity) with a JSON pointer.

docs/mkdocs/docs/api/basic_json_view/count.md

@@ -28,7 +28,8 @@ No-throw guarantee: this function never throws exceptions.
 Linear in the number of members: unlike the lookup functions, `count()` does not stop at the first match, because it
 has to count every member with the key. Members are compared one after another, in document order. Each comparison
 first checks the key's length -- already known from the index, without reading the key bytes -- before comparing its
-content.
+content. The hash index of the keys of objects with 128 or more members does not help here, since every member
+is visited.
 
 ## Notes
 

docs/mkdocs/docs/api/basic_json_view/find.md

@@ -28,6 +28,8 @@ No-throw guarantee: this function never throws exceptions.
 Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after another,
 in document order, stopping at the first match. Each comparison first checks the key's length
 -- already known from the index, without reading the key bytes -- before comparing its content.
+Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
+constant time on average.
 
 ## Notes
 

docs/mkdocs/docs/api/basic_json_view/operator[].md

@@ -83,6 +83,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
    another, in document order, stopping at the first match. Each comparison first checks the
    key's length -- already known from the index, without reading the key bytes -- before comparing its content, so a
    key of a different length than `key` is rejected without touching the source text.
+   Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
+   constant time on average.
 2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
    index (unlike `BasicJsonType`'s array, which is random-access).
 3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
@@ -114,6 +116,7 @@ document.
     member in order and so keep the *last* value for a repeated key, so `#!cpp v["a"]` and
     `#!cpp v.materialize()["a"]` can differ. To get the value `parse()` would give, use
     [`materialize()`](materialize.md) or iterate the members with [`items()`](items.md) and keep the last match.
+    The key hash index of a larger object (128 members or more) leads to the first member of a key as well.
     [`begin()`](begin.md)/[`end()`](end.md) and [`items()`](items.md) iterate over *all* members, including
     duplicates, in document order. See [Duplicate keys](../../features/json_view.md#duplicate-keys) and
     [`size()`](size.md#notes).

docs/mkdocs/docs/api/basic_json_view/value.md

@@ -70,6 +70,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
 1. Linear in the number of members: as for [`operator[]`](operator[].md#complexity), members are compared one after
    another, in document order, stopping at the first match. Plus the complexity of converting
    the found member to `T` (see [`get`](get.md)).
+   Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
+   constant time on average.
 2. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
    that level or the index into the array -- as for the [`operator[]`](operator[].md#complexity) and
    [`at`](at.md#complexity) overloads that take a JSON pointer. Plus the complexity of converting the resolved value

docs/mkdocs/docs/api/macros/index.md

@@ -42,6 +42,8 @@ header. See also the [macro overview page](../../features/macros.md).
 - [**JSON_SKIP_UNSUPPORTED_COMPILER_CHECK**](json_skip_unsupported_compiler_check.md) - do not warn about unsupported compilers
 - [**JSON_USE_GLOBAL_UDLS**](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace
 - [**JSON_USE_SIMDUTF**](json_use_simdutf.md) - use the simdutf library to accelerate UTF-8 validation
+- [**JSON_VIEW_NO_SIMD**](json_view_no_simd.md) - use only portable code in the parser of `json_view.hpp`
+- [**JSON_VIEW_USE_SSSE3**](json_view_use_ssse3.md) - validate non-ASCII strings with SSSE3 in the parser of `json_view.hpp`
 
 ## Library version
 

docs/mkdocs/docs/api/macros/json_view_no_simd.md

@@ -0,0 +1,52 @@
+# JSON_VIEW_NO_SIMD
+
+```cpp
+#define JSON_VIEW_NO_SIMD
+```
+
+When defined, the parser of [`basic_json_document`](../basic_json_document/index.md) (`<nlohmann/json_view.hpp>`)
+uses only portable C++ to scan strings. By default, it scans long runs of string bytes 16 at a time with NEON on
+AArch64 (with GCC and Clang) and SSE2 on x86-64, which are part of the baseline instruction sets of these
+architectures, and validates non-ASCII text with NEON (or SSSE3, see
+[`JSON_VIEW_USE_SSSE3`](json_view_use_ssse3.md)).
+
+The same input is accepted or rejected either way, with the same values, and errors are reported the same way; only
+the speed differs. The macro exists for platforms whose compilers lack the intrinsics headers, and to test the portable
+code.
+
+!!! note "ABI compatibility"
+
+    The macro selects between two definitions of the same inline functions. Whether it is defined is encoded in the
+    [namespace](../../features/namespace.md) (tag `_vns`), resulting in distinct symbol names. Translation units
+    compiled with and without it can therefore be linked into the same program without One Definition Rule (ODR)
+    violations, but they cannot exchange instances of library types.
+
+## Default definition
+
+By default, `#!cpp JSON_VIEW_NO_SIMD` is not defined, and the vector code is used where available.
+
+```cpp
+#undef JSON_VIEW_NO_SIMD
+```
+
+## Examples
+
+??? example
+
+    The code below uses the portable string scanning of the view.
+
+    ```cpp
+    #define JSON_VIEW_NO_SIMD
+    #include <nlohmann/json_view.hpp>
+
+    ...
+    ```
+
+## See also
+
+- [JSON_VIEW_USE_SSSE3](json_view_use_ssse3.md) - validate non-ASCII strings with SSSE3 on x86-64
+- [json_view](../../features/json_view.md) - the zero-copy view
+
+## Version history
+
+- Added in version 3.13.0.

docs/mkdocs/docs/api/macros/json_view_use_ssse3.md

@@ -0,0 +1,53 @@
+# JSON_VIEW_USE_SSSE3
+
+```cpp
+#define JSON_VIEW_USE_SSSE3
+```
+
+When defined on x86-64, the parser of [`basic_json_document`](../basic_json_document/index.md)
+(`<nlohmann/json_view.hpp>`) validates non-ASCII text in strings with SSSE3 without asking the CPU first.
+
+By default, the parser checks once at run time whether the CPU has SSSE3 (all x86-64 CPUs since about 2011 have it)
+and then validates non-ASCII text 16 bytes at a time, using the "lookup4" algorithm of
+[simdjson](https://github.com/simdjson/simdjson); on CPUs without SSSE3, it validates one UTF-8 sequence at a time.
+The vector check is compiled for SSSE3 with a function attribute (GCC 4.9 and later, Clang), so this needs no compiler
+option. With MSVC, the check uses `__cpuid`. On AArch64, the vector check uses NEON and is always on.
+
+Define the macro only together with a compiler option that enables SSSE3 (e.g. `-mssse3`, or `-march=` with a CPU that
+has it), and only for programs that run on such CPUs. It saves the check of the CPU, which costs little. The same
+input is accepted or rejected either way; only the speed of non-ASCII text differs.
+
+!!! note "ABI compatibility"
+
+    The macro selects between two definitions of the same inline functions. Whether it is defined is encoded in the
+    [namespace](../../features/namespace.md) (tag `_vssse3`, on every architecture), resulting in distinct symbol
+    names. Translation units compiled with and without it can therefore be linked into the same program without One
+    Definition Rule (ODR) violations, but they cannot exchange instances of library types.
+
+## Default definition
+
+By default, `#!cpp JSON_VIEW_USE_SSSE3` is not defined.
+
+```cpp
+#undef JSON_VIEW_USE_SSSE3
+```
+
+## Examples
+
+??? example
+
+    With CMake, for a program that only runs on CPUs with SSSE3:
+
+    ```cmake
+    target_compile_definitions(your_target PRIVATE JSON_VIEW_USE_SSSE3)
+    target_compile_options(your_target PRIVATE -mssse3)
+    ```
+
+## See also
+
+- [JSON_VIEW_NO_SIMD](json_view_no_simd.md) - use only portable code in the view's parser
+- [JSON_USE_SIMDUTF](json_use_simdutf.md) - validate UTF-8 with simdutf in `basic_json`'s parser
+
+## Version history
+
+- Added in version 3.13.0.

docs/mkdocs/docs/api/macros/nlohmann_json_namespace.md

@@ -39,6 +39,6 @@ the library.
 ## Version history
 
 - Added in version 3.11.0. Changed inline namespace name in version 3.11.2. Added the ABI tag `_dp` in
-  version 3.12.0, and the ABI tags `_bics`, `_psp`, `_snul`, `_sbu8`, `_ekmo`, `_dsbs`, and `_dufio` in version
-  3.13.0; see
+  version 3.12.0, and the ABI tags `_bics`, `_psp`, `_snul`, `_sbu8`, `_ekmo`, `_dsbs`, `_dufio`, `_vns`, and
+  `_vssse3` in version 3.13.0; see
   [`nlohmann` Namespace](../../features/namespace.md#structure).