* Scan json_view strings with SIMD and index large objects Speed up json_view's parser with SIMD scanning and a hash table for large objects. Long runs of string bytes are scanned 16 bytes at a time with NEON (AArch64, GCC and Clang) and SSE2 (x86-64), both baseline instruction sets. Keys keep 16 table checks before the vector loop, because their lengths repeat from record to record; string values get 8, because their lengths vary more. Non-ASCII text is validated 16 bytes at a time with simdjson's "lookup4" check (Keiser and Lemire, 2021), with NEON on AArch64 and, on x86-64, with SSSE3. SSSE3 is not part of baseline x86-64, so the check is compiled for SSSE3 with a function attribute and used only where CPUID reports it, which all x86-64 CPUs since about 2011 do; the answer is cached in a statically initialized atomic, so there is no guard of a local static and no global constructor. The same input is accepted either way. JSON_VIEW_NO_SIMD selects the portable code. On x86-64, string runs are now checked vector-first: one SSE2 compare from the first byte finds the end of most keys and short values, instead of a branch per byte for the first 8-16 bytes. AArch64 keeps the byte-wise steps, where a NEON mask costs more and the branches predict well. Entering an object or array no longer stalls: open() stores the parent's frame field by field instead of building it on the stack and reading it back with wider loads, which waited for the narrower stores to retire. Objects with 128 members or more get an open-addressing hash table built when the object closes, so operator[], at(), find(), contains(), count(), value(), and JSON pointers take constant time on average in such objects; of duplicate keys, the first is kept, as for the linear search. The idea comes from Boost.JSON. simdjson is credited in simd.hpp's SPDX block, the README, and license.md. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix GCC module build and MSVC C4127 in the json_view SIMD code GCC ignores the target attribute in modules, so the SSSE3 dispatch is disabled for the module interface (the check stays portable, SSE2 is kept). Make the 8-vs-16 byte unrolling condition in scan_string_run a preprocessor/template split to avoid a constant condition (C4127). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove an accidentally committed GCC module cache Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bound the probe length of the large-object hash index The key hash is not seeded, so keys chosen to collide made building the table quadratic (20,000 colliding keys took 470 ms to parse). A key may now sit at most 64 slots from its home slot; if a key would sit further away, the table is dropped and the object is searched linearly. Lookups stop after the same distance. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Release the spare capacity of the hash index in shrink_to_fit shrink_to_fit() now trims the tables of large objects like the node array and the decoded strings, and the list of large objects is released as soon as the tables are built. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use the portable string scan on non-x86 targets that define __SSE2__ WebAssembly with -msse2 defines __SSE2__, but has no <cpuid.h> or x86 intrinsics headers. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Document the hash index number in node::extra Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the last of duplicate keys in the object hash index Lookups in objects with 128 members or more now return the last member of a repeated key, like the linear search of smaller objects and like materialize() and basic_json::parse(). build_object_index let the first occurrence win, so the same text gave different results depending on the size of the object. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the first of duplicate keys in the object hash index again Lookups in objects with 128 members or more return the first member of a repeated key again, like the linear search of smaller objects. This undoes the code change of a69542046; its test now expects the first member from lookups (with and without a table) and the last value from materialize() and basic_json::parse(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix clang-tidy 22 findings in the json_view headers Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast would trip -Wcast-align=strict), use auto for a cast initialiser, add parentheses to a mixed expression, and name bugprone-std-namespace-modification in the NOLINTs of the tuple_size/tuple_element specialisations. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix clang-tidy 22 findings in unit-json_view.cpp and unit-json_view_builder.cpp Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Avoid GCC useless-cast and strict-overflow warnings in unit-json_view.cpp Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Index strings with size_t in the colliding-keys test (MSVC C4244 on Win32) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make json_view.hpp pass include-what-you-use ci_single_binaries runs IWYU with --error on every header. For json_view.hpp it suggested adding <array> (std::array is used), the headers that json.hpp already provides (abi_config, abi_macros, input_adapters, json_pointer, cpp_future, string_concat, value_t, json_fwd), and <version> for std::nullptr_t, and removing <cstddef>. - include <array> - keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>) - tell IWYU not to suggest the headers that json.hpp provides: the amalgamated json_view.hpp only includes json.hpp, so including them here would duplicate their definitions Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the dump, comparison, and large-object tests to unit-json_view_dump.cpp The dump and comparison tests moved to their own file in the dump pull request; the hash index tests of this branch join them there. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Regenerate the amalgamated headers Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Follow the renames from the review of the view builder The review of #5620 renamed document_data's tape/tape_size/tape_cap to nodes/node_count/node_capacity and made the decode arena a std::vector<char>; adapt the code of this change to it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Rename slots in the object index for Qt compatibility Qt defines slots as a macro, so a Qt header included before json_view.hpp broke the declarations of the local variable (see #5865). The test variable is renamed as well. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address the review of the json_view object index and SIMD scan From gregmarr's review of #5628: - Docs and comments say that the hash index is one of the object's keys. - object_index.hpp: conditional_static_cast for the hash, "slot" for the table position, no flag for a degenerate table, and a static_assert that the smallest indexed object is larger than the probe window. - scan.hpp unrolls with NLOHMANN_VIEW_REPEAT_FIRST_8/SECOND_8. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Regenerate the amalgamated headers Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Encode JSON_VIEW_NO_SIMD and JSON_VIEW_USE_SSSE3 in the ABI namespace Both macros select between definitions of the same inline functions, so translation units that disagree on them violated the ODR. Like the other ABI-affecting macros, they now add a tag to the inline namespace (_vns and _vssse3), and the docs replace the "Define consistently" warning with the usual ABI compatibility note. From gregmarr's review of #5628. nlohmann_json.natvis covers twice as many tags now, so it lists four times as many namespaces (14.7 MB instead of 3.4 MB). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Regenerate the amalgamated headers Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not include type_traits.hpp directly in the object index json.hpp, which object_index.hpp already includes, provides it; included directly, the json_view amalgamation (which only treats nlohmann/json.hpp as external) copied the json internals into json_view.hpp a second time, so the fuzzers, which include both single headers, failed with redefinitions (make_void, integer_sequence, priority_tag, ...). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Compare keys with key_equals() in the object hash index As the linear lookups do since the second review round of #5622. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Regenerate the amalgamated headers Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
6.0 KiB
nlohmann::basic_json_view::value
// (1)
template<typename T>
T value(string_view_t key, const T& default_value) const;
string_t value(string_view_t key, const char* default_value) const;
// (2)
template<typename T>
T value(const json_pointer& ptr, const T& default_value) const;
string_t value(const json_pointer& ptr, const char* default_value) const;
- Returns the value of the object member with key
key-- the first one, should the key occur more than once (see Notes on duplicate keys) -- converted toT, ordefault_valueif there is no such member. - Returns the value a JSON pointer
ptrrefers to, starting at this value, converted toT, ordefault_valueifptrcannot be resolved.
Both overloads have a dedicated #!cpp const char* overload, so #!cpp v.value(key, "default") (and the JSON pointer
equivalent) deduce string_t, not const char*, for their return type and for the comparison used to pick between
key and default_value.
Template parameters
T- the type to convert the found value to; also the type of
default_value
Parameters
key(in)- object key of the element to access
ptr(in)- JSON pointer to the element to access
default_value(in)- the value to return if
key/ptrresolves to no value
Return value
- the first member with key
key, converted toT, ordefault_value - the value
ptrresolves to, converted toT, ordefault_value
Exception safety
Strong guarantee: there are no changes to the view or the document it refers to.
Exceptions
- Throws
type_error.306if the value is not an object -- the same exception, with the same message, thatBasicJsonType::valuethrows for the same call. If a member with keykeyis found, throws whatever converting it toTthrows (typicallytype_error.302, with the same messageBasicJsonType::valuethrows for the same mismatch); a missing member never throws. - Throws
type_error.306if this value -- not the valueptrresolves to -- is neither an object nor an array. Throwsparse_error.106orparse_error.109ifptrcontains a malformed array index. Ifptrresolves to a value, throws whatever converting it toTthrows. Every other wayptrcan fail to resolve -- a missing key, an out-of-range or "-" array index, an unresolvable token on a primitive -- yieldsdefault_valueinstead of throwing, exactly asBasicJsonType::valuecatchesout_of_rangeand returnsdefault_value.
None of these exceptions carry a JSON_DIAGNOSTICS path: the view has no
BasicJsonType value to point at, so the exception is created without one, even if BasicJsonType was built with
JSON_DIAGNOSTICS enabled.
Complexity
- Linear in the number of members: as for
operator[], members are compared one after another, in document order, stopping at the first match. Plus the complexity of converting the found member toT(seeget). Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes constant time on average. - Linear in the number of reference tokens of
ptrand, for each token, in the number of members of the object at that level or the index into the array -- as for theoperator[]andatoverloads that take a JSON pointer. Plus the complexity of converting the resolved value toT.
Notes
!!! warning "Duplicate keys: the first member wins"
If the source text repeats a key, this resolves to the *first* member with it, not to the last one that
[`materialize()`](materialize.md) and `parse()` keep. See the [Notes on duplicate keys](operator[].md#notes) of
`operator[]` and [Duplicate keys](../../features/json_view.md#duplicate-keys) for the reasons and for how to get
the last value.
!!! info "Differences to at and operator[]"
Unlike [`at`](at.md), this function does not throw if `key`/`ptr` resolves to no value. Unlike
[`operator[]`](operator[].md), it never returns a [discarded](is_discarded.md) view -- it always returns a `T` --
and it is available on any view, since it never needs to insert a missing element the way the non-const
`BasicJsonType::operator[]` would.
!!! info "Which values can be asked"
As for [`BasicJsonType::value`](../basic_json/value.md), the key overload (1) requires an object, and the JSON
pointer overload (2) an object or an array.
Examples
??? example "Example: (1) access specified object element with default value"
The example below reads a couple of optional configuration fields with a default, so that a missing key never
needs a `#!cpp try`/`#!cpp catch` of its own.
```cpp
--8<-- "examples/basic_json_view__value.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__value.output"
```
??? example "Example: (2) access specified element via JSON pointer with default value"
The example below reads an optional, nested configuration value with a default, given as a JSON pointer.
```cpp
--8<-- "examples/basic_json_view__value_json_pointer.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__value_json_pointer.output"
```
See also
- at - access specified element with bounds checking (throws instead of returning a default value)
- operator[] - access specified element (returns a discarded view instead of a default value)
BasicJsonType::value- the corresponding function ofbasic_json
Version history
- Added in version 3.13.0.