wt/json

0 Fork 0

mirror of https://github.com/nlohmann/json.git synced 2026-10-22 11:36:14 +00:00

Files

T

Niels Lohmann e9970066c5 Scan strings with SSE2, SSSE3, or NEON (#5853 )

* Scan strings with SSE2, SSSE3, or NEON

The string fast path found the end of a run of plain characters 8 bytes at
a time (SWAR) and validated non-ASCII text one sequence at a time, calling
validate_one_utf8() out of line for each.

The vector code of json_view's scanner, in the new shared header
detail/input/simd_scan.hpp, now does both 16 bytes at a time: SSE2 finds
the end of a run on x86-64, and the UTF-8 check of Keiser and Lemire runs
with NEON on AArch64 and with SSSE3 on x86-64. SSSE3 is not part of x86-64,
so the check is compiled for it with a function attribute and used when the
CPU has it (cached in a constant-initialized atomic), else the scalar check;
code compiled with -mssse3 skips the CPU check. The first word of a run is
still checked with SWAR (the first four with NEON, where a vector mask
costs more), as most keys and short values end there, and the scalar
helpers are always inlined. On anything else, the scalar code is unchanged.

The new macro JSON_NO_SIMD selects the portable code everywhere, including
the SSE2/NEON digit formatting of the floating-point writer. The lexer tests
now also run with JSON_NO_SIMD and, on x86-64, with -mssse3, and a new test
places every byte that ends a run at each offset across several 16-byte
blocks, after ASCII and after CJK text.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the string scan from slowing down short strings and dump() on Arm

Three benchmarks were 7-9% slower than develop on Apple M1 (Apple Clang,
-O3 -flto): Accept/jeopardy, Dump/citm_catalog, and Dump/jeopardy.

- dump(): the serializer passes a string without its closing quote, so
  plain_run_end() ran out of input and called the out-of-line NEON loop
  for every string, with fewer than 16 bytes left. The loop is now only
  called for 16 bytes or more.
- accept(): no string of jeopardy.json reaches the vector UTF-8 check,
  yet compiling it into the lexer's scan function made it 5% slower: the
  function runs once per token and now saved more registers and kept a
  stack slot. lexer_string_bulk_run() checks the first four words inline
  and continues out of line, and the serializer keeps string_bulk_run()
  inline, which is faster there.
- The byte that ends a run is read from the scanned buffer instead of
  through get(), which Apple Clang merged with the other get() calls that
  load the read position again right after it was stored (3%).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Define string_bulk_run_from() only where lexer_string_bulk_run() calls it

With NEON and simdutf in C++17, lexer_string_bulk_run() takes the
string_bulk_run() path, so the out-of-line helper was unused.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not dispatch the SSSE3 string scan with GCC's C++20 modules

GCC does not keep the target("ssse3") attribute on functions it reads
back from a module interface, so the module build (ci_module_cpp20,
gcc:latest) rejected the intrinsics in vector_utf8_run() with "needs isa
option -mssse3". With __cpp_modules, GCC now uses the portable UTF-8 check
like other compilers without the target attribute; the SSE2 scan stays,
and -mssse3 or more still compiles the SSSE3 check directly. Clang is not
affected.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Include <cpuid.h> only once when json.hpp and json_view.hpp are both used

The SSSE3 dispatch of the string scan and the one of json_view each
include <cpuid.h>. Before GCC 11 the header has no include guard, so a
translation unit that includes both headers fails with GCC 10:
"redefinition of 'unsigned int __get_cpuid_max(unsigned int, unsigned int*)'".
Guard both includes with NLOHMANN_JSON_CPUID_INCLUDED.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add an ABI tag for JSON_NO_SIMD

JSON_NO_SIMD selects between two definitions of the internal scanning and
formatting functions. Like JSON_VIEW_NO_SIMD (_vns), it now appends _ns to
the inline namespace, so translation units built with and without it can
be linked into one program without ODR violations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

2026-10-21 20:00:50 +02:00

6.4 KiB

Raw Blame History

`nlohmann` Namespace

The 3.11.0 release introduced an inline namespace to allow different parts of a codebase to safely use different versions of the JSON library as long as they never exchange instances of library types.

Structure

The complete default namespace name is derived as follows:

The root namespace is always nlohmann.
The inline namespace starts with json_abi and is followed by several optional ABI tags according to the value of these ABI-affecting macros, in order:
- JSON_DIAGNOSTICS defined non-zero appends _diag.
- JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON defined non-zero appends _ldvcmp.
- JSON_DIAGNOSTIC_POSITIONS defined non-zero appends _dp.
- JSON_BRACE_INIT_COPY_SEMANTICS defined non-zero appends _bics.
- JSON_PRECISE_STREAM_POSITION defined non-zero appends _psp.
- JSON_STRICT_NUL_HANDLING defined non-zero appends _snul.
- JSON_STRICT_BINARY_UTF8 defined non-zero appends _sbu8.
- JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS defined non-zero appends _ekmo.
- JSON_DISABLE_STREAM_BULK_SCAN defined non-zero appends _dsbs.
- JSON_DISABLE_UNLOCKED_FILE_IO defined non-zero appends _dufio.
- JSON_NO_SIMD defined appends _ns.
- JSON_VIEW_NO_SIMD defined appends _vns.
- JSON_VIEW_USE_SSSE3 defined appends _vssse3.
The inline namespace ends with the suffix _v followed by the 3 components of the version number separated by underscores. To omit the version component, see Disabling the version component below.

For example, the namespace name for version 3.11.2 with JSON_DIAGNOSTICS defined to 1 is:

nlohmann::json_abi_diag_v3_11_2

Purpose

Several incompatibilities have been observed. Amongst the most common ones is linking code compiled with different definitions of JSON_DIAGNOSTICS. This is illustrated in the diagram below.

graph
    json["<strong>nlohmann_json (v3.10.5)</strong><br>JSON_DIAGNOSTICS=0"]
    json_diag["<strong>nlohmann_json (v3.10.5)</strong><br>JSON_DIAGNOSTICS=1"]
    library["<strong>some library</strong>"]
    app["<strong>application</strong>"]
    
    library --> json
    app --> json_diag
    app --> library

In releases prior to 3.11.0, mixing any version of the JSON library with different JSON_DIAGNOSTICS settings would result in a crashing application. If some_library never passes instances of JSON library types to the application, this scenario became safe in version 3.11.0 and above due to the inline namespace yielding distinct symbol names.

Limitations

Neither the compiler nor the linker will issue as much as a warning when translation units – intended to be linked together and that include different versions and/or configurations of the JSON library – exchange and use library types.

There is an exception when forward declarations are used (i.e., when including json_fwd.hpp) in which case the linker may complain about undefined references.

Disabling the version component

Different versions are not necessarily ABI-incompatible, but the project does not actively track changes in the ABI and recommends that all parts of a codebase exchanging library types be built with the same version. Users can, at their own risk, disable the version component of the inline namespace, allowing different versions – but not configurations – to be used in cases where the linker would otherwise output undefined reference errors.

To do so, define NLOHMANN_JSON_NAMESPACE_NO_VERSION to 1.

This applies to version 3.11.2 and above only; versions 3.11.0 and 3.11.1 can apply the technique described in the next section to emulate the effect of the NLOHMANN_JSON_NAMESPACE_NO_VERSION macro.

!!! danger "Use at your own risk"

Disabling the namespace version component and mixing ABI-incompatible versions will result in crashes or incorrect
behavior. You have been warned!

Disabling the inline namespace completely

When interoperability with code using a pre-3.11.0 version of the library is required, users can, at their own risk restore the old namespace layout by redefining NLOHMANN_JSON_NAMESPACE_BEGIN, NLOHMANN_JSON_NAMESPACE_END as follows:

#define NLOHMANN_JSON_NAMESPACE_BEGIN  namespace nlohmann {
#define NLOHMANN_JSON_NAMESPACE_END    }