Validate non-ASCII strings with SSSE3 on every x86-64 CPU that has it
The vector UTF-8 check of json_view needed SSSE3 at compile time (JSON_VIEW_USE_SSSE3 with -mssse3), so default x86-64 builds validated non-ASCII text one sequence at a time. The check is now compiled for SSSE3 with a function attribute (GCC 4.9 and later, Clang; MSVC compiles the intrinsics anyway) and used where CPUID reports SSSE3. The answer is kept in an atomic that is initialized at compile time, so neither a guard nor a global constructor is needed. The definitions do not depend on compiler flags, so there is no ODR issue. JSON_VIEW_USE_SSSE3 now only skips the CPU check. On x86-64 Linux (Haswell), twitter.json parses 23% faster with Clang 18 and 25-34% faster with GCC 13, now ahead of yyjson. Also always inline read_eight_bytes() and parse_eight_digits(): GCC called both in the number loops of the lexer and of json_view (52 call sites), which cost about 10% on citm_catalog.json at -O2. Document that reusing a document with read() avoids the page faults of a fresh node index (about 40% of a 55 MB parse on x86-64 Linux). Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
9 files changed
+173
-36
No files matched your search
@@ -249,8 +249,9 @@ template<typename FloatType>
|
||||
using native_float_t = typename std::conditional<std::numeric_limits<FloatType>::digits == 24, float, double>::type;
|
||||
|
||||
/// the value of the eight ASCII digits in @a v (see read_eight_bytes()), three
|
||||
/// multiplications instead of eight (after simdjson and fast_float)
|
||||
inline std::uint32_t parse_eight_digits(std::uint64_t v) noexcept
|
||||
/// multiplications instead of eight (after simdjson and fast_float); always
|
||||
/// inlined, as GCC otherwise calls it in the number loops
|
||||
JSON_HEDLEY_ALWAYS_INLINE std::uint32_t parse_eight_digits(std::uint64_t v) noexcept
|
||||
{
|
||||
v = ((v & 0x0F0F0F0F0F0F0F0Fu) * 2561u) >> 8u;
|
||||
v = ((v & 0x00FF00FF00FF00FFu) * 6553601u) >> 16u;
|
||||
|
||||
Reference in new issue
Block a user