Validate non-ASCII strings with SSSE3 on every x86-64 CPU that has it
The vector UTF-8 check of json_view needed SSSE3 at compile time (JSON_VIEW_USE_SSSE3 with -mssse3), so default x86-64 builds validated non-ASCII text one sequence at a time. The check is now compiled for SSSE3 with a function attribute (GCC 4.9 and later, Clang; MSVC compiles the intrinsics anyway) and used where CPUID reports SSSE3. The answer is kept in an atomic that is initialized at compile time, so neither a guard nor a global constructor is needed. The definitions do not depend on compiler flags, so there is no ODR issue. JSON_VIEW_USE_SSSE3 now only skips the CPU check. On x86-64 Linux (Haswell), twitter.json parses 23% faster with Clang 18 and 25-34% faster with GCC 13, now ahead of yyjson. Also always inline read_eight_bytes() and parse_eight_digits(): GCC called both in the number loops of the lexer and of json_view (52 call sites), which cost about 10% on citm_catalog.json at -O2. Document that reusing a document with read() avoids the page faults of a fresh node index (about 40% of a 55 MB parse on x86-64 Linux). Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
9 files changed
+173
-36
No files matched your search
@@ -8860,8 +8860,9 @@ inline uint128_parts full_multiplication(std::uint64_t a, std::uint64_t b) noexc
|
||||
}
|
||||
|
||||
/// eight bytes as a little-endian word (compilers fold this into one load on
|
||||
/// little-endian targets)
|
||||
inline std::uint64_t read_eight_bytes(const unsigned char* b) noexcept
|
||||
/// little-endian targets; always inlined, as GCC otherwise calls it in the
|
||||
/// number loops)
|
||||
JSON_HEDLEY_ALWAYS_INLINE std::uint64_t read_eight_bytes(const unsigned char* b) noexcept
|
||||
{
|
||||
return static_cast<std::uint64_t>(b[0]) | (static_cast<std::uint64_t>(b[1]) << 8u)
|
||||
| (static_cast<std::uint64_t>(b[2]) << 16u) | (static_cast<std::uint64_t>(b[3]) << 24u)
|
||||
@@ -8870,7 +8871,7 @@ inline std::uint64_t read_eight_bytes(const unsigned char* b) noexcept
|
||||
}
|
||||
|
||||
/// eight bytes as a little-endian word
|
||||
inline std::uint64_t read_eight_bytes(const char* p) noexcept
|
||||
JSON_HEDLEY_ALWAYS_INLINE std::uint64_t read_eight_bytes(const char* p) noexcept
|
||||
{
|
||||
return read_eight_bytes(reinterpret_cast<const unsigned char*>(p)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
|
||||
}
|
||||
@@ -9479,8 +9480,9 @@ template<typename FloatType>
|
||||
using native_float_t = typename std::conditional<std::numeric_limits<FloatType>::digits == 24, float, double>::type;
|
||||
|
||||
/// the value of the eight ASCII digits in @a v (see read_eight_bytes()), three
|
||||
/// multiplications instead of eight (after simdjson and fast_float)
|
||||
inline std::uint32_t parse_eight_digits(std::uint64_t v) noexcept
|
||||
/// multiplications instead of eight (after simdjson and fast_float); always
|
||||
/// inlined, as GCC otherwise calls it in the number loops
|
||||
JSON_HEDLEY_ALWAYS_INLINE std::uint32_t parse_eight_digits(std::uint64_t v) noexcept
|
||||
{
|
||||
v = ((v & 0x0F0F0F0F0F0F0F0Fu) * 2561u) >> 8u;
|
||||
v = ((v & 0x00FF00FF00FF00FFu) * 6553601u) >> 16u;
|
||||
|
||||
Reference in new issue
Block a user