SWAR and vector scans: test for special bytes in fewer operations (#6033)
* SWAR: test for special bytes in the reduced form The usual test for a zero byte, (x - ones) & ~x & high, needs `& ~x` only to keep bytes from 0x80 out. The tests for quotes, backslashes and control characters in string_scan.hpp and the serializer also flag every byte from 0x80 with `v & high`, so the `& ~x` terms can be dropped. Clang finds this reduction itself; GCC does not and emits 24 instead of 16 instructions. The masks are the same for 1.3 billion random words and for every pair of adjacent byte values. With GCC 15 on x86-64: accept() -2 to -4 %, sax_parse() -2 %, dump() -3 %. Clang is unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Vector scans: one compare for the quote and the control characters The SSE2/NEON tests for the bytes a string run stops at compared with the quote, with the backslash and with 0x20 separately. Flipping bit 1 maps 0x00-0x1F onto itself and the quote onto 0x20, and every other ASCII byte to 33 or more, so (c ^ 2) < 33 finds the quote and the control characters with one compare: signed where the bytes from 0x80 stop the run too, unsigned (or with a saturating subtraction / minimum on SSE2) where they do not. This saves one vector operation per 16 bytes in all twelve vector tests: the lexer's vector_plain_run() and vector_utf8_run(), dump()'s copy_plain_halves() and copy_plain_vectors(), and json_view's scanner and serializer. The DEL tests of the ensure_ascii paths stay. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Tests: sweep_allocator::construct() accepts pointers to const Recent libc++ (emsdk latest) probes construct(const T*, const T&) in an unevaluated context while checking for trivial copies; the static_cast to void* in the noexcept specification then fails to compile. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz (cherry picked from commit 794bf70823c12a2a1cb2a8a95f34dfb6357746a8) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Tests: guard sweep_allocator::construct() against const types instead of const_cast The const_cast added in the previous commit is rejected by cppcoreguidelines-pro-type-const-cast (clang-tidy). Make construct() not viable for const U instead, which is what libc++ probes for. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz (cherry picked from commit de099fa607001b710862fd1bbecef3609c54cbc4) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix CI: astyle and clang-tidy for sweep_allocator::construct() astyle 3.4.13 reformats the enable_if<!...> default template argument (Check amalgamation). Delete the const overload instead, which makes the libc++ probe for const objects fail in the SFINAE context just the same and needs no <type_traits>. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
8c0b48088c
commit
2a4aec3e69
7 files changed
+156
-120
No files matched your search
@@ -58,11 +58,10 @@ inline std::uint64_t swar_string_special(std::uint64_t v) noexcept
|
||||
constexpr std::uint64_t high = 0x8080808080808080ull;
|
||||
const std::uint64_t q = v ^ 0x2222222222222222ull; // '"' (0x22)
|
||||
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C)
|
||||
const std::uint64_t has_quote = (q - ones) & ~q & high;
|
||||
const std::uint64_t has_backslash = (b - ones) & ~b & high;
|
||||
const std::uint64_t has_control = (v - 0x2020202020202020ull) & ~v & high; // < 0x20
|
||||
const std::uint64_t has_non_ascii = v & high; // >= 0x80
|
||||
return has_quote | has_backslash | has_control | has_non_ascii;
|
||||
// The usual test for a zero byte is (x - ones) & ~x & high. The `& ~x`
|
||||
// only removes bytes from 0x80, and `v & high` flags those anyway, so it
|
||||
// can be dropped. Clang finds this reduction itself; GCC does not.
|
||||
return ((q - ones) | (b - ones) | (v - 0x2020202020202020ull) | v) & high;
|
||||
}
|
||||
|
||||
// return the index of the first is_string_special() byte in [data, data+n), or
|
||||
@@ -143,11 +142,8 @@ inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_
|
||||
const std::uint64_t q = v ^ 0x2222222222222222ull; // '"' (0x22)
|
||||
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C)
|
||||
const std::uint64_t d = v ^ 0x7F7F7F7F7F7F7F7Full; // DEL (0x7F)
|
||||
const std::uint64_t stop = ((q - ones) & ~q & high) // == '"'
|
||||
| ((b - ones) & ~b & high) // == '\\'
|
||||
| ((d - ones) & ~d & high) // == 0x7F
|
||||
| ((v - 0x2020202020202020ull) & ~v & high) // < 0x20
|
||||
| (v & high); // >= 0x80
|
||||
// (see swar_string_special(): the bytes from 0x80 are stops anyway)
|
||||
const std::uint64_t stop = ((q - ones) | (b - ones) | (d - ones) | (v - 0x2020202020202020ull) | v) & high;
|
||||
if (stop != 0)
|
||||
{
|
||||
// the lowest flagged byte is the first one to stop at (see
|
||||
|
||||
Reference in new issue
Block a user