valid_utf8_string() checks long strings 16 bytes at a time in
vector_utf8_valid_blocks(), which left its loop at the first block with
an error. Clang compiles "if (error) break; block += 16;" into a
conditional select on the block pointer, so each block waited for the
complete check of the block before: 28 cycles for 16 bytes on Apple M1.
The errors of all blocks are now collected and tested once at the end.
ASCII blocks are skipped in a loop of their own that needs none of the
lookup tables, and from the first block that is not ASCII, eight blocks
are checked without testing for ASCII in between, which was a
mispredicted branch for text with a few non-ASCII characters. If any
block had an error, the caller finds it one sequence at a time from the
start of the string, as before from the block with the error.
The check is used by the binary writers and readers with an error
handler other than keep, and always by BON8. On Apple M1 (Apple Clang),
it needs 0.27 instead of 1.78 cycles per byte for non-ASCII text, and
to_msgpack() with error_handler_t::strict is 24 to 43 % faster for such
text. With GCC 15 on x86-64, where the loop was not affected, the check
itself is 13 to 35 % faster.
A new test puts well-formed and ill-formed sequences at every offset of
strings that are longer than eight blocks.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann
9bee955345