With ensure_ascii, every non-ASCII character went through the UTF-8
state machine one byte at a time and its \uXXXX escape was assembled in
string_buffer and copied to the write buffer afterwards.
A run of well-formed multi-byte characters is now decoded and escaped
directly into the write buffer. The first byte that is ASCII or starts an
ill-formed or truncated sequence ends the run and is left to the
unchanged byte-at-a-time path, so error handling and its messages
(positions, replacement, ignore, keep) are as before.
Apple M1, Apple clang, retired instructions per dump(-1, ' ', true):
cjk text 15.9M -> 6.94M (-56 %) twitter 9.10M -> 6.16M (-32 %)
emoji text 8.89M -> 6.08M (-32 %) latin text 10.3M -> 9.23M (-10 %)
Pure ASCII documents with escapes (logs, JSON in JSON) and all modes
without ensure_ascii are unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann
209348c151