Binary readers and writers: faster UTF-8 checks of strings

With error_handler_t::strict (and replace/ignore), every string a binary
reader reads is checked for valid UTF-8, and every string a binary writer
writes. The readers used the byte-at-a-time decoder; the writers skipped
ASCII eight bytes at a time.

- New valid_utf8_string() decides strings of up to 16 ASCII bytes, most
  keys and many values, with one or two word tests, and checks longer
  strings 16 bytes at a time with the NEON/SSSE3 UTF-8 check the lexer
  uses (vector_utf8_valid_blocks: the same lookup, but quotes and control
  characters are ordinary characters). Other targets and JSON_NO_SIMD use
  valid_utf8_prefix() as before.
- The readers check strings with it (for one-byte character types) in an
  out-of-line function, so that reading without a check does not pay for
  its code.
- The writers' check_utf8() uses it, and the CBOR and MessagePack writers
  write a checked string of up to max_text_in_reservation characters in
  one reservation, as they write unchecked ones.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

include/nlohmann/detail/input/binary_reader.hpp

@@ -4293,6 +4293,19 @@ class binary_reader
         return get_bytes(len, "string", result);
     }
 
+    /// whether @a s is valid UTF-8: ASCII eight bytes at a time, the rest
+    /// sequence by sequence (as the binary writers check strings)
+    static bool is_valid_utf8_string(const string_t& s, std::true_type /*is_byte*/) noexcept
+    {
+        return valid_utf8_string(reinterpret_cast<const unsigned char*>(s.data()), s.size());
+    }
+
+    /// whether @a s, of a character type wider than a byte, is valid UTF-8
+    static bool is_valid_utf8_string(const string_t& s, std::false_type /*is_byte*/) noexcept
+    {
+        return is_valid_utf8(s);
+    }
+
     /*!
     @brief validate a decoded text string (value or object key) against @ref error_handler
 
@@ -4312,7 +4325,16 @@ class binary_reader
     */
     bool check_string_utf8(string_t& result, const char* context)
     {
-        if (error_handler == error_handler_t::keep || is_valid_utf8(result))
+        return error_handler == error_handler_t::keep || check_string_utf8_handler(result, context);
+    }
+
+    /// @ref check_string_utf8 for an error handler other than @ref
+    /// error_handler_t::keep; out of line, so that reading strings without a
+    /// check does not pay for the code of the check
+    JSON_HEDLEY_NEVER_INLINE
+    bool check_string_utf8_handler(string_t& result, const char* context)
+    {
+        if (is_valid_utf8_string(result, std::integral_constant<bool, sizeof(typename string_t::value_type) == 1>()))
         {
             return true;
         }