Merge branch develop into claude/perf-iterator-ranges

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

BUILD.bazel

@@ -30,7 +30,6 @@ cc_library(
         "include/nlohmann/detail/exceptions.hpp",
         "include/nlohmann/detail/hash.hpp",
         "include/nlohmann/detail/input/binary_reader.hpp",
-        "include/nlohmann/detail/input/contiguous_parser.hpp",
         "include/nlohmann/detail/input/input_adapters.hpp",
         "include/nlohmann/detail/input/json_sax.hpp",
         "include/nlohmann/detail/input/lexer.hpp",

docs/mkdocs/docs/api/macros/json_diagnostic_positions.md

@@ -34,9 +34,6 @@ or quotes.
 Note that enabling this macro increases the size of every JSON value by two `std::size_t` fields and adds slight runtime
 overhead to parsing, copying JSON value objects, and the generation of error messages for exceptions. It also causes
 these values to be reported in those error messages.
-In addition, `parse()`, `accept()`, and `sax_parse()` no longer use the
-[faster path for input in one block of memory](../../features/parsing/index.md#input-sources-and-performance), so
-parsing a string or a buffer is slower than without the macro.
 
 If [`JSON_DIAGNOSTICS`](json_diagnostics.md) is also set, the message contains the path and the byte range, such as
 `(/1) (bytes 4-8)`. An exception about the root value has no path and only the range, such as `(bytes 0-4)`.

docs/mkdocs/docs/api/macros/json_disable_stream_bulk_scan.md

@@ -48,8 +48,8 @@ The default value is `0`: streams are scanned in bulk.
 !!! note "Performance"
 
     Parsing from a `std::ifstream` or `std::istringstream` is typically 1.5 to 2 times as fast as before, and reading
-    many small values from one stream with `operator>>` 1.4 to 4 times as fast. Parsing from a stream then takes about
+    many small values from one stream with `operator>>` 1.4 to 4 times as fast. Parsing from a stream is then within
-    1.15 to 1.45 times as long as first reading the whole file into a `std::string` and parsing that.
+    about 10 to 20 % of first reading the whole file into a `std::string` and parsing that.
 
     A stream buffer without a get area (one that only overrides `underflow()` and `uflow()` and hands out every
     character from there) offers nothing to scan in bulk, so it is read one character at a time as before.

docs/mkdocs/docs/features/files.md

@@ -32,11 +32,11 @@ json read_json_file(const std::string& path)
   column 1 instead of saying that the file is missing. A relative path is resolved against the working directory of the
   running program, which is often not the directory of the source file (see the
   [FAQ](../home/faq.md#parsing-a-file-fails-with-unexpected-end-of-input)).
-- **Read the file into a string only if the memory is no concern.** Parsing from a `#!cpp std::ifstream` takes about
+- **Don't read the file into a string first.** Parsing from a `#!cpp std::ifstream` is only about 10 to 20 % slower
-  1.15 to 1.45 times as long as reading the whole file into a string with one `read()` and parsing that, but it needs no
+  than reading the whole file into a string with one `read()` and parsing that, and it needs no memory for the copy (see
-  memory for the copy (see [input sources and performance](parsing/index.md#input-sources-and-performance)). The common
+  [input sources and performance](parsing/index.md#input-sources-and-performance)). The common way to read a file with
-  way to read a file with `#!cpp std::istreambuf_iterator<char>` is so slow that it takes about twice as long as parsing
+  `#!cpp std::istreambuf_iterator<char>` is so slow that it takes about twice as long as parsing from the stream. Read
-  from the stream. This advice is for
+  the file into a string only if you need the text itself, for example for an error message. This advice is for
   `#!cpp json::parse`. [`json_document`](json_view.md) reads a stream into a buffer that it owns anyway, so there is no
   copy to save: reading the file into a `#!cpp std::string` yourself and passing that as an lvalue, which the document
   borrows, was about 6 to 12 % faster than parsing from the stream, with the same single buffer.

docs/mkdocs/docs/features/parsing/index.md

@@ -80,16 +80,8 @@ them faster than others:
 | `#!cpp FILE*`                                                                              | with glibc and the Apple C library, the characters the `FILE` holds in its buffer are scanned in bulk like a stream's; elsewhere one character at a time; the file is locked once for the whole parse |
 | other iterator ranges (e.g., `#!cpp std::list<char>`, `#!cpp std::istreambuf_iterator<char>`), wide strings | one character at a time                                                             |
 
-Input in one block of memory is read even faster with the default options (no
+Parsing a file from a `#!cpp std::ifstream` is therefore only about 10 to 20 % slower than reading the whole file into a
-[parser callback](../../api/basic_json/parser_callback_t.md), no comments, no trailing commas, and for `sax_parse()`
+`#!cpp std::string` first and parsing that, and it needs no memory for a copy of the file. The stream is never read
-`strict = true`): `parse()`, `accept()`, and `sax_parse()` then read the grammar and the tokens in one loop over the
-memory. Only after an error do `parse()` and `sax_parse()` read the input again, to report the error exactly as before.
-This faster path is not used when [`JSON_DIAGNOSTIC_POSITIONS`](../../api/macros/json_diagnostic_positions.md) is
-enabled, because the positions are recorded by the default parser: the input is then parsed as before, at the same speed
-as before.
-
-Parsing a file from a `#!cpp std::ifstream` therefore takes about 1.15 to 1.45 times as long as reading the whole file
-into a `#!cpp std::string` first and parsing that, but it needs no memory for a copy of the file. The stream is never read
 ahead: after parsing, it is positioned exactly as if every character had been read one at a time, so
 [`operator>>`](../../api/operator_gtgt.md) can still read several values from one stream (see also
 [JSON Lines](json_lines.md)).

docs/mkdocs/docs/features/performance.md

@@ -29,9 +29,9 @@ up. Nothing is read ahead, so the stream is left exactly where it would be after
 [`operator>>`](../api/operator_gtgt.md) can still read several values from one stream. A stream buffer without a get
 area is read one character at a time. A `#!cpp FILE*` is locked once per parse instead of once per character, and with
 glibc and the Apple C library, the characters in its buffer are scanned in bulk as a stream's get area is. See
-[input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. Parsing from a
+[input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. According to the
-stream takes about 1.15 to 1.45 times as long as parsing an equivalent `#!cpp std::string`, which `parse()` reads with
+description of [#5828](https://github.com/nlohmann/json/pull/5828), parsing from a stream is within about 10 to 20 % of
-a parser of its own (see there).
+parsing an equivalent `#!cpp std::string`.
 
 Inputs that are still read one character at a time: iterator ranges that are not contiguous (e.g.
 `#!cpp std::list<char>` or `#!cpp std::istreambuf_iterator<char>`), wide strings, and user-defined adapters. An

include/nlohmann/detail/input/input_adapters.hpp

@@ -1348,6 +1348,64 @@ struct iterator_input_adapter_factory<IteratorType, SentinelType, enable_if_t<is
     }
 };
 
+/*!
+@brief whether @a IteratorType is an iterator of std::string or of a
+       std::vector of 1-byte elements
+
+Their elements are contiguous, but before C++20 that cannot be detected from
+the iterator type (std::contiguous_iterator), so an iterator pair of them would
+be read one character at a time through the iterators. They are read as the
+pointer range they are instead, as the containers themselves are (see the
+container overload of input_adapter()). The types are matched exactly, so no
+other iterator can be mistaken for one. (std::basic_string is only matched for
+char: other character types need std::char_traits specializations that the
+standard library need not have.)
+*/
+template<typename CharType>
+struct is_byte_char_type
+{
+    // a member, not an integral_constant argument, which astyle misformats
+    static constexpr bool value = sizeof(CharType) == 1 && std::is_integral<CharType>::value && !std::is_same<CharType, bool>::value;
+};
+
+template<typename IteratorType, typename CharType, bool IsByte = is_byte_char_type<CharType>::value>
+struct is_std_contiguous_byte_iterator_impl : std::false_type {};
+
+// (the containers are only named for byte types)
+template<typename IteratorType, typename CharType>
+struct is_std_contiguous_byte_iterator_impl<IteratorType, CharType, true>
+{
+    static constexpr bool value = !std::is_pointer<IteratorType>::value
+                                  && (std::is_same<IteratorType, typename std::vector<CharType>::iterator>::value
+                                      || std::is_same<IteratorType, typename std::vector<CharType>::const_iterator>::value
+                                      || std::is_same<IteratorType, std::string::iterator>::value
+                                      || std::is_same<IteratorType, std::string::const_iterator>::value);
+};
+
+template<typename IteratorType>
+struct is_std_contiguous_byte_iterator
+    : is_std_contiguous_byte_iterator_impl<IteratorType, typename std::remove_cv<typename std::iterator_traits<IteratorType>::value_type>::type> {};
+
+template<typename IteratorType>
+struct iterator_input_adapter_factory<IteratorType, IteratorType, enable_if_t<is_std_contiguous_byte_iterator<IteratorType>::value>>
+{
+    using iterator_type = IteratorType;
+    using sentinel_type = IteratorType;
+    using char_type = typename std::remove_cv<typename std::iterator_traits<iterator_type>::value_type>::type;
+    using adapter_type = iterator_input_adapter<const char_type*>;
+
+    static adapter_type create(IteratorType first, IteratorType last)
+    {
+        if (first == last)
+        {
+            // (an empty range must not be dereferenced)
+            return adapter_type(nullptr, nullptr);
+        }
+        const char_type* const begin = std::addressof(*first);
+        return adapter_type(begin, begin + (last - first));
+    }
+};
+
 // General purpose iterator-based input (iterator+sentinel pair; SentinelType
 // defaults to IteratorType for the common same-type case, but may differ for
 // C++20 ranges-style iterator+sentinel pairs). Only enable for types that can

include/nlohmann/detail/input/number_parse.hpp

@@ -316,30 +316,6 @@ JSON_HEDLEY_ALWAYS_INLINE std::size_t accumulate_digits(const char* data, std::s
     return i;
 }
 
-/// the index of the first byte at or after @a i that is not a digit: the
-/// digits of accumulate_digits() without their value
-JSON_HEDLEY_ALWAYS_INLINE std::size_t skip_digits(const char* data, std::size_t i, std::size_t avail) noexcept
-{
-    while (avail - i >= 8)
-    {
-        if (!is_eight_digits(read_eight_bytes(data + i)))
-        {
-            // a byte of the word is not a digit: no bounds checks before it
-            while (static_cast<unsigned char>(data[i] - '0') <= 9)
-            {
-                ++i;
-            }
-            return i;
-        }
-        i += 8;
-    }
-    while (i < avail && data[i] >= '0' && data[i] <= '9')
-    {
-        ++i;
-    }
-    return i;
-}
-
 /// whether [first, last) contains a digit other than '0'; a plain loop, as
 /// GCC calls std::any_of out of line even for the usual empty range
 JSON_HEDLEY_ALWAYS_INLINE bool has_nonzero_digit(const char* first, const char* last) noexcept

include/nlohmann/detail/input/simd_scan.hpp

@@ -92,12 +92,8 @@ namespace detail
 character, or not ASCII, 16 bytes per step
 
 Stops at such a byte, or where fewer than 16 bytes are left (the caller tells
-the two apart). One signed compare finds the quote, the control characters and
+the two apart). A signed compare with 0x20 finds control characters and
-the non-ASCII bytes at once: flipping bit 1 maps 0x00-0x1F onto itself and the
+non-ASCII bytes at once.
-quote (0x22) onto 0x20, keeps 0x80-0xFF negative, and maps every other byte to
-33 or more, so these are exactly the bytes with (int8_t)(c ^ 2) < 33 (see
-D. Lemire, "Detect control characters, quotes and backslashes efficiently
-using SWAR", 2025). The backslash needs its own compare.
 */
 JSON_HEDLEY_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned char* p, const unsigned char* e) noexcept
 {
@@ -105,8 +101,8 @@ JSON_HEDLEY_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned c
     {
 #if JSON_SCAN_NEON
         const uint8x16_t in = vld1q_u8(p);
-        const uint8x16_t special = vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')),
+        const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-                                            vcltq_s8(vreinterpretq_s8_u8(veorq_u8(in, vdupq_n_u8(2))), vdupq_n_s8(33)));
+                                            vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
         // one nibble per byte (the usual NEON replacement of x86's movemask, see
         // D. Kutenin, "Porting x86 vector bitmask optimizations to Arm NEON", 2022)
         const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
@@ -116,8 +112,8 @@ JSON_HEDLEY_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned c
         }
 #else
         const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
-        const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                             _mm_cmplt_epi8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(33)));
+                                             _mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
         const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
         if (bits != 0)
         {
@@ -254,8 +250,7 @@ JSON_SCAN_SSSE3_TARGET JSON_HEDLEY_NEVER_INLINE inline bool vector_utf8_run(cons
     while (e - block >= 16)
     {
         const uint8x16_t in = vld1q_u8(block);
-        // the quote and the control characters: (c ^ 2) < 33 (see vector_plain_run())
+        const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))), vcltq_u8(in, vdupq_n_u8(0x20)));
-        const uint8x16_t special = vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')), vcltq_u8(veorq_u8(in, vdupq_n_u8(2)), vdupq_n_u8(33)));
         const uint8x16_t prev1 = vextq_u8(prev, in, 15);
         const uint8x16_t sc = vandq_u8(vandq_u8(vqtbl1q_u8(t1h, vshrq_n_u8(prev1, 4)), vqtbl1q_u8(t1l, vandq_u8(prev1, vdupq_n_u8(0x0F)))), vqtbl1q_u8(t2h, vshrq_n_u8(in, 4)));
         const uint8x16_t must23 = vorrq_u8(vqsubq_u8(vextq_u8(prev, in, 14), vdupq_n_u8(0xE0 - 0x80)), vqsubq_u8(vextq_u8(prev, in, 13), vdupq_n_u8(0xF0 - 0x80)));
@@ -295,10 +290,8 @@ JSON_SCAN_SSSE3_TARGET JSON_HEDLEY_NEVER_INLINE inline bool vector_utf8_run(cons
     while (e - block >= 16)
     {
         const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(block))); // NOLINT(bugprone-casting-through-void)
-        // the quote and the control characters: (c ^ 2) < 33, that is,
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-        // (c ^ 2) - 32 saturates to 0 (see vector_plain_run())
+                                             _mm_cmpeq_epi8(_mm_subs_epu8(in, _mm_set1_epi8(0x1F)), zero)); // in < 0x20
-        const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
-                                             _mm_cmpeq_epi8(_mm_subs_epu8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(32)), zero));
         const __m128i prev1 = _mm_alignr_epi8(in, prev, 15);
         const __m128i sc = _mm_and_si128(_mm_and_si128(_mm_shuffle_epi8(t1h, _mm_and_si128(_mm_srli_epi16(prev1, 4), nibble)),
                                          _mm_shuffle_epi8(t1l, _mm_and_si128(prev1, nibble))),

include/nlohmann/detail/input/string_scan.hpp

@@ -58,10 +58,11 @@ inline std::uint64_t swar_string_special(std::uint64_t v) noexcept
     constexpr std::uint64_t high = 0x8080808080808080ull;
     const std::uint64_t q = v ^ 0x2222222222222222ull; // '"'  (0x22)
     const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C)
-    // The usual test for a zero byte is (x - ones) & ~x & high. The `& ~x`
+    const std::uint64_t has_quote     = (q - ones) & ~q & high;
-    // only removes bytes from 0x80, and `v & high` flags those anyway, so it
+    const std::uint64_t has_backslash = (b - ones) & ~b & high;
-    // can be dropped. Clang finds this reduction itself; GCC does not.
+    const std::uint64_t has_control   = (v - 0x2020202020202020ull) & ~v & high; // < 0x20
-    return ((q - ones) | (b - ones) | (v - 0x2020202020202020ull) | v) & high;
+    const std::uint64_t has_non_ascii = v & high;                                // >= 0x80
+    return has_quote | has_backslash | has_control | has_non_ascii;
 }
 
 // return the index of the first is_string_special() byte in [data, data+n), or
@@ -142,8 +143,11 @@ inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_
         const std::uint64_t q = v ^ 0x2222222222222222ull; // '"'  (0x22)
         const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C)
         const std::uint64_t d = v ^ 0x7F7F7F7F7F7F7F7Full; // DEL  (0x7F)
-        // (see swar_string_special(): the bytes from 0x80 are stops anyway)
+        const std::uint64_t stop = ((q - ones) & ~q & high)   // == '"'
-        const std::uint64_t stop = ((q - ones) | (b - ones) | (d - ones) | (v - 0x2020202020202020ull) | v) & high;
+                                   | ((b - ones) & ~b & high)  // == '\\'
+                                   | ((d - ones) & ~d & high)  // == 0x7F
+                                   | ((v - 0x2020202020202020ull) & ~v & high) // < 0x20
+                                   | (v & high);               // >= 0x80
         if (stop != 0)
         {
             // the lowest flagged byte is the first one to stop at (see
@@ -516,10 +520,7 @@ JSON_HEDLEY_NEVER_INLINE inline std::size_t string_bulk_run_from(const unsigned
 }
 #endif
 
-// string_bulk_run() for the lexer. The run must be the longest one: the lexer
+// string_bulk_run() for the lexer. With NEON, only the first four words are
-// would only lose speed with a shorter run, but contiguous_parser rejects a
-// string whose run ends at a byte other than a quote or a backslash, so a
-// shorter run would reject valid input there. With NEON, only the first four words are
 // checked inline (most keys and short values end there), and longer runs and
 // non-ASCII text continue out of line. The lexer's scan function runs once per
 // token, and the vector code inlined into it needs more registers, which every

include/nlohmann/detail/output/serializer.hpp

@@ -724,8 +724,10 @@ class serializer
         constexpr std::uint64_t ones = 0x0101010101010101ull;
         const std::uint64_t q = v ^ 0x2222222222222222ull; // '"'
         const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\'
-        // (see swar_string_special(): the bytes from 0x80 are flagged anyway)
+        return (((q - ones) & ~q) | ((b - ones) & ~b)         // == '"', == '\\'
-        return ((q - ones) | (b - ones) | (v - 0x2020202020202020ull) | v | (v + ones)) & 0x8080808080808080ull;
+                | ((v - 0x2020202020202020ull) & ~v)          // < 0x20
+                | v | (v + ones))                             // >= 0x7F
+               & 0x8080808080808080ull;
     }
 
     template<typename WordType>
@@ -834,10 +836,8 @@ class serializer
         const uint8x8_t lo = vld1_u8(reinterpret_cast<const std::uint8_t*>(p));
         const uint8x8_t hi = vld1_u8(reinterpret_cast<const std::uint8_t*>(p + n - 8));
         const uint8x16_t in = vcombine_u8(lo, hi);
-        // (int8_t)(c ^ 2) < 33: the quote, the control characters and the
+        const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-        // non-ASCII bytes (see vector_plain_run())
+                                            vorrq_u8(vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)), vceqq_u8(in, vdupq_n_u8(0x7F))));
-        const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')), vceqq_u8(in, vdupq_n_u8(0x7F))),
-                                            vcltq_s8(vreinterpretq_s8_u8(veorq_u8(in, vdupq_n_u8(2))), vdupq_n_s8(33)));
         if (vmaxvq_u8(special) != 0)
         {
             return false;
@@ -848,8 +848,8 @@ class serializer
         const __m128i lo = _mm_loadl_epi64(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
         const __m128i hi = _mm_loadl_epi64(static_cast<const __m128i*>(static_cast<const void*>(p + n - 8))); // NOLINT(bugprone-casting-through-void)
         const __m128i in = _mm_unpacklo_epi64(lo, hi);
-        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')), _mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F))),
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                             _mm_cmplt_epi8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(33)));
+                                             _mm_or_si128(_mm_cmplt_epi8(in, _mm_set1_epi8(0x20)), _mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F))));
         if (_mm_movemask_epi8(special) != 0)
         {
             return false;
@@ -868,13 +868,10 @@ class serializer
 #if JSON_SCAN_NEON
         const uint8x16_t a = vld1q_u8(reinterpret_cast<const std::uint8_t*>(p));
         const uint8x16_t b = vld1q_u8(reinterpret_cast<const std::uint8_t*>(p + n - 16));
-        // (int8_t)(c ^ 2) < 33: the quote, the control characters and the
+        const uint8x16_t special = vorrq_u8(vorrq_u8(vorrq_u8(vceqq_u8(a, vdupq_n_u8('"')), vceqq_u8(a, vdupq_n_u8('\\'))),
-        // non-ASCII bytes (see vector_plain_run())
+                                            vorrq_u8(vceqq_u8(b, vdupq_n_u8('"')), vceqq_u8(b, vdupq_n_u8('\\')))),
-        const uint8x16_t flip = vdupq_n_u8(2);
+                                            vorrq_u8(vorrq_u8(vcltq_s8(vreinterpretq_s8_u8(a), vdupq_n_s8(0x20)), vcltq_s8(vreinterpretq_s8_u8(b), vdupq_n_s8(0x20))),
-        const int8x16_t limit = vdupq_n_s8(33);
+                                                vorrq_u8(vceqq_u8(a, vdupq_n_u8(0x7F)), vceqq_u8(b, vdupq_n_u8(0x7F)))));
-        const uint8x16_t special = vorrq_u8(vorrq_u8(vorrq_u8(vceqq_u8(a, vdupq_n_u8('\\')), vceqq_u8(b, vdupq_n_u8('\\'))),
-                                            vorrq_u8(vceqq_u8(a, vdupq_n_u8(0x7F)), vceqq_u8(b, vdupq_n_u8(0x7F)))),
-                                            vorrq_u8(vcltq_s8(vreinterpretq_s8_u8(veorq_u8(a, flip)), limit), vcltq_s8(vreinterpretq_s8_u8(veorq_u8(b, flip)), limit)));
         if (vmaxvq_u8(special) != 0)
         {
             return false;
@@ -884,15 +881,14 @@ class serializer
 #else
         const __m128i a = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
         const __m128i b = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p + n - 16))); // NOLINT(bugprone-casting-through-void)
-        // (int8_t)(c ^ 2) < 33: the quote, the control characters and the
+        const __m128i quote = _mm_set1_epi8('"');
-        // non-ASCII bytes (see vector_plain_run())
         const __m128i backslash = _mm_set1_epi8('\\');
+        const __m128i space = _mm_set1_epi8(0x20);
         const __m128i del = _mm_set1_epi8(0x7F);
-        const __m128i flip = _mm_set1_epi8(2);
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(a, quote), _mm_cmpeq_epi8(a, backslash)),
-        const __m128i limit = _mm_set1_epi8(33);
+                                             _mm_or_si128(_mm_cmpeq_epi8(b, quote), _mm_cmpeq_epi8(b, backslash))),
-        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(a, backslash), _mm_cmpeq_epi8(b, backslash)),
+                                             _mm_or_si128(_mm_or_si128(_mm_cmplt_epi8(a, space), _mm_cmplt_epi8(b, space)),
-                                             _mm_or_si128(_mm_cmpeq_epi8(a, del), _mm_cmpeq_epi8(b, del))),
+                                                 _mm_or_si128(_mm_cmpeq_epi8(a, del), _mm_cmpeq_epi8(b, del))));
-                                             _mm_or_si128(_mm_cmplt_epi8(_mm_xor_si128(a, flip), limit), _mm_cmplt_epi8(_mm_xor_si128(b, flip), limit)));
         if (_mm_movemask_epi8(special) != 0)
         {
             return false;

include/nlohmann/detail/view/serializer.hpp

@@ -152,12 +152,9 @@ inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexc
     for (; i + 16 <= n; i += 16)
     {
         const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(s + i))); // NOLINT(bugprone-casting-through-void)
-        // the quote and the control characters are the bytes with
+        // (a byte is below 0x20 if it is its own minimum with 0x1F)
-        // (c ^ 2) <= 32, that is, those that are their own minimum with 32
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-        // (see vector_plain_run())
+                                             _mm_cmpeq_epi8(_mm_min_epu8(in, _mm_set1_epi8(0x1F)), in));
-        const __m128i flipped = _mm_xor_si128(in, _mm_set1_epi8(2));
-        const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
-                                             _mm_cmpeq_epi8(_mm_min_epu8(flipped, _mm_set1_epi8(32)), flipped));
         const auto mask = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
         if (mask != 0)
         {
@@ -220,21 +217,20 @@ inline bool is_ascii_without_delete(const char* s, std::size_t n) noexcept
 
 #if NLOHMANN_VIEW_VECTOR
 /// the bytes of the 16 at p that ensure_ascii cannot copy (a quote, a
-/// backslash, a control character, DEL, or not ASCII; the signed compare
+/// backslash, a control character, DEL, or not ASCII; a signed compare with
-/// (int8_t)(c ^ 2) < 33 finds the quote, the control characters and the
+/// 0x20 finds control characters and non-ASCII bytes at once), one bit per
-/// non-ASCII bytes at once, see vector_plain_run()), one bit per byte (x86) or
+/// byte (x86) or one nibble per byte (NEON)
-/// one nibble per byte (NEON)
 NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t ascii_stop_mask(const unsigned char* p) noexcept
 {
 #if NLOHMANN_VIEW_NEON
     const uint8x16_t in = vld1q_u8(p);
-    const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')), vceqq_u8(in, vdupq_n_u8(0x7F))),
+    const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-                                        vcltq_s8(vreinterpretq_s8_u8(veorq_u8(in, vdupq_n_u8(2))), vdupq_n_s8(33)));
+                                        vorrq_u8(vceqq_u8(in, vdupq_n_u8(0x7F)), vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20))));
     return vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
 #else
     const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
-    const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')), _mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F))),
+    const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                         _mm_cmplt_epi8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(33)));
+                                         _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F)), _mm_cmplt_epi8(in, _mm_set1_epi8(0x20))));
     return static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
 #endif
 }

include/nlohmann/detail/view/simd.hpp

@@ -91,10 +91,8 @@ namespace view
 character, or not ASCII, 16 bytes per step
 
 Stops at such a byte, or where fewer than 16 bytes are left (the caller tells
-the two apart). One signed compare finds the quote, the control characters and
+the two apart). A signed compare with 0x20 finds control characters and
-the non-ASCII bytes at once: these are exactly the bytes with
+non-ASCII bytes at once.
-(int8_t)(c ^ 2) < 33 (see detail::vector_plain_run()). The backslash needs its
-own compare.
 */
 NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned char* p, const unsigned char* e) noexcept
 {
@@ -102,8 +100,8 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned
     {
 #if NLOHMANN_VIEW_NEON
         const uint8x16_t in = vld1q_u8(p);
-        const uint8x16_t special = vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')),
+        const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-                                            vcltq_s8(vreinterpretq_s8_u8(veorq_u8(in, vdupq_n_u8(2))), vdupq_n_s8(33)));
+                                            vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
         // one nibble per byte (the usual NEON replacement of x86's movemask, see
         // D. Kutenin, "Porting x86 vector bitmask optimizations to Arm NEON", 2022)
         const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
@@ -113,8 +111,8 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned
         }
 #else
         const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
-        const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                             _mm_cmplt_epi8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(33)));
+                                             _mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
         const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
         if (bits != 0)
         {
@@ -140,17 +138,15 @@ NLOHMANN_VIEW_ALWAYS_INLINE unsigned copy_plain_run16(const unsigned char* p, ch
 #if NLOHMANN_VIEW_NEON
     const uint8x16_t in = vld1q_u8(p);
     vst1q_u8(reinterpret_cast<std::uint8_t*>(w), in); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-    // (int8_t)(c ^ 2) < 33: the quote, the control characters and the
+    const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-    // non-ASCII bytes (see vector_plain_run())
+                                        vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
-    const uint8x16_t special = vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')),
-                                        vcltq_s8(vreinterpretq_s8_u8(veorq_u8(in, vdupq_n_u8(2))), vdupq_n_s8(33)));
     const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
     return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits) >> 2u) : 16u;
 #else
     const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
     _mm_storeu_si128(static_cast<__m128i*>(static_cast<void*>(w)), in); // NOLINT(bugprone-casting-through-void)
-    const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
+    const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                         _mm_cmplt_epi8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(33)));
+                                         _mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
     const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
     return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits)) : 16u;
 #endif
@@ -310,8 +306,7 @@ NLOHMANN_VIEW_SSSE3_TARGET NLOHMANN_VIEW_NOINLINE NLOHMANN_VIEW_NOCLONE inline c
     while (e - block >= 16)
     {
         const uint8x16_t in = vld1q_u8(block);
-        // the quote and the control characters: (c ^ 2) < 33 (see vector_plain_run())
+        const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))), vcltq_u8(in, vdupq_n_u8(0x20)));
-        const uint8x16_t special = vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')), vcltq_u8(veorq_u8(in, vdupq_n_u8(2)), vdupq_n_u8(33)));
         const uint8x16_t prev1 = vextq_u8(prev, in, 15);
         const uint8x16_t sc = vandq_u8(vandq_u8(vqtbl1q_u8(t1h, vshrq_n_u8(prev1, 4)), vqtbl1q_u8(t1l, vandq_u8(prev1, vdupq_n_u8(0x0F)))), vqtbl1q_u8(t2h, vshrq_n_u8(in, 4)));
         const uint8x16_t must23 = vorrq_u8(vqsubq_u8(vextq_u8(prev, in, 14), vdupq_n_u8(0xE0 - 0x80)), vqsubq_u8(vextq_u8(prev, in, 13), vdupq_n_u8(0xF0 - 0x80)));
@@ -350,10 +345,8 @@ NLOHMANN_VIEW_SSSE3_TARGET NLOHMANN_VIEW_NOINLINE NLOHMANN_VIEW_NOCLONE inline c
     while (e - block >= 16)
     {
         const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(block))); // NOLINT(bugprone-casting-through-void)
-        // the quote and the control characters: (c ^ 2) < 33, that is,
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-        // (c ^ 2) - 32 saturates to 0 (see vector_plain_run())
+                                             _mm_cmpeq_epi8(_mm_subs_epu8(in, _mm_set1_epi8(0x1F)), zero)); // in < 0x20
-        const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
-                                             _mm_cmpeq_epi8(_mm_subs_epu8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(32)), zero));
         const __m128i prev1 = _mm_alignr_epi8(in, prev, 15);
         const __m128i sc = _mm_and_si128(_mm_and_si128(_mm_shuffle_epi8(t1h, _mm_and_si128(_mm_srli_epi16(prev1, 4), nibble)),
                                          _mm_shuffle_epi8(t1l, _mm_and_si128(prev1, nibble))),

include/nlohmann/json.hpp

@@ -58,7 +58,6 @@
 #include <nlohmann/detail/exceptions.hpp> // IWYU pragma: export
 #include <nlohmann/detail/hash.hpp>
 #include <nlohmann/detail/input/binary_reader.hpp>
-#include <nlohmann/detail/input/contiguous_parser.hpp>
 #include <nlohmann/detail/input/input_adapters.hpp>
 #include <nlohmann/detail/input/json_sax.hpp>
 #include <nlohmann/detail/input/lexer.hpp>
@@ -3110,15 +3109,6 @@ public:
                                  const bool ignore_trailing_commas)
     {
         basic_json result;
-        // JSON in one block of memory, with the default options, is read by
-        // the faster contiguous_parser; on an error, the default parser reads
-        // it again to report it (see contiguous_parser)
-        using contiguous = std::integral_constant<bool, detail::can_parse_contiguous<basic_json, typename std::decay<InputAdapterType>::type>::value>;
-        if (cb == nullptr && !ignore_comments && !ignore_trailing_commas
-                && detail::parse_contiguous(adapter, result, allow_exceptions, contiguous {}))
-        {
-            return result;
-        }
         parser(std::forward<InputAdapterType>(adapter), std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas).parse(true, result);
         return result;
     }
@@ -3129,17 +3119,8 @@ public:
                             const bool ignore_comments,
                             const bool ignore_trailing_commas)
     {
-        // JSON in one block of memory, with the default options, is checked by
-        // the faster contiguous_parser (see parse_impl())
-        using adapter_t = typename std::decay<InputAdapterType>::type;
-        bool accepted = false;
-        if (!ignore_comments && !ignore_trailing_commas
-                && detail::accept_contiguous<basic_json>(adapter, accepted, std::integral_constant<bool, detail::can_parse_contiguous<basic_json, adapter_t>::value> {}))
-        {
-            return accepted;
-        }
         // the parser and lexer that discard the values of tokens (see lexer)
-        return ::nlohmann::detail::parser<basic_json, adapter_t, true>(std::forward<InputAdapterType>(adapter),
+        return ::nlohmann::detail::parser<basic_json, typename std::decay<InputAdapterType>::type, true>(std::forward<InputAdapterType>(adapter),
                 nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
     }
 
@@ -3154,15 +3135,6 @@ public:
     {
         if (format == input_format_t::json)
         {
-            // JSON in one block of memory, with the default options, is read by
-            // the faster contiguous_parser; on an error, the default parser
-            // reads it again to report it (see parse_impl())
-            bool result = false;
-            if (strict && !ignore_comments && !ignore_trailing_commas
-                    && detail::sax_parse_contiguous<basic_json>(ia, sax, result, std::integral_constant<bool, detail::can_parse_contiguous<basic_json, InputAdapterType>::value> {}))
-            {
-                return result;
-            }
             return parser(std::move(ia), nullptr, true, ignore_comments, ignore_trailing_commas).sax_parse(sax, strict);
         }
         return detail::binary_reader<basic_json, InputAdapterType, SAX>(std::move(ia), format).sax_parse(sax, strict, tag_handler);

single_include/nlohmann/json_view.hpp

@@ -759,10 +759,8 @@ namespace view
 character, or not ASCII, 16 bytes per step
 
 Stops at such a byte, or where fewer than 16 bytes are left (the caller tells
-the two apart). One signed compare finds the quote, the control characters and
+the two apart). A signed compare with 0x20 finds control characters and
-the non-ASCII bytes at once: these are exactly the bytes with
+non-ASCII bytes at once.
-(int8_t)(c ^ 2) < 33 (see detail::vector_plain_run()). The backslash needs its
-own compare.
 */
 NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned char* p, const unsigned char* e) noexcept
 {
@@ -770,8 +768,8 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned
     {
 #if NLOHMANN_VIEW_NEON
         const uint8x16_t in = vld1q_u8(p);
-        const uint8x16_t special = vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')),
+        const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-                                            vcltq_s8(vreinterpretq_s8_u8(veorq_u8(in, vdupq_n_u8(2))), vdupq_n_s8(33)));
+                                            vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
         // one nibble per byte (the usual NEON replacement of x86's movemask, see
         // D. Kutenin, "Porting x86 vector bitmask optimizations to Arm NEON", 2022)
         const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
@@ -781,8 +779,8 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned
         }
 #else
         const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
-        const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                             _mm_cmplt_epi8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(33)));
+                                             _mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
         const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
         if (bits != 0)
         {
@@ -808,17 +806,15 @@ NLOHMANN_VIEW_ALWAYS_INLINE unsigned copy_plain_run16(const unsigned char* p, ch
 #if NLOHMANN_VIEW_NEON
     const uint8x16_t in = vld1q_u8(p);
     vst1q_u8(reinterpret_cast<std::uint8_t*>(w), in); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-    // (int8_t)(c ^ 2) < 33: the quote, the control characters and the
+    const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-    // non-ASCII bytes (see vector_plain_run())
+                                        vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
-    const uint8x16_t special = vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')),
-                                        vcltq_s8(vreinterpretq_s8_u8(veorq_u8(in, vdupq_n_u8(2))), vdupq_n_s8(33)));
     const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
     return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits) >> 2u) : 16u;
 #else
     const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
     _mm_storeu_si128(static_cast<__m128i*>(static_cast<void*>(w)), in); // NOLINT(bugprone-casting-through-void)
-    const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
+    const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                         _mm_cmplt_epi8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(33)));
+                                         _mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
     const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
     return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits)) : 16u;
 #endif
@@ -978,8 +974,7 @@ NLOHMANN_VIEW_SSSE3_TARGET NLOHMANN_VIEW_NOINLINE NLOHMANN_VIEW_NOCLONE inline c
     while (e - block >= 16)
     {
         const uint8x16_t in = vld1q_u8(block);
-        // the quote and the control characters: (c ^ 2) < 33 (see vector_plain_run())
+        const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))), vcltq_u8(in, vdupq_n_u8(0x20)));
-        const uint8x16_t special = vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')), vcltq_u8(veorq_u8(in, vdupq_n_u8(2)), vdupq_n_u8(33)));
         const uint8x16_t prev1 = vextq_u8(prev, in, 15);
         const uint8x16_t sc = vandq_u8(vandq_u8(vqtbl1q_u8(t1h, vshrq_n_u8(prev1, 4)), vqtbl1q_u8(t1l, vandq_u8(prev1, vdupq_n_u8(0x0F)))), vqtbl1q_u8(t2h, vshrq_n_u8(in, 4)));
         const uint8x16_t must23 = vorrq_u8(vqsubq_u8(vextq_u8(prev, in, 14), vdupq_n_u8(0xE0 - 0x80)), vqsubq_u8(vextq_u8(prev, in, 13), vdupq_n_u8(0xF0 - 0x80)));
@@ -1018,10 +1013,8 @@ NLOHMANN_VIEW_SSSE3_TARGET NLOHMANN_VIEW_NOINLINE NLOHMANN_VIEW_NOCLONE inline c
     while (e - block >= 16)
     {
         const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(block))); // NOLINT(bugprone-casting-through-void)
-        // the quote and the control characters: (c ^ 2) < 33, that is,
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-        // (c ^ 2) - 32 saturates to 0 (see vector_plain_run())
+                                             _mm_cmpeq_epi8(_mm_subs_epu8(in, _mm_set1_epi8(0x1F)), zero)); // in < 0x20
-        const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
-                                             _mm_cmpeq_epi8(_mm_subs_epu8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(32)), zero));
         const __m128i prev1 = _mm_alignr_epi8(in, prev, 15);
         const __m128i sc = _mm_and_si128(_mm_and_si128(_mm_shuffle_epi8(t1h, _mm_and_si128(_mm_srli_epi16(prev1, 4), nibble)),
                                          _mm_shuffle_epi8(t1l, _mm_and_si128(prev1, nibble))),
@@ -8607,12 +8600,9 @@ inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexc
     for (; i + 16 <= n; i += 16)
     {
         const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(s + i))); // NOLINT(bugprone-casting-through-void)
-        // the quote and the control characters are the bytes with
+        // (a byte is below 0x20 if it is its own minimum with 0x1F)
-        // (c ^ 2) <= 32, that is, those that are their own minimum with 32
+        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-        // (see vector_plain_run())
+                                             _mm_cmpeq_epi8(_mm_min_epu8(in, _mm_set1_epi8(0x1F)), in));
-        const __m128i flipped = _mm_xor_si128(in, _mm_set1_epi8(2));
-        const __m128i special = _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')),
-                                             _mm_cmpeq_epi8(_mm_min_epu8(flipped, _mm_set1_epi8(32)), flipped));
         const auto mask = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
         if (mask != 0)
         {
@@ -8675,21 +8665,20 @@ inline bool is_ascii_without_delete(const char* s, std::size_t n) noexcept
 
 #if NLOHMANN_VIEW_VECTOR
 /// the bytes of the 16 at p that ensure_ascii cannot copy (a quote, a
-/// backslash, a control character, DEL, or not ASCII; the signed compare
+/// backslash, a control character, DEL, or not ASCII; a signed compare with
-/// (int8_t)(c ^ 2) < 33 finds the quote, the control characters and the
+/// 0x20 finds control characters and non-ASCII bytes at once), one bit per
-/// non-ASCII bytes at once, see vector_plain_run()), one bit per byte (x86) or
+/// byte (x86) or one nibble per byte (NEON)
-/// one nibble per byte (NEON)
 NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t ascii_stop_mask(const unsigned char* p) noexcept
 {
 #if NLOHMANN_VIEW_NEON
     const uint8x16_t in = vld1q_u8(p);
-    const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')), vceqq_u8(in, vdupq_n_u8(0x7F))),
+    const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-                                        vcltq_s8(vreinterpretq_s8_u8(veorq_u8(in, vdupq_n_u8(2))), vdupq_n_s8(33)));
+                                        vorrq_u8(vceqq_u8(in, vdupq_n_u8(0x7F)), vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20))));
     return vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
 #else
     const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
-    const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('\\')), _mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F))),
+    const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                         _mm_cmplt_epi8(_mm_xor_si128(in, _mm_set1_epi8(2)), _mm_set1_epi8(33)));
+                                         _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F)), _mm_cmplt_epi8(in, _mm_set1_epi8(0x20))));
     return static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
 #endif
 }

tests/src/unit-bson.cpp

@@ -1674,6 +1674,8 @@ TEST_CASE("BSON input that cannot be read is discarded by every overload")
     json _;
     CHECK_THROWS_AS(_ = json::from_bson(input.begin(), input.end()), json::parse_error&);
     CHECK(json::from_bson(input, true, false).is_discarded());
+    // false positive: Infer takes std::integral_constant<bool, false>::value for a pointer
+    // @infer-ignore NULLPTR_DEREFERENCE
     CHECK(json::from_bson(input.begin(), input.end(), true, false).is_discarded());
 #ifndef JSON_TEST_DEPRECATED_FUNCTIONS_DELETED
     CHECK(json::from_bson(input.data(), input.size(), true, false).is_discarded());

tests/src/unit-contiguous_parser.cpp

@@ -1,564 +0,0 @@
-//     __ _____ _____ _____
-//  __|  |   __|     |   | |  JSON for Modern C++ (supporting code)
-// |  |  |__   |  |  | | | |  version 3.12.0
-// |_____|_____|_____|_|___|  https://github.com/nlohmann/json
-//
-// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
-// SPDX-License-Identifier: MIT
-
-// parse() and accept() read text in one block of memory with
-// detail::contiguous_parser, and everything else (here: std::list iterators)
-// with the lexer's byte path. These tests check that the two agree on every
-// input: the same value, with the same number types and float bits, or the
-// same exception with the same message.
-
-#include "doctest_compatibility.h"
-
-#include <nlohmann/json.hpp>
-using nlohmann::json;
-using nlohmann::ordered_json;
-
-#include <cstddef>
-#include <cstdint>
-#include <cstring>
-#include <list>
-#include <map>
-#include <random>
-#include <string>
-#include <type_traits>
-#include <vector>
-
-namespace
-{
-
-/// the value with its number types and float bits; below 100 levels, the
-/// library's dump() (whose recursion is bounded) describes the rest
-template<typename Json>
-std::string describe_value(const Json& j, int depth = 0)
-{
-    std::string out;
-    if (depth >= 100)
-    {
-        out = j.dump();
-        return out;
-    }
-    switch (j.type())
-    {
-        case Json::value_t::number_float:
-        {
-            const double d = j.template get<double>();
-            std::uint64_t bits = 0;
-            std::memcpy(&bits, &d, sizeof(bits));
-            out += "f" + std::to_string(bits);
-            break;
-        }
-        case Json::value_t::number_integer:
-            out += "i" + j.dump();
-            break;
-        case Json::value_t::number_unsigned:
-            out += "u" + j.dump();
-            break;
-        case Json::value_t::array:
-            out += "[";
-            for (const auto& e : j)
-            {
-                out += describe_value(e, depth + 1) + ",";
-            }
-            out += "]";
-            break;
-        case Json::value_t::object:
-            out += "{";
-            for (auto it = j.begin(); it != j.end(); ++it)
-            {
-                out += Json(it.key()).dump() + ":" + describe_value(it.value(), depth + 1) + ",";
-            }
-            out += "}";
-            break;
-        case Json::value_t::null:
-        case Json::value_t::boolean:
-        case Json::value_t::string:
-        case Json::value_t::binary:
-        case Json::value_t::discarded:
-        default:
-            out += j.dump();
-            break;
-    }
-    return out;
-}
-
-#if !defined(JSON_NOEXCEPTION)
-template<typename Json, typename Parse>
-std::string outcome(Parse parse)
-{
-    try
-    {
-        return describe_value(parse());
-    }
-    catch (const typename Json::exception& e)
-    {
-        return std::string("exception: ") + e.what();
-    }
-}
-#endif
-
-/// every SAX event with its values, the text of float tokens, and the parse
-/// error with its position, token and message; returns false after
-/// @a stop_after events (to stop the parse)
-class event_log : public nlohmann::json_sax<json>
-{
-  public:
-    explicit event_log(std::size_t stop_after_ = static_cast<std::size_t>(-1), bool continue_after_error_ = false)
-        : stop_after(stop_after_), continue_after_error(continue_after_error_)
-    {}
-
-    bool null() override
-    {
-        return add("null");
-    }
-    bool boolean(bool val) override
-    {
-        return add(val ? "true" : "false");
-    }
-    bool number_integer(number_integer_t val) override
-    {
-        return add("i" + std::to_string(val));
-    }
-    bool number_unsigned(number_unsigned_t val) override
-    {
-        return add("u" + std::to_string(val));
-    }
-    bool number_float(number_float_t val, const string_t& s) override
-    {
-        std::uint64_t bits = 0;
-        std::memcpy(&bits, &val, sizeof(bits));
-        return add("f" + std::to_string(bits) + " " + s);
-    }
-    bool string(string_t& val) override
-    {
-        return add("s" + val);
-    }
-    bool binary(binary_t& /*val*/) override
-    {
-        return add("binary");
-    }
-    bool start_object(std::size_t elements) override
-    {
-        return add("{" + std::to_string(elements));
-    }
-    bool key(string_t& val) override
-    {
-        return add("k" + val);
-    }
-    bool end_object() override
-    {
-        return add("}");
-    }
-    bool start_array(std::size_t elements) override
-    {
-        return add("[" + std::to_string(elements));
-    }
-    bool end_array() override
-    {
-        return add("]");
-    }
-    bool parse_error(std::size_t position, const std::string& last_token, const nlohmann::detail::exception& ex) override
-    {
-        events.push_back("error " + std::to_string(position) + " " + last_token + " " + ex.what());
-        return continue_after_error;
-    }
-
-    const std::vector<std::string>& log() const noexcept
-    {
-        return events;
-    }
-
-  private:
-    bool add(const std::string& event)
-    {
-        events.push_back(event);
-        return events.size() != stop_after;
-    }
-
-    std::vector<std::string> events {}; // NOLINT(readability-redundant-member-init)
-    std::size_t stop_after;
-    bool continue_after_error;
-};
-
-/// sax_parse() of the contiguous path against the byte path, also with a
-/// handler that stops the parse
-void check_same_events(const std::string& text, const std::list<char>& bytes)
-{
-    for (const std::size_t stop_after :
-            {
-                static_cast<std::size_t>(-1), std::size_t{1}, std::size_t{3}
-            })
-    {
-        CAPTURE(stop_after)
-        event_log contiguous(stop_after);
-        event_log byte_path(stop_after);
-        const bool r1 = json::sax_parse(text, &contiguous);
-        const bool r2 = json::sax_parse(bytes.begin(), bytes.end(), &byte_path);
-        CHECK(r1 == r2);
-        CHECK(contiguous.log() == byte_path.log());
-    }
-    // a handler that asks to go on after an error (the default parser then
-    // reports what follows, e.g., the missing end of the input)
-    event_log contiguous(static_cast<std::size_t>(-1), true);
-    event_log byte_path(static_cast<std::size_t>(-1), true);
-    const bool r1 = json::sax_parse(text, &contiguous);
-    const bool r2 = json::sax_parse(bytes.begin(), bytes.end(), &byte_path);
-    CHECK(r1 == r2);
-    CHECK(contiguous.log() == byte_path.log());
-}
-
-/// the contiguous path (a std::string) against the byte path (std::list iterators)
-template<typename Json>
-void check_same(const std::string& text)
-{
-    CAPTURE(text)
-    const std::list<char> bytes(text.begin(), text.end());
-
-#if !defined(JSON_NOEXCEPTION)
-    // with exceptions: the value, or the same exception for both
-    const std::string contiguous = outcome<Json>([&]
-    {
-        return Json::parse(text);
-    });
-    const std::string byte_path = outcome<Json>([&]
-    {
-        return Json::parse(bytes.begin(), bytes.end());
-    });
-    CHECK(contiguous == byte_path);
-#endif
-
-    // without exceptions: the value, or discarded for both
-    const Json c = Json::parse(text, nullptr, false);
-    const Json b = Json::parse(bytes.begin(), bytes.end(), nullptr, false);
-    CHECK(c.is_discarded() == b.is_discarded());
-
-    CHECK(Json::accept(text) == Json::accept(bytes.begin(), bytes.end()));
-    if (std::is_same<Json, json>::value)
-    {
-        check_same_events(text, bytes);
-    }
-    // a pointer range without a terminating NUL
-    std::vector<char> exact(text.begin(), text.end());
-    if (!exact.empty())
-    {
-        CHECK(Json::accept(exact.data(), exact.data() + exact.size()) == Json::accept(bytes.begin(), bytes.end()));
-    }
-}
-
-void check_both(const std::string& text)
-{
-    check_same<json>(text);
-    check_same<ordered_json>(text);
-}
-
-/// a random JSON text, mostly valid, with the corner cases of the parser
-class generator
-{
-  public:
-    explicit generator(std::uint32_t seed) : rng(seed) {}
-
-    std::string document(int depth)
-    {
-        std::string out;
-        value(out, depth);
-        return out;
-    }
-
-  private:
-    std::uint32_t pick(std::uint32_t n)
-    {
-        return static_cast<std::uint32_t>(rng() % n);
-    }
-
-    void whitespace(std::string& out)
-    {
-        static const char* const choices[] = {"", "", "", " ", "\n", "\n        ", "\t", "\r\n  ", "\n                  ", "  "}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
-        out += choices[pick(10)];
-    }
-
-    void number(std::string& out)
-    {
-        static const char* const choices[] = // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
-        {
-            "0", "-0", "1", "-1", "42", "123456789012345678", "-123456789012345678", "1234567890123456789",
-            "-1234567890123456789", "9223372036854775807", "-9223372036854775808", "-9223372036854775809",
-            "18446744073709551615", "18446744073709551616", "123456789012345678901234567890", "0.0", "-0.0",
-            "1.5", "-65.613616999999977", "1e10", "1E+10", "1e-10", "2.2250738585072014e-308", "4.9e-324",
-            "1e-400", "1.7976931348623157e308", "1e308", "1e309", "-1e309", "123.456e-789", "0.1e1",
-            "12345678901234567890.5", "1.00000000000000011102230246251565404236316680908203125",
-            "9007199254740993", "0.30000000000000004", "1e22", "1e23", "100000000000000000000000",
-            "00", "01", "-", "1.", ".5", "1e", "1e+", "--1", "+1", "1.5e3.2", "0x10", "1_000"
-        };
-        out += choices[pick(sizeof(choices) / sizeof(choices[0]))];
-    }
-
-    void string(std::string& out)
-    {
-        static const char* const pieces[] = // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
-        {
-            "a", "key", "a longer piece of plain ASCII text for the vector paths", "\\n", "\\t", "\\\"",
-            "\\\\", "\\/", "\\b", "\\f", "\\r", "\\u0000", "\\u00e4", "\\u20AC", "\\uD83D\\uDE00",
-            "\\uD83D", "\\uDE00", "\\uD83Dx", "\\u12", "\\x", "\xC3\xA4", "\xE2\x82\xAC", "\xF0\x9F\x98\x80",
-            "\xC0\xAF", "\xED\xA0\x80", "\xF4\x90\x80\x80", "\x80", "\xFF", "\x01", "\x1F", "\x7F", "\t"
-        };
-        out += '"';
-        const auto n = pick(6);
-        for (std::uint32_t i = 0; i < n; ++i)
-        {
-            // mostly well-formed pieces
-            out += pieces[pick(5) == 0 ? pick(sizeof(pieces) / sizeof(pieces[0])) : pick(20)];
-        }
-        out += '"';
-    }
-
-    void value(std::string& out, int depth)
-    {
-        const auto kind = depth > 0 ? pick(9) : 3 + pick(6);
-        switch (kind)
-        {
-            case 0:
-            case 1:
-            {
-                out += '[';
-                whitespace(out);
-                const auto n = pick(6);
-                for (std::uint32_t i = 0; i < n; ++i)
-                {
-                    if (i != 0)
-                    {
-                        out += ',';
-                        whitespace(out);
-                    }
-                    value(out, depth - 1);
-                    whitespace(out);
-                }
-                out += ']';
-                break;
-            }
-            case 2:
-            {
-                out += '{';
-                whitespace(out);
-                const auto n = pick(6);
-                for (std::uint32_t i = 0; i < n; ++i)
-                {
-                    if (i != 0)
-                    {
-                        out += ',';
-                        whitespace(out);
-                    }
-                    // a few duplicate keys
-                    if (pick(4) == 0)
-                    {
-                        out += "\"dup\"";
-                    }
-                    else
-                    {
-                        string(out);
-                    }
-                    whitespace(out);
-                    out += pick(2) == 0 ? ":" : ": ";
-                    whitespace(out);
-                    value(out, depth - 1);
-                    whitespace(out);
-                }
-                out += '}';
-                break;
-            }
-            case 3:
-            case 4:
-                number(out);
-                break;
-            case 5:
-            case 6:
-                string(out);
-                break;
-            case 7:
-                out += pick(2) == 0 ? "true" : "false";
-                break;
-            default:
-                out += "null";
-                break;
-        }
-    }
-
-    std::mt19937 rng;
-};
-
-} // namespace
-
-TEST_CASE("contiguous parser")
-{
-    SECTION("fixed inputs")
-    {
-        const char* const inputs[] = // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
-        {
-            "", " ", "\n", "null", "true", "false", "nul", "tru", "fals", "nullx", "[]", "{}", "[", "{", "]", "}",
-            "[1,]", "[,1]", R"({"a":1,})", R"({"a" 1})", R"({"a":})", "{1:2}", "[1 2]", R"("")", R"("abc)", R"("\")",
-            "\xEF\xBB\xBF[1]", "\xEF\xBB[1]", "\xEF[1]", "\xEF\xBB\xBF", " [1] ", "[1]x", "[1] [2]", "1 2",
-            R"({"a":1,"a":2})", R"({"b":1,"a":2,"b":3})", R"(["\u0000"])", "[1e400]", "[-1e400]",
-            R"({"a": [1, 2, {"b": null}], "c": "d"})", "[true,false,null]", "[[[[[]]]]]",
-        };
-        for (const char* input : inputs)
-        {
-            check_both(input);
-        }
-
-        // NUL bytes: by default the end of the input (between values), else an error
-        check_both(std::string("[1]\0garbage", 11));
-        check_both(std::string("[1,\0]", 5));
-        check_both(std::string("\"a\0b\"", 5));
-        check_both(std::string("\0", 1));
-        check_both(std::string("1\0", 2));
-    }
-
-    SECTION("deep nesting")
-    {
-        // across the 64-level words of the nesting stack
-        const std::vector<std::size_t> depths = {63, 64, 65, 127, 128, 129, 200, 1000, 10000};
-        for (const std::size_t depth : depths)
-        {
-            check_both(std::string(depth, '[') + std::string(depth, ']'));
-            std::string objects;
-            for (std::size_t i = 0; i < depth; ++i)
-            {
-                objects += (i % 2 == 0) ? "{\"k\":" : "[";
-            }
-            objects += "1";
-            for (std::size_t i = depth; i-- > 0;)
-            {
-                objects += (i % 2 == 0) ? "}" : "]";
-            }
-            check_both(objects);
-            // unbalanced
-            check_both(std::string(depth, '[') + std::string(depth - 1, ']'));
-            check_both(std::string(depth, '[') + std::string(depth + 1, ']'));
-        }
-        // a value after a container that closes a 64-level word
-        std::string mixed = std::string(64, '[') + "1" + std::string(63, ']') + ",{\"a\":2}]";
-        check_both(mixed);
-        mixed = "{\"x\":" + std::string(64, '[') + std::string(64, ']') + ",\"y\":[1]}";
-        check_both(mixed);
-    }
-
-    SECTION("generated documents and their mutations")
-    {
-        generator gen(20261105u);
-        std::mt19937 rng(42u); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
-        const char alphabet[] = "{}[]:,\"\\ \t\n\r0123456789-+.eEtrufalsn/u\x80\xbf\xc3\xe2\xed\xf0\xff"; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
-        for (int round = 0; round < 1500; ++round)
-        {
-            const std::string text = gen.document(1 + (round % 5));
-            check_both(text);
-            // every truncation of short documents, and a few mutations
-            if (text.size() < 60)
-            {
-                for (std::size_t n = 0; n < text.size(); ++n)
-                {
-                    check_same<json>(text.substr(0, n));
-                }
-            }
-            for (int m = 0; m < 3 && !text.empty(); ++m)
-            {
-                std::string mutated = text;
-                const std::size_t pos = rng() % mutated.size();
-                switch (rng() % 3)
-                {
-                    case 0:
-                        mutated[pos] = alphabet[rng() % (sizeof(alphabet) - 1)];
-                        break;
-                    case 1:
-                        mutated.insert(pos, 1, alphabet[rng() % (sizeof(alphabet) - 1)]);
-                        break;
-                    default:
-                        mutated.erase(pos, 1);
-                        break;
-                }
-                check_same<json>(mutated);
-            }
-        }
-    }
-
-    SECTION("narrow number types")
-    {
-        // integers that do not fit become floats, floats are converted to float
-        using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
-        const std::vector<std::string> inputs =
-        {
-            "[2147483647, 2147483648, -2147483648, -2147483649, 4294967295, 4294967296, 18446744073709551615]",
-            "[1.5, 3.4028234663852886e38, 3.4028235677973366e38, 1e39, -1e39, 1e-46, 0.1, 16777217]",
-            "[-0, 0.0, 123456789012345678901234567890, -9223372036854775809]", "1e39", "4294967296", "-2147483649"
-        };
-        for (const auto& input : inputs)
-        {
-            check_same<narrow_json>(input);
-        }
-        generator gen(17u);
-        for (int round = 0; round < 200; ++round)
-        {
-            check_same<narrow_json>(gen.document(1 + (round % 4)));
-        }
-    }
-
-    SECTION("number corner cases at the end of the input")
-    {
-        // the number ends at the end of the block (no closing bracket after it)
-        for (const char* number :
-                {"0", "-0", "1", "12345678", "123456789", "1234567890123456789", "18446744073709551616",
-                 "1.5", "1e5", "1e", "1.", "-", "1.5e-3", "123456789012345678901234567890e-10"
-                })
-        {
-            check_both(number);
-            check_both(std::string("[") + number + "]");
-            check_both(std::string("{\"n\":") + number + "}");
-        }
-    }
-
-    SECTION("accept() of floats near the largest value")
-    {
-        // accept() skips digits without their value and computes it only for
-        // a float whose exponent may reach the largest value
-        for (const char* number :
-                {"17976931348623158e292", "17976931348623159e292", "1797693134862315.8e293", "1797693134862315.9e293",
-                 "0.17976931348623158e309", "0.17976931348623159e309", "-17976931348623159e292", "9999999999999999999e289"
-                })
-        {
-            check_both(number);
-            check_both(std::string("[") + number + "]");
-        }
-        CHECK(json::accept("17976931348623158e292"));
-        CHECK_FALSE(json::accept("17976931348623159e292"));
-        CHECK(json::accept("[1797693134862315.8e293]"));
-        CHECK_FALSE(json::accept("[1797693134862315.9e293]"));
-        CHECK(json::accept("0.17976931348623158e309"));
-        CHECK_FALSE(json::accept("0.17976931348623159e309"));
-        CHECK_FALSE(json::accept("-17976931348623159e292"));
-
-        using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
-        check_same<narrow_json>("34028235e31");
-        check_same<narrow_json>("34028236e31");
-        CHECK(narrow_json::accept("34028235e31"));
-        CHECK_FALSE(narrow_json::accept("34028236e31"));
-    }
-
-    SECTION("callbacks, comments and trailing commas take the default parser")
-    {
-        // (the results are those of the options; the contiguous parser does not support them)
-        CHECK(json::parse("[1,]", nullptr, true, false, true) == json({1}));
-        CHECK(json::parse("[1/*c*/]", nullptr, true, true) == json({1}));
-        CHECK(json::accept("[1,]", false, true));
-        CHECK(json::accept("// c\n[1]", true));
-        int calls = 0;
-        const json j = json::parse("[1,2]", [&calls](int /*depth*/, json::parse_event_t /*event*/, json& /*parsed*/) noexcept
-        {
-            ++calls;
-            return true;
-        });
-        CHECK(j == json({1, 2}));
-        CHECK(calls == 4);
-    }
-}

tests/src/unit-deserialization.cpp

@@ -32,6 +32,7 @@ using nlohmann::json;
 
 #include <iostream>
 #include <iterator>
+#include <list>
 #include <sstream>
 #include <string>
 #include <valarray>
@@ -1409,3 +1410,77 @@ TEST_CASE("indentation runs of every length around the 8-byte skip")
     CHECK(parse_result(indented("[\n~1]", std::string(17, ' ')), true) == "[1]");
 }
 #endif
+
+#if !defined(JSON_NOEXCEPTION)
+namespace
+{
+/// the value of parsing [first, last), or the message of the exception
+template<typename Iterator>
+std::string outcome_of(Iterator first, Iterator last)
+{
+    try
+    {
+        return json::parse(first, last).dump();
+    }
+    catch (const json::exception& e)
+    {
+        return e.what();
+    }
+}
+
+/// the bytes of @a text as a vector of @a T (explicit casts: no implicit char conversion)
+template<typename T>
+std::vector<T> bytes_as(const std::string& text)
+{
+    std::vector<T> result;
+    result.reserve(text.size());
+    for (const char c : text)
+    {
+        result.push_back(static_cast<T>(c));
+    }
+    return result;
+}
+
+/// a std::string, std::vector<char>, <unsigned char> and <signed char> as
+/// iterator pairs: the same as the byte path (std::list iterators)
+void check_iterator_pairs(const std::string& text)
+{
+    CAPTURE(text)
+    const std::list<char> bytes(text.begin(), text.end());
+    const std::string expected = outcome_of(bytes.begin(), bytes.end());
+    const bool expected_accept = json::accept(bytes.begin(), bytes.end());
+
+    std::string s = text;
+    CHECK(outcome_of(s.begin(), s.end()) == expected);
+    CHECK(outcome_of(s.cbegin(), s.cend()) == expected);
+    CHECK(json::accept(s.cbegin(), s.cend()) == expected_accept);
+
+    const std::vector<char> vc(text.begin(), text.end());
+    CHECK(outcome_of(vc.begin(), vc.end()) == expected);
+    const std::vector<unsigned char> vu = bytes_as<unsigned char>(text);
+    CHECK(outcome_of(vu.begin(), vu.end()) == expected);
+    CHECK(json::accept(vu.begin(), vu.end()) == expected_accept);
+    const std::vector<signed char> vs = bytes_as<signed char>(text);
+    CHECK(outcome_of(vs.begin(), vs.end()) == expected);
+}
+}  // namespace
+
+TEST_CASE("deserialization from iterator pairs of std::string and std::vector")
+{
+    // these are read as the pointer range they are, also before C++20
+    for (const char* text :
+            {
+                "", " ", R"([1, 2.5, "a", true, null, {"k": ["v"]}])", "[1, 2", R"({"a" 1})", "\"\xC3\xA4\"", "\"\xC3\"", "[1]x", "1e400"
+            })
+    {
+        check_iterator_pairs(text);
+    }
+    check_iterator_pairs(std::string("[1]" "\0" "x", 5));
+
+    // an empty range is not dereferenced
+    std::string empty;
+    CHECK_FALSE(json::accept(empty.begin(), empty.end()));
+    json _;
+    CHECK_THROWS_AS(_ = json::parse(empty.begin(), empty.end()), json::parse_error&);
+}
+#endif