json_view: skip the digits of a fraction 16 bytes at a time (SSE2)

The digits after the decimal point were skipped by skip_digits(), 16 compare-and-branch
steps in a row: about 3.5 instructions per digit, and an exit branch that is
mispredicted when the lengths of the fractions vary. On x86 with SSE2, the
fraction is now skipped by skip_fraction_digits(): 16 bytes are compared at
once and the end of the run is taken from the bit mask, which is about a third
of the instructions for a fraction of 12 digits. The last bytes of the input
are skipped one at a time as before. Where there is no SSE2 (AArch64, other
targets, JSON_NO_SIMD) skip_fraction_digits() is skip_digits(): nothing changes
there. Integer digits keep skip_digits(), whose exit is predictable.

GCC unrolls the loop over the last bytes in every inlined copy of the new
function, which made the parse loop 4 KiB larger and parse() of documents with
hardly any fractions about 4 % slower. For GCC that loop is therefore out of
line (skip_digits_tail()); Clang is faster with it inline.

json_document::parse(), A/B in one binary, 6 code layouts, x86-64 Haswell,
time after / before (GCC 16.2 / Clang 23):

- numbers.json (10,001 floats): 0.715 / 0.680
- canada.json: 0.778 / 0.790
- mesh.json: 0.908 / 0.883
- marine_ik.json: 0.937 / 0.926
- mesh.pretty.json: 0.923 / 0.960

Files with few or no fractions and files with fractions of a fixed length
(20,000 prices with two decimals, 60,000 coordinates with six): between 0.98 and
1.014 on both compilers, except citm_catalog.json with GCC (1.05: the
instruction count is +0.3 %, and that row varies by +-7 % between builds of
unrelated variants on this machine).

New test: fractions of every length from 0 to 40 digits followed by every kind of
byte that ends a number or lies next to the digits in ASCII, in several
positions, and cut in every place near the end of the text, against json::parse()
(the length of the number is part of what is compared), for all option
combinations and with and without a terminating NUL. Checked on x86-64 with GCC 16.2,
Clang 23 and Apple clang under Rosetta, and with a deliberate off-by-one at the 16th byte.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

include/nlohmann/detail/input/binary_reader.hpp

@@ -741,13 +741,11 @@ class binary_reader
     bool get_cbor_negative_integer()
     {
         NumberType number{};
-        return get_number(number) && emit_cbor_negative_integer(number);
-    }
+        if (JSON_HEDLEY_UNLIKELY(!get_number(number)))
+        {
+            return false;
+        }
 
-    /// pass the CBOR negative integer -1 - @a number to the SAX parser
-    template<typename NumberType>
-    bool emit_cbor_negative_integer(const NumberType number)
-    {
         // the value is -1 - number, which fits into number_integer_t
         // whenever number does; the outer cast undoes the integral promotion
         // for number_integer_t types narrower than int
@@ -972,7 +970,7 @@ class binary_reader
             case 0x95:
             case 0x96:
             case 0x97:
-                return enter_cbor_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
+                return enter_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
 
             case 0x98: // array (one-byte uint8_t for n follows)
             case 0x99: // array (two-byte uint16_t for n follow)
@@ -981,7 +979,7 @@ class binary_reader
             {
                 std::uint64_t len{};
                 std::size_t size{};
-                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_cbor_array(size);
+                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_array(size);
             }
 
             case 0x9F: // array (indefinite length)
@@ -1678,157 +1676,6 @@ class binary_reader
         return true;
     }
 
-    /*!
-    @brief read a big-endian number of type @a NumberType at @a data
-    */
-    template<typename NumberType>
-    NumberType load_big_endian(const unsigned char* data) const noexcept
-    {
-        NumberType result{};
-        std::memcpy(&result, data, sizeof(NumberType));
-        if (is_little_endian)
-        {
-            byte_swap(result);
-        }
-        return result;
-    }
-
-    /*!
-    @brief read the integers, floats, booleans, and nulls that follow in a
-           definite-length array, without the per-element bookkeeping of
-           @ref parse_cbor_internal
-
-    Stops at the end of the array, at any other kind of value, and when fewer
-    than 9 bytes (the longest of these values) are left in the input, leaving
-    those to @ref parse_cbor_value, which reads them the same way.
-
-    @param[in,out] remaining  the number of elements of the array not yet read
-    @return whether the SAX parser accepted the values
-    */
-    bool parse_cbor_scalar_run(std::size_t& remaining)
-    {
-        while (remaining != 0 && ia.bulk_remaining() >= 9)
-        {
-            const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
-            const unsigned char head = data[0];
-            std::size_t size = 0;
-            bool accepted = true;
-            switch (head)
-            {
-                case 0x18:
-                    size = 2;
-                    accepted = emit_unsigned(data[1]);
-                    break;
-                case 0x19:
-                    size = 3;
-                    accepted = emit_unsigned(load_big_endian<std::uint16_t>(data + 1));
-                    break;
-                case 0x1A:
-                    size = 5;
-                    accepted = emit_unsigned(load_big_endian<std::uint32_t>(data + 1));
-                    break;
-                case 0x1B:
-                    size = 9;
-                    accepted = emit_unsigned(load_big_endian<std::uint64_t>(data + 1));
-                    break;
-                case 0x38:
-                    size = 2;
-                    accepted = emit_cbor_negative_integer(data[1]);
-                    break;
-                case 0x39:
-                    size = 3;
-                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint16_t>(data + 1));
-                    break;
-                case 0x3A:
-                    size = 5;
-                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint32_t>(data + 1));
-                    break;
-                case 0x3B:
-                    size = 9;
-                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint64_t>(data + 1));
-                    break;
-                case 0xF4:
-                    size = 1;
-                    accepted = sax->boolean(false);
-                    break;
-                case 0xF5:
-                    size = 1;
-                    accepted = sax->boolean(true);
-                    break;
-                case 0xF6:
-                    size = 1;
-                    accepted = sax->null();
-                    break;
-                case 0xFA:
-                    size = 5;
-                    accepted = emit_float(load_big_endian<float>(data + 1));
-                    break;
-                case 0xFB:
-                    size = 9;
-                    accepted = emit_float(load_big_endian<double>(data + 1));
-                    break;
-                default:
-                    if (head <= 0x17)
-                    {
-                        size = 1;
-                        accepted = sax->number_unsigned(static_cast<number_unsigned_t>(head));
-                    }
-                    else if (head >= 0x20 && head <= 0x37)
-                    {
-                        size = 1;
-                        accepted = sax->number_integer(static_cast<std::int8_t>(0x20 - 1 - head));
-                    }
-                    else
-                    {
-                        // any other value is read by parse_cbor_value
-                        return true;
-                    }
-                    break;
-            }
-            // as if read by get() and get_number(): current is the head byte
-            ia.bulk_skip(size);
-            chars_read += size;
-            current = head;
-            --remaining;
-            if (JSON_HEDLEY_UNLIKELY(!accepted))
-            {
-                return false;
-            }
-        }
-        return true;
-    }
-
-    /*!
-    @brief open a definite-length array and read the scalars it begins with
-           (see @ref parse_cbor_scalar_run)
-
-    Only the start of an array is tried, which covers arrays of numbers
-    (e.g. coordinates) but costs arrays of other values only one test, not one
-    per element.
-
-    @param[in] len  the number of elements
-    @return whether the SAX parser accepted the array and the values read
-    */
-    bool enter_cbor_array(const std::size_t len)
-    {
-        return enter_array(len) && (len == 0 || parse_cbor_array_start(std::integral_constant<bool, bulk_scan> {}));
-    }
-
-    /// read the scalars the array just opened begins with; out of line, so
-    /// that @ref parse_cbor_value stays small enough to be inlined into
-    /// @ref parse_cbor_internal
-    JSON_HEDLEY_NEVER_INLINE
-    bool parse_cbor_array_start(std::true_type /*bulk*/)
-    {
-        return parse_cbor_scalar_run(container_stack.back().remaining);
-    }
-
-    /// input that is not contiguous: every value is read by parse_cbor_value
-    bool parse_cbor_array_start(std::false_type /*bulk*/) const noexcept
-    {
-        return true;
-    }
-
     /*!
     @brief read a CBOR value and everything nested inside it
 
@@ -1858,7 +1705,6 @@ class binary_reader
                 // pop_back() below, which destroys the container_stack element
                 // it would otherwise alias
                 const container_frame top = container_stack.back();
-                const bool is_object = top.is_object;
                 bool at_end = false;
 
                 if (top.remaining != npos)
@@ -1870,7 +1716,7 @@ class binary_reader
                     {
                         // claim the element about to be read
                         --container_stack.back().remaining;
-                        if (is_object)
+                        if (top.is_object)
                         {
                             get();
                         }
@@ -1883,7 +1729,7 @@ class binary_reader
                     // Testing for it consumes a byte, which is the first byte
                     // of the next element when it is not one.
                     at_end = (get() == 0xFF);
-                    fetch = is_object;
+                    fetch = top.is_object;
                 }
 
                 if (at_end)
@@ -1900,7 +1746,7 @@ class binary_reader
                     continue;
                 }
 
-                if (is_object)
+                if (top.is_object)
                 {
                     if (JSON_HEDLEY_UNLIKELY(!skip_cbor_key_tags(tag_handler)))
                     {
@@ -3845,13 +3691,13 @@ class binary_reader
                 return sax->null();
 
             case 0xFB:
-                return sax->number_float(static_cast<number_float_t>(-1.0), no_source);
+                return sax->number_float(static_cast<number_float_t>(-1.0), "");
 
             case 0xFC:
-                return sax->number_float(static_cast<number_float_t>(0.0), no_source);
+                return sax->number_float(static_cast<number_float_t>(0.0), "");
 
             case 0xFD:
-                return sax->number_float(static_cast<number_float_t>(1.0), no_source);
+                return sax->number_float(static_cast<number_float_t>(1.0), "");
 
             case 0xFF: // empty string
             {
@@ -4355,7 +4201,7 @@ class binary_reader
         {
             return out_of_range_error(exception_id::number_overflow, "number overflow", "value");
         }
-        return sax->number_float(result, no_source);
+        return sax->number_float(result, "");
     }
 
     /*!
@@ -4415,7 +4261,7 @@ class binary_reader
         }();
         return sax->number_float((half & 0x8000u) != 0
                                  ? static_cast<number_float_t>(-val)
-                                 : static_cast<number_float_t>(val), no_source);
+                                 : static_cast<number_float_t>(val), "");
     }
 
     /*!
@@ -4740,10 +4586,6 @@ class binary_reader
     /// the SAX parser
     json_sax_t* sax = nullptr;
 
-    /// the (empty) source text of every number_float event: binary formats
-    /// have none, and a string_t built for each float would cost more
-    const string_t no_source{};
-
     /// the containers that have been opened and not closed yet; see @ref container_frame
     std::vector<container_frame> container_stack{};
 

include/nlohmann/detail/view/builder.hpp

@@ -1082,7 +1082,7 @@ indent_done:
             {
                 ++p;
                 const unsigned char* const f0 = p;
-                p = skip_digits(p, e);
+                p = skip_fraction_digits(p, e);
                 if (NLOHMANN_VIEW_UNLIKELY(p == f0))
                 {
                     return fail(error_code::number_after_dot);

include/nlohmann/detail/view/scan.hpp

@@ -198,6 +198,57 @@ inline const unsigned char* find_block_comment_end(const unsigned char* p, const
     return p;
 }
 
+/// The digits of a fraction, whose number varies from one number to the next: skip_digits() with 16 bytes compared at
+/// once and the end of the run taken from a bit mask (SSE2), instead of a compare and a branch for each digit. That
+/// is about a third of the instructions for a fraction of 12 digits, and no branch that is mispredicted when the
+/// lengths vary. Where there is no SSE2, skip_digits() is used.
+#if NLOHMANN_VIEW_SSE2
+// The last bytes of the input, one at a time. GCC unrolls this loop in every inlined copy of the function below, which
+// makes the parse loop 4 KiB larger and slows down documents that have hardly any fractions; out of line, it does
+// not. Clang does better with the loop inline (a call in the parse loop costs it more than the unrolling).
+#if defined(__GNUC__) && !defined(__clang__)
+inline NLOHMANN_VIEW_NOINLINE const unsigned char* skip_digits_tail(const unsigned char* p, const unsigned char* e) noexcept
+{
+    while (p != e && is_digit(*p))
+    {
+        ++p;
+    }
+    return p;
+}
+#else
+NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* skip_digits_tail(const unsigned char* p, const unsigned char* e) noexcept
+{
+    while (p != e && is_digit(*p))
+    {
+        ++p;
+    }
+    return p;
+}
+#endif
+
+NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* skip_fraction_digits(const unsigned char* p, const unsigned char* e) noexcept
+{
+    while (e - p >= 16)
+    {
+        const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
+        // a digit is above '/' and below ':' (signed compares: a byte with the top bit set is negative, and neither)
+        const __m128i digits = _mm_and_si128(_mm_cmpgt_epi8(in, _mm_set1_epi8('/')), _mm_cmplt_epi8(in, _mm_set1_epi8(':')));
+        const auto not_digit = static_cast<std::uint64_t>(~static_cast<unsigned>(_mm_movemask_epi8(digits)) & 0xFFFFu);
+        if (not_digit != 0)
+        {
+            return p + count_trailing_zeros(not_digit);
+        }
+        p += 16;
+    }
+    return skip_digits_tail(p, e);
+}
+#else
+NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* skip_fraction_digits(const unsigned char* p, const unsigned char* e) noexcept
+{
+    return skip_digits(p, e);
+}
+#endif
+
 /// powers of ten up to 10^19 as integers
 inline std::uint64_t int_pow10(unsigned k) noexcept
 {

single_include/nlohmann/json.hpp

@@ -18517,13 +18517,11 @@ class binary_reader
     bool get_cbor_negative_integer()
     {
         NumberType number{};
-        return get_number(number) && emit_cbor_negative_integer(number);
-    }
+        if (JSON_HEDLEY_UNLIKELY(!get_number(number)))
+        {
+            return false;
+        }
 
-    /// pass the CBOR negative integer -1 - @a number to the SAX parser
-    template<typename NumberType>
-    bool emit_cbor_negative_integer(const NumberType number)
-    {
         // the value is -1 - number, which fits into number_integer_t
         // whenever number does; the outer cast undoes the integral promotion
         // for number_integer_t types narrower than int
@@ -18748,7 +18746,7 @@ class binary_reader
             case 0x95:
             case 0x96:
             case 0x97:
-                return enter_cbor_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
+                return enter_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
 
             case 0x98: // array (one-byte uint8_t for n follows)
             case 0x99: // array (two-byte uint16_t for n follow)
@@ -18757,7 +18755,7 @@ class binary_reader
             {
                 std::uint64_t len{};
                 std::size_t size{};
-                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_cbor_array(size);
+                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_array(size);
             }
 
             case 0x9F: // array (indefinite length)
@@ -19454,157 +19452,6 @@ class binary_reader
         return true;
     }
 
-    /*!
-    @brief read a big-endian number of type @a NumberType at @a data
-    */
-    template<typename NumberType>
-    NumberType load_big_endian(const unsigned char* data) const noexcept
-    {
-        NumberType result{};
-        std::memcpy(&result, data, sizeof(NumberType));
-        if (is_little_endian)
-        {
-            byte_swap(result);
-        }
-        return result;
-    }
-
-    /*!
-    @brief read the integers, floats, booleans, and nulls that follow in a
-           definite-length array, without the per-element bookkeeping of
-           @ref parse_cbor_internal
-
-    Stops at the end of the array, at any other kind of value, and when fewer
-    than 9 bytes (the longest of these values) are left in the input, leaving
-    those to @ref parse_cbor_value, which reads them the same way.
-
-    @param[in,out] remaining  the number of elements of the array not yet read
-    @return whether the SAX parser accepted the values
-    */
-    bool parse_cbor_scalar_run(std::size_t& remaining)
-    {
-        while (remaining != 0 && ia.bulk_remaining() >= 9)
-        {
-            const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
-            const unsigned char head = data[0];
-            std::size_t size = 0;
-            bool accepted = true;
-            switch (head)
-            {
-                case 0x18:
-                    size = 2;
-                    accepted = emit_unsigned(data[1]);
-                    break;
-                case 0x19:
-                    size = 3;
-                    accepted = emit_unsigned(load_big_endian<std::uint16_t>(data + 1));
-                    break;
-                case 0x1A:
-                    size = 5;
-                    accepted = emit_unsigned(load_big_endian<std::uint32_t>(data + 1));
-                    break;
-                case 0x1B:
-                    size = 9;
-                    accepted = emit_unsigned(load_big_endian<std::uint64_t>(data + 1));
-                    break;
-                case 0x38:
-                    size = 2;
-                    accepted = emit_cbor_negative_integer(data[1]);
-                    break;
-                case 0x39:
-                    size = 3;
-                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint16_t>(data + 1));
-                    break;
-                case 0x3A:
-                    size = 5;
-                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint32_t>(data + 1));
-                    break;
-                case 0x3B:
-                    size = 9;
-                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint64_t>(data + 1));
-                    break;
-                case 0xF4:
-                    size = 1;
-                    accepted = sax->boolean(false);
-                    break;
-                case 0xF5:
-                    size = 1;
-                    accepted = sax->boolean(true);
-                    break;
-                case 0xF6:
-                    size = 1;
-                    accepted = sax->null();
-                    break;
-                case 0xFA:
-                    size = 5;
-                    accepted = emit_float(load_big_endian<float>(data + 1));
-                    break;
-                case 0xFB:
-                    size = 9;
-                    accepted = emit_float(load_big_endian<double>(data + 1));
-                    break;
-                default:
-                    if (head <= 0x17)
-                    {
-                        size = 1;
-                        accepted = sax->number_unsigned(static_cast<number_unsigned_t>(head));
-                    }
-                    else if (head >= 0x20 && head <= 0x37)
-                    {
-                        size = 1;
-                        accepted = sax->number_integer(static_cast<std::int8_t>(0x20 - 1 - head));
-                    }
-                    else
-                    {
-                        // any other value is read by parse_cbor_value
-                        return true;
-                    }
-                    break;
-            }
-            // as if read by get() and get_number(): current is the head byte
-            ia.bulk_skip(size);
-            chars_read += size;
-            current = head;
-            --remaining;
-            if (JSON_HEDLEY_UNLIKELY(!accepted))
-            {
-                return false;
-            }
-        }
-        return true;
-    }
-
-    /*!
-    @brief open a definite-length array and read the scalars it begins with
-           (see @ref parse_cbor_scalar_run)
-
-    Only the start of an array is tried, which covers arrays of numbers
-    (e.g. coordinates) but costs arrays of other values only one test, not one
-    per element.
-
-    @param[in] len  the number of elements
-    @return whether the SAX parser accepted the array and the values read
-    */
-    bool enter_cbor_array(const std::size_t len)
-    {
-        return enter_array(len) && (len == 0 || parse_cbor_array_start(std::integral_constant<bool, bulk_scan> {}));
-    }
-
-    /// read the scalars the array just opened begins with; out of line, so
-    /// that @ref parse_cbor_value stays small enough to be inlined into
-    /// @ref parse_cbor_internal
-    JSON_HEDLEY_NEVER_INLINE
-    bool parse_cbor_array_start(std::true_type /*bulk*/)
-    {
-        return parse_cbor_scalar_run(container_stack.back().remaining);
-    }
-
-    /// input that is not contiguous: every value is read by parse_cbor_value
-    bool parse_cbor_array_start(std::false_type /*bulk*/) const noexcept
-    {
-        return true;
-    }
-
     /*!
     @brief read a CBOR value and everything nested inside it
 
@@ -19634,7 +19481,6 @@ class binary_reader
                 // pop_back() below, which destroys the container_stack element
                 // it would otherwise alias
                 const container_frame top = container_stack.back();
-                const bool is_object = top.is_object;
                 bool at_end = false;
 
                 if (top.remaining != npos)
@@ -19646,7 +19492,7 @@ class binary_reader
                     {
                         // claim the element about to be read
                         --container_stack.back().remaining;
-                        if (is_object)
+                        if (top.is_object)
                         {
                             get();
                         }
@@ -19659,7 +19505,7 @@ class binary_reader
                     // Testing for it consumes a byte, which is the first byte
                     // of the next element when it is not one.
                     at_end = (get() == 0xFF);
-                    fetch = is_object;
+                    fetch = top.is_object;
                 }
 
                 if (at_end)
@@ -19676,7 +19522,7 @@ class binary_reader
                     continue;
                 }
 
-                if (is_object)
+                if (top.is_object)
                 {
                     if (JSON_HEDLEY_UNLIKELY(!skip_cbor_key_tags(tag_handler)))
                     {
@@ -21621,13 +21467,13 @@ class binary_reader
                 return sax->null();
 
             case 0xFB:
-                return sax->number_float(static_cast<number_float_t>(-1.0), no_source);
+                return sax->number_float(static_cast<number_float_t>(-1.0), "");
 
             case 0xFC:
-                return sax->number_float(static_cast<number_float_t>(0.0), no_source);
+                return sax->number_float(static_cast<number_float_t>(0.0), "");
 
             case 0xFD:
-                return sax->number_float(static_cast<number_float_t>(1.0), no_source);
+                return sax->number_float(static_cast<number_float_t>(1.0), "");
 
             case 0xFF: // empty string
             {
@@ -22131,7 +21977,7 @@ class binary_reader
         {
             return out_of_range_error(exception_id::number_overflow, "number overflow", "value");
         }
-        return sax->number_float(result, no_source);
+        return sax->number_float(result, "");
     }
 
     /*!
@@ -22191,7 +22037,7 @@ class binary_reader
         }();
         return sax->number_float((half & 0x8000u) != 0
                                  ? static_cast<number_float_t>(-val)
-                                 : static_cast<number_float_t>(val), no_source);
+                                 : static_cast<number_float_t>(val), "");
     }
 
     /*!
@@ -22516,10 +22362,6 @@ class binary_reader
     /// the SAX parser
     json_sax_t* sax = nullptr;
 
-    /// the (empty) source text of every number_float event: binary formats
-    /// have none, and a string_t built for each float would cost more
-    const string_t no_source{};
-
     /// the containers that have been opened and not closed yet; see @ref container_frame
     std::vector<container_frame> container_stack{};
 

single_include/nlohmann/json_view.hpp

@@ -1218,6 +1218,57 @@ inline const unsigned char* find_block_comment_end(const unsigned char* p, const
     return p;
 }
 
+/// The digits of a fraction, whose number varies from one number to the next: skip_digits() with 16 bytes compared at
+/// once and the end of the run taken from a bit mask (SSE2), instead of a compare and a branch for each digit. That
+/// is about a third of the instructions for a fraction of 12 digits, and no branch that is mispredicted when the
+/// lengths vary. Where there is no SSE2, skip_digits() is used.
+#if NLOHMANN_VIEW_SSE2
+// The last bytes of the input, one at a time. GCC unrolls this loop in every inlined copy of the function below, which
+// makes the parse loop 4 KiB larger and slows down documents that have hardly any fractions; out of line, it does
+// not. Clang does better with the loop inline (a call in the parse loop costs it more than the unrolling).
+#if defined(__GNUC__) && !defined(__clang__)
+inline NLOHMANN_VIEW_NOINLINE const unsigned char* skip_digits_tail(const unsigned char* p, const unsigned char* e) noexcept
+{
+    while (p != e && is_digit(*p))
+    {
+        ++p;
+    }
+    return p;
+}
+#else
+NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* skip_digits_tail(const unsigned char* p, const unsigned char* e) noexcept
+{
+    while (p != e && is_digit(*p))
+    {
+        ++p;
+    }
+    return p;
+}
+#endif
+
+NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* skip_fraction_digits(const unsigned char* p, const unsigned char* e) noexcept
+{
+    while (e - p >= 16)
+    {
+        const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
+        // a digit is above '/' and below ':' (signed compares: a byte with the top bit set is negative, and neither)
+        const __m128i digits = _mm_and_si128(_mm_cmpgt_epi8(in, _mm_set1_epi8('/')), _mm_cmplt_epi8(in, _mm_set1_epi8(':')));
+        const auto not_digit = static_cast<std::uint64_t>(~static_cast<unsigned>(_mm_movemask_epi8(digits)) & 0xFFFFu);
+        if (not_digit != 0)
+        {
+            return p + count_trailing_zeros(not_digit);
+        }
+        p += 16;
+    }
+    return skip_digits_tail(p, e);
+}
+#else
+NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* skip_fraction_digits(const unsigned char* p, const unsigned char* e) noexcept
+{
+    return skip_digits(p, e);
+}
+#endif
+
 /// powers of ten up to 10^19 as integers
 inline std::uint64_t int_pow10(unsigned k) noexcept
 {
@@ -2324,7 +2375,7 @@ indent_done:
             {
                 ++p;
                 const unsigned char* const f0 = p;
-                p = skip_digits(p, e);
+                p = skip_fraction_digits(p, e);
                 if (NLOHMANN_VIEW_UNLIKELY(p == f0))
                 {
                     return fail(error_code::number_after_dot);

tests/src/unit-cbor.cpp

@@ -29,9 +29,7 @@ using nlohmann::json;
 #include "round_trip_corpus.hpp"
 #include "test_utils.hpp"
 #include "sax_countdown.hpp"
-#include "sax_event_loggers.hpp"
 using utils::SaxCountdown;
-using utils::SaxEventLogger;
 
 
 TEST_CASE("CBOR")
@@ -2448,78 +2446,6 @@ TEST_CASE("CBOR SAX parsing stops at every event")
     CHECK(count_events(std::vector<std::uint8_t>({0xBF, 0x61, 'a', 0x9F, 0x01, 0xFF, 0xFF})) == 6);
 }
 
-TEST_CASE("CBOR arrays of scalars from contiguous and non-contiguous input")
-{
-    // contiguous input reads the integers, floats, booleans, and nulls an
-    // array begins with in one loop, other input value by value; both must
-    // give the same events, value, or error for the complete document and for
-    // every truncation of it, and stop at every event a SAX handler rejects
-    const std::vector<std::uint8_t> cbor =
-    {
-        0xA3,                                                       // map(3)
-        0x61, 'a',                                                  // "a"
-        0x98, 0x16,                                                 // array(22)
-        0x00, 0x17, 0x18, 0xFF, 0x19, 0x01, 0x00,                   // 0, 23, 255, 256
-        0x1A, 0x00, 0x01, 0x00, 0x00,                               // 65536
-        0x1B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,       // 18446744073709551615
-        0x20, 0x37, 0x38, 0xFF, 0x39, 0x01, 0x00,                   // -1, -24, -256, -257
-        0x3A, 0x00, 0x01, 0x00, 0x00,                               // -65537
-        0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,       // -18446744073709551616 (a float)
-        0x3B, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,       // -9223372036854775808
-        0xF4, 0xF5, 0xF6,                                           // false, true, null
-        0xFA, 0x3F, 0xC0, 0x00, 0x00,                               // 1.5 (float32)
-        0xFB, 0xBF, 0xB9, 0x99, 0x99, 0x99, 0x99, 0x99, 0x9A,       // -0.1
-        0xF9, 0x3C, 0x00,                                           // 1.0 (float16, ends the loop)
-        0x01, 0x61, 'a',                                            // 1, "a"
-        0xFB, 0x40, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,       // 2.5
-        0x61, 'b',                                                  // "b"
-        0x83,                                                       // array(3)
-        0x82, 0xFB, 0x3F, 0xF4, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // [1.25,
-        0xFB, 0xC0, 0x0E, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,       //  -3.75]
-        0x82, 0xFB, 0x3F, 0xF4, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // [1.25,
-        0xFB, 0xC0, 0x0E, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,       //  -3.75]
-        0x80,                                                       // []
-        0x61, 'c',                                                  // "c"
-        0x84, 0xA0, 0x01, 0xC1, 0x02, 0x81, 0x03                    // [{}, 1, tag(1) 2, [3]]
-    };
-    const auto tags = json::cbor_tag_handler_t::ignore;
-
-    const json j = json::from_cbor(cbor, true, true, tags);
-    CHECK(j["a"].size() == 22);
-    CHECK(j["a"][5] == 18446744073709551615u);
-    CHECK(j["a"][11].is_number_float());
-    CHECK(j["a"][11] == -18446744073709551616.0);
-    CHECK(j["a"][12] == (std::numeric_limits<std::int64_t>::min)());
-    CHECK(j["a"][17] == -0.1);
-    CHECK(j["b"] == json::parse("[[1.25, -3.75], [1.25, -3.75], []]"));
-    CHECK(j["c"] == json::parse("[{}, 1, 2, [3]]"));
-
-    for (std::size_t length = 0; length <= cbor.size(); ++length)
-    {
-        CAPTURE(length)
-        const std::vector<std::uint8_t> contiguous(cbor.begin(), cbor.begin() + static_cast<std::ptrdiff_t>(length));
-        const std::list<std::uint8_t> bytewise(contiguous.begin(), contiguous.end());
-
-        CHECK(json::from_cbor(contiguous, true, false, tags).dump() == json::from_cbor(bytewise.begin(), bytewise.end(), true, false, tags).dump());
-
-        SaxEventLogger from_contiguous;
-        SaxEventLogger from_bytewise;
-        CHECK(json::sax_parse(contiguous, &from_contiguous, json::input_format_t::cbor, true, false, false, tags)
-              == json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::cbor, true, false, false, tags));
-        CHECK(from_contiguous.events == from_bytewise.events);
-    }
-
-    for (int events = 0; events <= 60; ++events)
-    {
-        CAPTURE(events)
-        const std::list<std::uint8_t> bytewise(cbor.begin(), cbor.end());
-        SaxCountdown from_contiguous(events);
-        SaxCountdown from_bytewise(events);
-        CHECK(json::sax_parse(cbor, &from_contiguous, json::input_format_t::cbor, true, false, false, tags)
-              == json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::cbor, true, false, false, tags));
-    }
-}
-
 TEST_CASE("CBOR indefinite-length strings do not recurse per chunk")
 {
     // Reading an indefinite-length string or byte array used to call itself

tests/src/unit-json_view_builder.cpp

@@ -523,6 +523,47 @@ TEST_CASE("json_view builder: strings across vector blocks")
     }
 }
 
+TEST_CASE("json_view builder: fraction digits")
+{
+    // The digits of a fraction are skipped 16 bytes at a time where there are
+    // that many left (SSE2), and one at a time after that. Fractions of every
+    // length from 0 to 40 digits, followed by every kind of byte that ends a
+    // number or lies next to the digits in ASCII ('/' and ':'), as values, as
+    // keys' neighbours, and at the very end of the text (a vector without a
+    // terminating NUL). The length of the number is part of the value that
+    // the test compares with json::parse().
+    const std::string followers = ",]}/:.eE+-x \n\t\"09";
+    std::mt19937 rng(2028); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
+    for (std::size_t digits = 0; digits <= 40; ++digits)
+    {
+        for (int i = 0; i < 3; ++i)
+        {
+            std::string fraction;
+            for (std::size_t k = 0; k < digits; ++k)
+            {
+                fraction += static_cast<char>('0' + (rng() % 10));
+            }
+            const std::string number = std::string(rng() % 2 == 0 ? "-" : "") + std::to_string(rng() % 100000) + "." + fraction;
+            check_same(number);
+            check_same("[" + number + "]");
+            check_same("[" + number + "," + number + "]");
+            check_same("{\"a\":" + number + "}");
+            for (const char follower : followers)
+            {
+                check_same(number + follower);
+                check_same("[" + number + follower + "]");
+                check_same("[" + number + follower + std::string(rng() % 20, '1'));
+            }
+            // everything near the end of the text: the number cut in every place
+            for (std::size_t cut = 1; cut <= 18 && cut <= number.size(); ++cut)
+            {
+                check_same(number.substr(0, number.size() - cut));
+                check_same("[" + number.substr(0, number.size() - cut));
+            }
+        }
+    }
+}
+
 TEST_CASE("json_view builder: escapes between runs of text")
 {
     // The text between escapes is copied 16 bytes at a time where there are