Parse JSON in one block of memory with a fused pointer-based parser (#6031)

* Parse JSON in one block of memory with a fused pointer-based parser

The parser and the lexer exchange one token at a time, and the lexer keeps
its state (the current character, a pending unget, where the token starts)
in members, which every character pays for: json::accept() of twitter.json
took 2.8x as long as json_document::accept(). detail::contiguous_parser
reads text in one block of memory through a local pointer, with the
grammar, the whitespace and the tokens in one loop (after yyjson's reader),
and drives a SAX handler: parse()'s DOM handler, accept()'s handler, or the
user's handler of sax_parse(). It is used with the default options (no
callback, no comments, no trailing commas; sax_parse() with strict = true).

Errors are reported by the library parser, so the exception, its message and
its position stay the same. contiguous_parser stops at the token that the
library parser would reject, before reporting it, so the events up to the
error are the same; then:
- parse() runs the library parser with a handler that only reports the
  error (json_sax_error_reporter), with the lexer of parse(), which is
  instantiated anyway;
- sax_parse() runs it with a handler that forwards only parse_error() to
  the user's handler (json_sax_error_forwarder), so no event arrives twice;
- accept() returns the verdict.
A handler that returns false stops the parse as before. An error at the end
of a large input costs parse() about 1.2x of what it cost before.

On an Apple M1 Max: accept() 1.6-2.0x as fast, parse() 10-16% faster,
ordered_json 11-16%, sax_parse() with a user's handler 1.36-1.46x, small
messages 12-22% (parse()) and 2.1-2.4x (accept()). On x86-64 (GCC 15,
Clang 21): accept() 1.6-1.9x, parse() 5-17%, ordered_json 12-19%,
sax_parse() 1.32-1.57x. A translation unit that parses a string compiles 3%
(-O0) and 9% (-O2) slower.

unit-contiguous_parser.cpp compares the contiguous path with the byte path
(std::list iterators) on corner cases, generated documents, their
truncations and mutations, deep nesting, and narrow number types: the same
values with the same number types and float bits, exception messages,
accept() verdicts and sax_parse() events.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI: unused parameter without exceptions, cpplint, BUILD.bazel

- json_sax_error_reporter::parse_error: mark the exception unused, as
  JSON_THROW does not use it with JSON_NOEXCEPTION
- unit-contiguous_parser.cpp: compare the exceptions of both paths only
  when exceptions are enabled (invalid input aborted the test otherwise)
- silence cpplint's readability/braces false positives on the if
  statements inside the parser's macros
- add contiguous_parser.hpp to BUILD.bazel

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* accept(): skip digits without computing their value

accept() needs the value of a number only for a float whose exponent may
reach the largest value. skip_digits() finds the end of the digits without
the multiply-adds of accumulate_digits(), and once an 8-byte word fails the
digit test, the remaining loop needs no bounds check: a non-digit is in
that word. The value is computed afterwards for the rare floats that need
it.

accept() of canada.json: -16 % on Apple M1, -22 % with GCC 15 and -21 %
with Clang 21 on x86-64; files with many numbers -9 to -15 %.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: mark parser callback in unit-contiguous_parser noexcept

Fixes the -Werror=noexcept failure with GCC 26 (ci_test_standards_gcc (26)).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: sweep_allocator::construct() accepts pointers to const

Recent libc++ (emsdk latest) probes construct(const T*, const T&) in an
unevaluated context while checking for trivial copies; the static_cast to
void* in the noexcept specification then fails to compile.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
(cherry picked from commit 794bf70823c12a2a1cb2a8a95f34dfb6357746a8)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: guard sweep_allocator::construct() against const types instead of const_cast

The const_cast added in the previous commit is rejected by
cppcoreguidelines-pro-type-const-cast (clang-tidy). Make construct() not
viable for const U instead, which is what libc++ probes for.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
(cherry picked from commit de099fa607001b710862fd1bbecef3609c54cbc4)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI: astyle and clang-tidy for sweep_allocator::construct()

astyle 3.4.13 reformats the enable_if<!...> default template argument
(Check amalgamation). Delete the const overload instead, which makes the
libc++ probe for const objects fail in the SFINAE context just the same
and needs no <type_traits>.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Contiguous parser: review fixes (wording, macros, diagnostic positions)

- "default parser" instead of "library parser" in comments and tests
- the three comments in json.hpp say the same thing
- current() replaces the repeated `p != end ? *p : 0`
- the backslashes of each macro are aligned
- the docs say that JSON_DIAGNOSTIC_POSITIONS turns the fast path off

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

include/nlohmann/detail/input/number_parse.hpp

@@ -316,6 +316,30 @@ JSON_HEDLEY_ALWAYS_INLINE std::size_t accumulate_digits(const char* data, std::s
     return i;
 }
 
+/// the index of the first byte at or after @a i that is not a digit: the
+/// digits of accumulate_digits() without their value
+JSON_HEDLEY_ALWAYS_INLINE std::size_t skip_digits(const char* data, std::size_t i, std::size_t avail) noexcept
+{
+    while (avail - i >= 8)
+    {
+        if (!is_eight_digits(read_eight_bytes(data + i)))
+        {
+            // a byte of the word is not a digit: no bounds checks before it
+            while (static_cast<unsigned char>(data[i] - '0') <= 9)
+            {
+                ++i;
+            }
+            return i;
+        }
+        i += 8;
+    }
+    while (i < avail && data[i] >= '0' && data[i] <= '9')
+    {
+        ++i;
+    }
+    return i;
+}
+
 /// whether [first, last) contains a digit other than '0'; a plain loop, as
 /// GCC calls std::any_of out of line even for the usual empty range
 JSON_HEDLEY_ALWAYS_INLINE bool has_nonzero_digit(const char* first, const char* last) noexcept

include/nlohmann/detail/input/string_scan.hpp

@@ -520,7 +520,10 @@ JSON_HEDLEY_NEVER_INLINE inline std::size_t string_bulk_run_from(const unsigned
 }
 #endif
 
-// string_bulk_run() for the lexer. With NEON, only the first four words are
+// string_bulk_run() for the lexer. The run must be the longest one: the lexer
+// would only lose speed with a shorter run, but contiguous_parser rejects a
+// string whose run ends at a byte other than a quote or a backslash, so a
+// shorter run would reject valid input there. With NEON, only the first four words are
 // checked inline (most keys and short values end there), and longer runs and
 // non-ASCII text continue out of line. The lexer's scan function runs once per
 // token, and the vector code inlined into it needs more registers, which every

include/nlohmann/json.hpp

@@ -58,6 +58,7 @@
 #include <nlohmann/detail/exceptions.hpp> // IWYU pragma: export
 #include <nlohmann/detail/hash.hpp>
 #include <nlohmann/detail/input/binary_reader.hpp>
+#include <nlohmann/detail/input/contiguous_parser.hpp>
 #include <nlohmann/detail/input/input_adapters.hpp>
 #include <nlohmann/detail/input/json_sax.hpp>
 #include <nlohmann/detail/input/lexer.hpp>
@@ -3109,6 +3110,15 @@ public:
                                  const bool ignore_trailing_commas)
     {
         basic_json result;
+        // JSON in one block of memory, with the default options, is read by
+        // the faster contiguous_parser; on an error, the default parser reads
+        // it again to report it (see contiguous_parser)
+        using contiguous = std::integral_constant<bool, detail::can_parse_contiguous<basic_json, typename std::decay<InputAdapterType>::type>::value>;
+        if (cb == nullptr && !ignore_comments && !ignore_trailing_commas
+                && detail::parse_contiguous(adapter, result, allow_exceptions, contiguous {}))
+        {
+            return result;
+        }
         parser(std::forward<InputAdapterType>(adapter), std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas).parse(true, result);
         return result;
     }
@@ -3119,8 +3129,17 @@ public:
                             const bool ignore_comments,
                             const bool ignore_trailing_commas)
     {
+        // JSON in one block of memory, with the default options, is checked by
+        // the faster contiguous_parser (see parse_impl())
+        using adapter_t = typename std::decay<InputAdapterType>::type;
+        bool accepted = false;
+        if (!ignore_comments && !ignore_trailing_commas
+                && detail::accept_contiguous<basic_json>(adapter, accepted, std::integral_constant<bool, detail::can_parse_contiguous<basic_json, adapter_t>::value> {}))
+        {
+            return accepted;
+        }
         // the parser and lexer that discard the values of tokens (see lexer)
-        return ::nlohmann::detail::parser<basic_json, typename std::decay<InputAdapterType>::type, true>(std::forward<InputAdapterType>(adapter),
+        return ::nlohmann::detail::parser<basic_json, adapter_t, true>(std::forward<InputAdapterType>(adapter),
                 nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
     }
 
@@ -3135,6 +3154,15 @@ public:
     {
         if (format == input_format_t::json)
         {
+            // JSON in one block of memory, with the default options, is read by
+            // the faster contiguous_parser; on an error, the default parser
+            // reads it again to report it (see parse_impl())
+            bool result = false;
+            if (strict && !ignore_comments && !ignore_trailing_commas
+                    && detail::sax_parse_contiguous<basic_json>(ia, sax, result, std::integral_constant<bool, detail::can_parse_contiguous<basic_json, InputAdapterType>::value> {}))
+            {
+                return result;
+            }
             return parser(std::move(ia), nullptr, true, ignore_comments, ignore_trailing_commas).sax_parse(sax, strict);
         }
         return detail::binary_reader<basic_json, InputAdapterType, SAX>(std::move(ia), format).sax_parse(sax, strict, tag_handler);