wt/json
1
0 Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-11-10 16:31:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
9bee955345cde47305d1470ece021817fbce2224
json/BUILD.bazel
T
Niels LohmannandClaude Sonnet 5.5 33f15727ce Parse JSON in one block of memory with a fused pointer-based parser (#6031) 
* Parse JSON in one block of memory with a fused pointer-based parser

The parser and the lexer exchange one token at a time, and the lexer keeps
its state (the current character, a pending unget, where the token starts)
in members, which every character pays for: json::accept() of twitter.json
took 2.8x as long as json_document::accept(). detail::contiguous_parser
reads text in one block of memory through a local pointer, with the
grammar, the whitespace and the tokens in one loop (after yyjson's reader),
and drives a SAX handler: parse()'s DOM handler, accept()'s handler, or the
user's handler of sax_parse(). It is used with the default options (no
callback, no comments, no trailing commas; sax_parse() with strict = true).

Errors are reported by the library parser, so the exception, its message and
its position stay the same. contiguous_parser stops at the token that the
library parser would reject, before reporting it, so the events up to the
error are the same; then:
- parse() runs the library parser with a handler that only reports the
  error (json_sax_error_reporter), with the lexer of parse(), which is
  instantiated anyway;
- sax_parse() runs it with a handler that forwards only parse_error() to
  the user's handler (json_sax_error_forwarder), so no event arrives twice;
- accept() returns the verdict.
A handler that returns false stops the parse as before. An error at the end
of a large input costs parse() about 1.2x of what it cost before.

On an Apple M1 Max: accept() 1.6-2.0x as fast, parse() 10-16% faster,
ordered_json 11-16%, sax_parse() with a user's handler 1.36-1.46x, small
messages 12-22% (parse()) and 2.1-2.4x (accept()). On x86-64 (GCC 15,
Clang 21): accept() 1.6-1.9x, parse() 5-17%, ordered_json 12-19%,
sax_parse() 1.32-1.57x. A translation unit that parses a string compiles 3%
(-O0) and 9% (-O2) slower.

unit-contiguous_parser.cpp compares the contiguous path with the byte path
(std::list iterators) on corner cases, generated documents, their
truncations and mutations, deep nesting, and narrow number types: the same
values with the same number types and float bits, exception messages,
accept() verdicts and sax_parse() events.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI: unused parameter without exceptions, cpplint, BUILD.bazel

- json_sax_error_reporter::parse_error: mark the exception unused, as
  JSON_THROW does not use it with JSON_NOEXCEPTION
- unit-contiguous_parser.cpp: compare the exceptions of both paths only
  when exceptions are enabled (invalid input aborted the test otherwise)
- silence cpplint's readability/braces false positives on the if
  statements inside the parser's macros
- add contiguous_parser.hpp to BUILD.bazel

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* accept(): skip digits without computing their value

accept() needs the value of a number only for a float whose exponent may
reach the largest value. skip_digits() finds the end of the digits without
the multiply-adds of accumulate_digits(), and once an 8-byte word fails the
digit test, the remaining loop needs no bounds check: a non-digit is in
that word. The value is computed afterwards for the rare floats that need
it.

accept() of canada.json: -16 % on Apple M1, -22 % with GCC 15 and -21 %
with Clang 21 on x86-64; files with many numbers -9 to -15 %.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: mark parser callback in unit-contiguous_parser noexcept

Fixes the -Werror=noexcept failure with GCC 26 (ci_test_standards_gcc (26)).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: sweep_allocator::construct() accepts pointers to const

Recent libc++ (emsdk latest) probes construct(const T*, const T&) in an
unevaluated context while checking for trivial copies; the static_cast to
void* in the noexcept specification then fails to compile.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
(cherry picked from commit 794bf70823c12a2a1cb2a8a95f34dfb6357746a8)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: guard sweep_allocator::construct() against const types instead of const_cast

The const_cast added in the previous commit is rejected by
cppcoreguidelines-pro-type-const-cast (clang-tidy). Make construct() not
viable for const U instead, which is what libc++ probes for.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
(cherry picked from commit de099fa607001b710862fd1bbecef3609c54cbc4)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI: astyle and clang-tidy for sweep_allocator::construct()

astyle 3.4.13 reformats the enable_if<!...> default template argument
(Check amalgamation). Delete the const overload instead, which makes the
libc++ probe for const objects fail in the SFINAE context just the same
and needs no <type_traits>.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Contiguous parser: review fixes (wording, macros, diagnostic positions)

- "default parser" instead of "library parser" in comments and tests
- the three comments in json.hpp say the same thing
- current() replaces the repeated `p != end ? *p : 0`
- the backslashes of each macro are aligned
- the docs say that JSON_DIAGNOSTIC_POSITIONS turns the fast path off

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 07:08:44 +01:00

118 lines
5.1 KiB
Python
Raw Blame History

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_license//rules:license.bzl", "license")
package(
default_applicable_licenses = [":license"],
)
exports_files([
"LICENSE.MIT",
])
license(
name = "license",
license_kinds = ["@rules_license//licenses/spdx:MIT"],
license_text = "LICENSE.MIT",
)
cc_library(
name = "json",
hdrs = [
"include/nlohmann/adl_serializer.hpp",
"include/nlohmann/byte_container_with_subtype.hpp",
"include/nlohmann/detail/abi_config.hpp",
"include/nlohmann/detail/abi_macros.hpp",
"include/nlohmann/detail/bit_ops.hpp",
"include/nlohmann/detail/conversions/from_json.hpp",
"include/nlohmann/detail/conversions/to_chars.hpp",
"include/nlohmann/detail/conversions/to_json.hpp",
"include/nlohmann/detail/conversions/zmij.hpp",
"include/nlohmann/detail/exceptions.hpp",
"include/nlohmann/detail/hash.hpp",
"include/nlohmann/detail/input/binary_reader.hpp",
"include/nlohmann/detail/input/contiguous_parser.hpp",
"include/nlohmann/detail/input/input_adapters.hpp",
"include/nlohmann/detail/input/json_sax.hpp",
"include/nlohmann/detail/input/lexer.hpp",
"include/nlohmann/detail/input/number_parse.hpp",
"include/nlohmann/detail/input/parser.hpp",
"include/nlohmann/detail/input/position_t.hpp",
"include/nlohmann/detail/input/pow5_table.hpp",
"include/nlohmann/detail/input/simd_scan.hpp",
"include/nlohmann/detail/input/string_scan.hpp",
"include/nlohmann/detail/iterators/internal_iterator.hpp",
"include/nlohmann/detail/iterators/iter_impl.hpp",
"include/nlohmann/detail/iterators/iteration_proxy.hpp",
"include/nlohmann/detail/iterators/iterator_traits.hpp",
"include/nlohmann/detail/iterators/json_reverse_iterator.hpp",
"include/nlohmann/detail/iterators/primitive_iterator.hpp",
"include/nlohmann/detail/json_custom_base_class.hpp",
"include/nlohmann/detail/json_pointer.hpp",
"include/nlohmann/detail/json_ref.hpp",
"include/nlohmann/detail/macro_scope.hpp",
"include/nlohmann/detail/macro_unscope.hpp",
"include/nlohmann/detail/meta/call_std/begin.hpp",
"include/nlohmann/detail/meta/call_std/end.hpp",
"include/nlohmann/detail/meta/cpp_future.hpp",
"include/nlohmann/detail/meta/detected.hpp",
"include/nlohmann/detail/meta/identity_tag.hpp",
"include/nlohmann/detail/meta/is_sax.hpp",
"include/nlohmann/detail/meta/std_fs.hpp",
"include/nlohmann/detail/meta/type_traits.hpp",
"include/nlohmann/detail/meta/void_t.hpp",
"include/nlohmann/detail/output/binary_writer.hpp",
"include/nlohmann/detail/output/error_handler.hpp",
"include/nlohmann/detail/output/output_adapters.hpp",
"include/nlohmann/detail/output/serializer.hpp",
"include/nlohmann/detail/recursion_depth_limit.hpp",
"include/nlohmann/detail/string_concat.hpp",
"include/nlohmann/detail/string_escape.hpp",
"include/nlohmann/detail/string_utils.hpp",
"include/nlohmann/detail/value_t.hpp",
"include/nlohmann/detail/view/builder.hpp",
"include/nlohmann/detail/view/compare.hpp",
"include/nlohmann/detail/view/document_data.hpp",
"include/nlohmann/detail/view/edit.hpp",
"include/nlohmann/detail/view/edit_storage.hpp",
"include/nlohmann/detail/view/errors.hpp",
"include/nlohmann/detail/view/image.hpp",
"include/nlohmann/detail/view/input.hpp",
"include/nlohmann/detail/view/iterator.hpp",
"include/nlohmann/detail/view/lookup.hpp",
"include/nlohmann/detail/view/macro_scope.hpp",
"include/nlohmann/detail/view/macro_unscope.hpp",
"include/nlohmann/detail/view/materialize.hpp",
"include/nlohmann/detail/view/node.hpp",
"include/nlohmann/detail/view/number.hpp",
"include/nlohmann/detail/view/object_index.hpp",
"include/nlohmann/detail/view/pointer.hpp",
"include/nlohmann/detail/view/scan.hpp",
"include/nlohmann/detail/view/serializer.hpp",
"include/nlohmann/detail/view/simd.hpp",
"include/nlohmann/detail/view/string_ref.hpp",
"include/nlohmann/detail/view/value.hpp",
"include/nlohmann/detail/view/view_mode.hpp",
"include/nlohmann/json.hpp",
"include/nlohmann/json_fwd.hpp",
"include/nlohmann/json_literals.hpp",
"include/nlohmann/json_view.hpp",
"include/nlohmann/ordered_map.hpp",
"include/nlohmann/thirdparty/hedley/hedley.hpp",
"include/nlohmann/thirdparty/hedley/hedley_undef.hpp",
],
includes = ["include"],
visibility = ["//visibility:public"],
)
cc_library(
name = "singleheader-json",
hdrs = [
"single_include/nlohmann/json.hpp",
"single_include/nlohmann/json_fwd.hpp",
"single_include/nlohmann/json_literals.hpp",
"single_include/nlohmann/json_view.hpp",
],
includes = ["single_include"],
visibility = ["//visibility:public"],
)
Reference in new issue View Git Blame Copy Permalink