wt/json - json - Gitea: Git with a cup of tea

wt/json

0 Fork 0

mirror of https://github.com/nlohmann/json.git synced 2026-10-25 12:04:42 +00:00

Author	SHA1	Message	Date
Niels Lohmann	7833865dbd	Merge develop into perf/view-compare single_include is a fresh amalgamation. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-25 12:00:01 +01:00
Niels Lohmann	cc917d7123	docs: cross-reference json_view and json_document for 3.13.0 (#5892 ) * docs: cross-reference json_view and json_document for 3.13.0 Document the new zero-copy document/view API across the docs: add a "Documents and views" section to the basic_json API hub linking to all specializations, point parsing/conversion/modifying pages to the view-based alternatives, add document/view sections to feature pages, update the landing page to mention json_view.hpp/json_literals.hpp, and cross-link debugging/exceptions. Also document wide-string inputs (and ill-formed UTF-16/32 rejection), add parse/accept examples for wide strings, fix related doc gaps, and align features/parsing and FAQ with the added support. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: fix article before std::istream in FAQ Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: clarify document inputs, decoding, get<T>() and editing Address review comments: say which inputs are borrowed and that a wide string pointer is an extra input, which values are decoded during parsing, that get<T>() reads only built-in types directly, all forms of set() and that the document's insert() is array-only, and how to store wide strings. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: integers are converted while parsing a document Also say "They accept every input" in basic_json_document::parse. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-25 06:18:07 +01:00
Niels Lohmann	3eae5680d6	from_json: number-first dispatch for arithmetic types, loop for std::array (#5902 ) * from_json: number-first dispatch for arithmetic types, loop for std::array (#5891) from_json(ArithmeticType&) now reads numbers through get_number_value() and handles a boolean on a cold path, which puts the three number types first in the per-element type dispatch. get<std::array<T, N>>() always took the identity_tag overload, a pack expansion of N calls j.at(i).get<T>() that clang unrolls into straight-line code (about 28,000 instructions for N = 1000), so its speed depended on inlining decisions in that body. For trivial element types, which cannot tell default construction plus assignment from construction in place, it now fills a std::array in a loop. That loop, shared with get_to(), checks the size once and reads the inner array directly; a short array still converts the elements it has and then throws the out_of_range error at() would throw. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Skip the get_to result check without exceptions Without exceptions, the get_to calls in CHECK_THROWS_WITH_AS do not run, so the array keeps its initial values (ci_test_noexceptions). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make is_trivial_element a struct so astyle does not indent it Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Expect the element path in the std::array type-error tests with JSON_DIAGNOSTICS Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Shorten from_json_inplace_array so astyle aligns its parameters The opening parenthesis of the previous name sat in column 41, beyond astyle's default --max-continuation-indent of 40, so the continuation lines fell back to a plain double indent. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use the compact template<...> style for the new from_json overloads Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-25 06:17:51 +01:00
Niels Lohmann	fe5e401870	Consolidate duplicated detail helpers and bodies (#5891 ) (#5901 ) * Add shared detail helpers for the #5891 cleanup Add detail::is_primitive(value_t), detail::hex4_lower(), detail::utf8_shape(), detail::input_format_name(), detail::empty_input_message(), the exception ids 116, 319, 320 and 416 to detail::exception_id, and make detail::is_ascii_copyable() constexpr. Later commits switch the duplicated code over to them. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 1.4: BSON writers share one write-pass UTF-8 sanitizer (#5891) write_bson_entry_header and write_bson_string re-derived the error-handler policy differently. Both now use sanitize_utf8_for_bson_write, which treats strict like keep (the size pass already validated) and sanitizes replace/ignore into the bytes the size pass measured. No context plumbing is needed. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.1.1: one MessagePack container prefix helper (#5891) Array and object prefixes differed only in the fix marker and the 16/32-bit markers; both now delegate to write_msgpack_container_prefix. Output is byte-identical. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.1.2: one iterative walker for CBOR and MessagePack (#5891) write_cbor_iterative and write_msgpack_iterative (and their _value_or_push helpers) were identical but for the head, scalar, and key writers. They are now one template over a small policy struct, so everything inlines. BON8 keeps its own walker (open string run, closing marker), noted in a comment. Signed-off-by: Niels Lohmann <mail@nlohmann.me> 2.1.3, 2.1.8: one UBJSON container start helper, shared frame base (#5891) write_ubjson_start_array/_object are now write_ubjson_start_container<IsArray>, keeping the array-only begin()+1 scan and excessive_valueless rule. ubjson_frame derives from binary_container_frame instead of duplicating it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.1.4: write_bytes helper for the byte-string write sites (#5891) The 13 oa.write_characters(reinterpret_cast<const CharType>(x.data()), n) sites now call write_bytes(const void, size_t); copy_characters sites and the buffer-based writes are untouched. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.1.5: shared object-key writer for CBOR and MessagePack (#5891) write_cbor_key and write_msgpack_key were the same SFINAE pair apart from the string/scalar writers; both now delegate to write_object_key<Walk>. UBJSON key and to_format_length stay as they are. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.3.10: writer-side format names from input_format_name (#5891) The CBOR/MessagePack/UBJSON/BSON/BJData/BON8 literals in throw_on_discarded and throw_integer_out_of_range calls now come from detail::input_format_name; the texts are identical, and UBJSON vs BJData still follows use_bjdata. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * serializer: constexpr ascii table, hex4_lower, dedupe helpers (#5891) 1.1: build ascii_copyable() at compile time from is_ascii_copyable and test find_ascii_copyable_run against it for every byte. 1.2/2.5.4: use detail::hex4_lower in both \uXXXX writers. 2.1.6: factor the repeated flush/emit blocks in dump_escaped_impl into always-inline helpers. 2.1.7: comment why the escape copy does not use put_buffer. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 1.6: use string_t-aware to_string/concat in meta() (#5891) Replaces the four std::to_string sites; rendered text unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 1.7: use detail::is_primitive in json.hpp switches (#5891) has_no_children, set_parents, max_size and the two diff walkers. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.2.1: factor insert()/iterator throw helpers (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.2.2: share swap() overloads via swap_impl (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.2.4: factor primitive reset and range check in erase()/range ctor (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.2.6: share push_back array bodies via push_back_array_impl (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.2.7: route parse/accept/sax_parse through _impl helpers (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> 2.2.8: share the vector-returning to_* bodies via to_binary_vector (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.2.10: merge destroy_string/destroy_binary into destroy_pointer (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.4.1/1.7: exceptions: share what_impl, flatten diagnostics switch (#5891) The four create() bodies build their message through one protected static helper; the diagnostics parent walk handles array/object and ignores all other types. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.4.2/2.4.3/1.7: json_pointer: share slash and escape checks, drop primitive case lists (#5891) The leading-slash and invalid '~' escape checks are shared helpers; the reference-token walks test object and array first and treat everything else as primitive. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 1.7/2.4.9: iter_impl, iteration_proxy: drop primitive case lists, share compare preamble (#5891) Object and array stay the first branches; everything else is the primitive case. operator== and operator< share the different-container check. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.4.5/2.4.6: from_json: share get_number_value and reserve_if_possible (#5891) The three number cases of get_arithmetic_value and the arithmetic from_json share one helper (boolean stays accepted only by the latter); the array and object reserve overload pairs become one. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.4.8: ordered_map: share erase_impl between the key erase overloads (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 1.7: json_pointer: flatten walk drops the primitive case list (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 1.2/1.3/1.5/2.5.1/2.5.2/2.5.5: share helpers in the view code (#5891) - detail::exception_id at every numeric id; throw_* helpers take it - hex_byte, hex4_lower, utf8_shape replace local hex/table code - throw_array_index_out_of_range/_past_end, throw_key_not_found - has_native_float_format replaces the hand-written IEEE check No change of messages, ids or output. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 1.5: use utf8_shape in get_bon8_string, add exhaustive UTF-8 validator test (#5891) get_bon8_string takes the valid range of the second byte from utf8_shape() (0xF5..0xF7 stay errors, unget behavior unchanged). validate_one_utf8 keeps its if-chain (hot bulk path); the new test checks decode, validate_one_utf8, utf8_shape, json::accept and the BON8 reader against each other for every lead and second byte. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.3.1: get_cbor_sized_bytes for the CBOR chunk getters (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.3.2: get_string_with_length for the plain-order string sites (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.3.3, 2.3.4, 2.3.10: unsupported_key_type, empty_input_message, input_format_name in the readers (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.3.7, 2.3.9: read_object_key and leave_container_and_check for CBOR/MessagePack/BON8 (#5891) UBJSON keys (different getter signature) and its container tail stay as is. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.3.5: json_sax_dom_error_state base for the DOM SAX parsers (#5891) Holds errored, allow_exceptions, parse_error() and is_errored(). The scalar handlers stay in each class (parse hot path). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.3.6: table of control-character messages in the lexer (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.3.8: finish_input helper for the strict-EOF block in the parser (#5891) sax_parse keeps its if (result) guard and return value. Also adds the clang-tidy NOLINT for the 2.3.6 message table. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.5.7: share parse_array_index between json_pointer and json_view (#5891) Move json_pointer::parse_array_index() and its status enum to namespace detail (json_pointer keeps a forwarding member), and let the json_view pointer resolution use it instead of its own copy of the RFC 6901 index grammar. Both accepted the same tokens with the same outcome. Also update the comment on detail::decode(), which listed four of the five UTF-8 validators and now points to the test that keeps them in sync. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Amalgamate (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.4.5: keep the inlined number cases in from_json(ArithmeticType&) (#5891) Calling get_number_value() from from_json(ArithmeticType&) made GetArray<std::array<int, 1000>> 2.4x slower (it changes the inlining of the std::array conversion), so that overload keeps its own switch, with a comment. get_arithmetic_value() still uses the shared helper. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 2.5.7: force-inline detail::parse_array_index (#5891) Moving parse_array_index out of json_pointer stopped clang from inlining it into the pointer resolution (JsonPointerAt/citm_catalog +17%, JsonPointerAt/twitter +8%); JSON_HEDLEY_ALWAYS_INLINE restores the baseline. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 1.5: keep get_bon8_string's own UTF-8 ranges (#5891) Taking the second-byte range from utf8_shape() made BON8 decoding 2-3% slower (FromBinaryShape/scalars, FromBinaryBuffer, alignment-pinned builds), so the byte path keeps its hand-written ranges. The exhaustive validator test still checks that it accepts the same sequences as the other validators. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Amalgamate (#5891) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix CI failures: constexpr, NRVO, cppcheck, IWYU (#5891) - json_sax_dom_error_state::is_errored() cannot be constexpr: the class is not a literal type (GCC 4.8, GCC -Wpedantic, Clang 3.5). - to_binary_vector: destroy the vector writer before returning, as the output_vector_sink sets the size of the vector in its destructor. Without NRVO (MSVC Debug in C++14 mode), the vector was moved out before it was resized. - parser::sax_parse: return false instead of result after the early exit (cppcheck identicalConditionAfterEarlyExit). - json.hpp: export detail/exceptions.hpp for IWYU, so json_view.hpp can name exception_id without including a detail header. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review: throw helpers take input_format_t, simpler type checks - throw_on_discarded() and throw_integer_out_of_range() take an input_format_t and call input_format_name() themselves - has_no_children(), set_parents() and max_size() check array, then object, instead of calling is_primitive() first - parse_impl() chains like accept_impl(); sax_parse_impl() uses if/else - std::array for the control character messages; negation<> in write_object_key(); clarify the is_primitive() comment Signed-off-by: Niels Lohmann <mail@nlohmann.me> * json_view: use exception_id in the value() overloads Signed-off-by: Niels Lohmann <mail@nlohmann.me> * json_sax: assert the invariant of place_container for the static analyzer Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-25 06:17:50 +01:00
Niels Lohmann	9707519b24	Move the json_view comparison tests into their own file With the tests this branch adds, the object file of unit-json_view_dump.cpp got too large for the MinGW x64 assembler ("file too big"). The test case "json_view comparison" is now unit-json_view_compare.cpp, unchanged and also built with C++17. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-24 20:03:58 +02:00
Niels Lohmann	52b72b5684	Improve documentation: byte layouts, release pages, See also, dark-mode images (#5906 ) - Make the iterator range illustrations legible in dark mode (prefers-color-scheme rules inside the SVGs). - Add a "Byte layout" section with a color-coded byte table to each binary format page (BJData, BON8, BSON, CBOR, MessagePack, UBJSON), and highlight the smallest size per file in the comparison table. - Add "See also" sections to the overview pages that lacked one. - Split the release notes into one page per release, grouped by minor version on the overview page, with an "API changes" section per release generated from the API pages' version histories (hooks/api_changes.py). - Fact-check the release notes and version histories against the release tags and correct wrong or misleading entries (e.g., json_serializer 2.1.0, merge_patch 3.1.0, items, count, operator[], to_bjdata, std::string_view key support). Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-24 17:57:44 +02:00
Niels Lohmann	2d040c89d0	Merge develop into perf/view-compare Develop silenced the same Infer false positive in materialize() (#5905): its version is kept. single_include is a fresh amalgamation. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-24 17:14:02 +02:00
Niels Lohmann	01139dc192	Silence an Infer false positive in json_view materialize (#5905 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-24 17:10:31 +02:00
Niels Lohmann	182b31f902	Mark Infer's null dereference in materialize() as a false positive Infer (ci_infer, also red on develop since #5882) reports a null dereference in json_sax_dom_parser::null() called from materialize(). The member pointer the parser stores a value of an object in is set by the key() before it, so it is not null there. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-24 15:40:31 +02:00
Niels Lohmann	6775994761	Merge develop into perf/view-compare Resolves the conflict in unit-json_view_image.cpp, where develop (#5889) and this branch each added a section to "json_view images: errors": both are kept. single_include is a fresh amalgamation. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-24 07:48:54 +02:00
Niels Lohmann	9c50bdd140	Make the DOM parser cheaper per container and per member (#5882 ) * Reserve the parser's nesting stack once The stack of open arrays and objects was a std::vector<std::uint8_t> that allocated at the first push_back and then reallocated at 1, 2, 4, 8 bytes, so a document nested four levels deep made four allocations before it parsed a value. A document that starts with a container now reserves 32 levels in one allocation; deeper documents grow as before. The loop that reads the tokens is the code it was, so large documents are not affected (a stack object with its own storage, and a reserve at every container, both cost the token loop instructions or cycles). Fixed cost of parsing a 124-byte object (macOS instructions): accept -7.6 %, DOM -2.5 %; no change for a single number. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Build arrays of the DOM parser in one exact-size allocation Reading an array of unknown length appended its elements to a std::vector one at a time, so a small array reallocated (and moved its elements) at sizes 1, 2, 4, ... The DOM SAX parser now collects the elements of arrays whose length is unknown on one stack shared by all open arrays and moves them into an array of exactly the right size when the closing bracket arrives. An array that grows past 16 elements switches to appending to its own vector, so large arrays pay no second copy. Binary formats, which announce the length, and builds with JSON_DIAGNOSTICS or JSON_DIAGNOSTIC_POSITIONS (whose values need their final place from the start) keep storing into the array directly. The stacks of open containers are reserved on first use instead of growing from one element. The binary readers get a DOM parser of their own, json_sax_dom_binary_parser: it is the plain parser that stores every value directly in its container (the code the readers used before). Their containers announce their size, so the bookkeeping of the collecting parser (frame entries, the destination of a finished container) was pure overhead for them: with the collecting parser FromBinaryShape nested was 3-12 % and containers up to 5 % more instructions (GCC) than before. With the plain parser Linux Ir of FromBinaryShape and FromBinaryBuffer is within 0.8 % of develop (Clang: identical). canada.json has 56 000 two-element arrays that took three allocations each and now take two. DOM parse Ir on Linux against develop (GCC / Clang): canada -7.0 % / -7.7 %, citm -0.8 % / -1.6 %, 124-byte object -5.4 % / -5.5 %; accept and SAX are untouched. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Reuse one value for the key events of the callback parser parse() with a callback built a new string value for every key it passed to the callback (a heap allocation and release per key), and copied the key a second time into the object. The callback parser now keeps one string value for key events, which it assigns, and moves the key into the object. Callback parse, Linux Ir (GCC / Clang): twitter -13.3 % / -11.7 %, citm -9.5 % / -7.6 %. (Earlier macOS measurement: twitter -19.5 % instructions.) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remember where the DOM parser stores the next value The DOM SAX parser looked up the innermost open container for every value: the end of its frame stack, then the object or array pointer in the frame. Compilers have to reload all of that after each value is stored, since a store of a value may alias the frame. The parser now keeps the innermost container's kind and, for an array that stores directly, the array itself in members, updated when a container opens or closes. Flat arrays of numbers are the case that mattered: parsing floats.json (1 000 000 doubles) into a DOM took 2 % more cycles than before the previous commit on macOS and now takes 0.4 % fewer than the code before it (signed_ints.json -0.8 %). Linux Ir is unchanged (canada -6.1 % GCC / -6.9 % Clang, citm -0.3 % / -0.8 %, twitter +0.2 % / 0.0 %). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Look up the keys of big ordered_json objects in a hash table ordered_map keeps its elements in a vector and searches it linearly for every key it adds, so parsing an object with n keys into ordered_json took O(n^2) key comparisons: 33 billion instructions for one object with 30 000 keys, where json needs 245 million. From 64 keys on, the DOM parser now keeps a hash table of the keys of the open object (hashes and positions only; the elements stay in the vector) and finds duplicate keys there. The result is the same, including the position of a key that occurs twice. Smaller objects, std::map based objects and key types without std::hash are untouched. ordered_json parse, macOS instructions: object with 1 000 keys 64.8 M -> 14.6 M, 10 000 keys 5 104 M -> 38.7 M, 30 000 keys 33 358 M -> 89 M. Linux Ir of an object with 2 000 keys: 43.0 M -> 1.9 M (GCC), 37.9 M -> 1.8 M (Clang). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Trim the work the DOM parser does per container and per member Opening and closing an array or object cost the DOM parser more than before the exact-size array change, which showed on documents of many small objects (jeopardy.json: +1.4 % Ir with GCC). This takes it back: - the stack of open containers keeps its first eight entries in the parser object instead of in a vector, so it needs no capacity check beyond the push itself and no allocation for typical documents; - a container is created in the place it ends up in, instead of as a temporary that is then moved there; - a new object member, which is null, is constructed over the null instead of being swapped with it (only with the default base class of basic_json); - the key index that big ordered_json objects use is only a part of the stack entries of objects that can have one; - an array that outgrows the stack reserves twice what it has so far, as a vector that grew by itself would have. DOM parse Ir of the series up to here against develop, Linux GCC / Clang: canada -6.6 % / -7.3 %, citm +1.2 % / +0.2 %, jeopardy +0.4 % / -0.1 %, 124-byte object -6.5 % / -6.2 %. (twitter measures +6.8 % / +6.6 % at this point in steady state, almost all of it in glibc's malloc/consolidate for the new allocation sizes; with the last commit of the series it is -0.1 % / 0.0 %.) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Test parsing containers of the sizes where the DOM parser changes storage The DOM parser collects the elements of an array on a stack (up to 16) and the members of an ordered_json object (up to 32) before it moves them into the container, and uses a hash table for keys from 64 members on. Parse arrays and objects of the sizes around these limits, nested into each other, with repeated keys, and as random trees that are built next to their text, and compare json and ordered_json against the value built through the API. The tests pass on the code from before these changes as well. The random trees are what finds the mistakes that are easy to make here: while the collecting change was being written, a collected array that outgrew the stack at the moment its parent had just reached the limit left its elements in the parent, and only these tests (142 failed assertions) showed it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not convert floats that cannot overflow in accept() accept() only has to find out whether a float token is finite: json_sax_acceptor ignores the value, and the parser rejects only a float that overflows. A float whose digits fit into 19 digits and whose decimal exponent is at most max_exponent10 - 19 is below 10^max_exponent10 and cannot overflow, so the lexer, told by accept() that the value is not needed, skips the conversion for those. A token that may overflow, with more digits, or in another float format is converted as before, so 1e309 and its neighbours are rejected as they were (the new test compares accept() with parse() on tokens around the limit). json::accept(): canada.json Ir -8.6 % (GCC) / -10.9 % (Clang), 124-byte object -3.2 %; cycles on the M1 (release benchmarks, LTO): canada -7.8 %, floats.json -9.0 %. The check tests the accept() flag first, so the parser that needs the value pays one load and branch: SaxParse canada +0.35 % Ir (GCC and Clang), DOM canada +0.4 %. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Collect the members of ordered_json objects on a stack until they are known An ordered_json object keeps its members in a vector and appends every new member to it, so reading an object of unknown size reallocated the vector at 1, 2, 4, 8, ... members, and ordered_map copies all keys on every reallocation. The DOM SAX parser now collects the members of such objects on one stack shared by all open objects, as it does the elements of arrays, and moves them into a vector of exactly the right size when the closing brace arrives. An object that gets past 32 members switches to appending to its own vector (and, from 64, to the key index of the earlier commit). The members are the same, in the same order, and a key that occurs twice keeps the place of its first occurrence. std::map based objects, objects whose size the input announces (binary formats), and builds with JSON_DIAGNOSTICS or JSON_DIAGNOSTIC_POSITIONS are untouched. What only ordered objects need (the collected members, the key indexes) is a member of the parser only for them, and the code that stores a value tries the array that takes its elements directly first. An array that outgrows the stack now reserves 32 elements, the capacity a vector that grew by itself has at that point; with the larger reserve before, parsing floats.json (1 000 000 numbers) was 2 % slower on Apple M1 than before this series. ordered_json parse, Linux Ir against develop (GCC / Clang): citm -11.6 % / -11.7 %, twitter -8.4 % / -8.4 %, jeopardy (12 000 objects of 7 keys) -10.5 % / -8.0 %, canada -3.9 % / -5.2 %, 124-byte object -20.9 % / -16.5 %. macOS time (M1, earlier base): citm -19 %, twitter -13 %, jeopardy -13 %, canada -14 %, 124-byte object -26 %. json parse: steady-state twitter -0.1 % (see the previous commits), citm -1.2 %. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Rename the key index's match callback so Flawfinder doesn't read it as std::equal Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bind the parser test's strings by reference in the range-for loop GCC and Clang 14 (-Wrange-loop-construct) reported the copy. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Collect container values only in basic_json's own parse functions A json_sax_dom_parser that a user constructs (as in the sax_no_exception example) must hold the values read before a parse error, as on develop: its value is used after the error. The values of containers of unknown size are now collected on a stack only with the new template parameter Collect (default false), which json::parse() and the json_view materialization set; without it, every value is stored in place as it arrives. A test checks the values kept after an error for json and ordered_json. Also fixes the clang-tidy, cpplint and Infer findings: the frame stack keeps its first frames in a std::array (left uninitialized) and the others in a std::unique_ptr, auto for create<>(), NOLINT for member initializers -Weffc++ needs, and an @infer-ignore for the nullptr that place_container() does not use. json::parse() and ordered_json::parse() instructions are unchanged (Apple clang; twitter, citm, canada, jeopardy: -0.03 % to +0.25 %). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Clarify the DOM parser's collection stacks and rename reenter_enclosing_container() Address review comments: explain why a closed container erases only the top of the shared stack, set the payload before the type, rename the helper that makes the enclosing container current again, and declare the parser's state stack right before reserving it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Export json_sax_dom_parser from the C++20 module for MSVC MSVC needs the detail classes that basic_json befriends to be visible in the module (#3970, as json_sax_dom_callback_parser); with its new template parameter, json_sax_dom_parser failed in ci_module_cpp20 (C2039, not a member of nlohmann::detail). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the overflow-limit numbers of the parser test in a std::vector clang-tidy (cppcoreguidelines-avoid-c-arrays) rejected the C array. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the clang-tidy findings of the ordered_json parser tests The container sizes are a std::array, the expected text uses raw string literals, the generator's arithmetic is in parentheses, auto for the result of make<>(), and single characters are appended as characters. clang-tidy now reports nothing on the lines the branch adds to unit-ordered_json.cpp and unit-class_parser.cpp (apart from checks CI's clang-tidy does not have). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-24 07:41:21 +02:00
Niels Lohmann	dd7686a269	json_view: faster reading, indexing and C-stream parsing (#5888 ) * Read C streams into a json_document in blocks json_document::parse(FILE) (and read(), accept(), parse_copy()) went through the library's file input adapter and appended the text character by character with fgetc. Read it with fread in blocks instead, like the std::istream overload already does. Parsing citm_catalog.json (1.7 MB) from a FILE: Apple M1: 80.6 M -> 8.4 M instructions, 4.8 ms -> 0.55 ms (parse of the same text from a std::string: 8.7 M) Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 47.0 M -> 13.7 M / 45.8 M -> 14.2 M The parsed documents and the error cases are identical (checked over the test corpus and 30 mutants of each file). A null FILE* is handled as before (by the library's adapter). Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Index arrays without subtrees directly operator[](idx), at(idx) and back() skipped the elements of an array one after the other, whatever they were. In an array none of whose elements is an array or object (numbers, strings, booleans, null) the elements are adjacent nodes, which the array node's own sizes tell: its subtree has exactly one node more than it has elements. Such an element is now the node idx after the array node; any other array is walked as before. Reading r[i] for every i of an array of 3000 integers: Apple M1: 28.8 M instructions, 3.1 ms -> 0.1 M instructions, 4 us Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 84.1 M -> 3.1 M / 75.0 M -> 3.0 M Arrays that contain arrays are unchanged. Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Skip the bounds checks of number and string tokens in NUL-terminated input For a NUL-terminated input (std::string, C string, string literal), the parser reads the terminator instead of comparing p with the end, via cursor::cur(). Number and string scanning still tested `p != e` before each look at the next byte (after the integer digits for '.', for 'e', after the exponent sign and digits, and after a string for its closing quote), although the terminator can never be one of those characters. They now use cur(), which is the plain load for NUL-terminated input and the bounds check for ptr+len input, as before. Parsing, Apple M1 instructions per parse: twitter 3.126 M -> 3.089 M (-1.2 %), citm_catalog 8.70 M -> 8.58 M (-1.3 %), jeopardy 306.1 M -> 302.5 M (-1.2 %), signed integers 281 M -> 266 M (-5.4 %), canada and floats -0.1 % / -0.9 %. Linux x86-64 Ir, GCC 13 / Clang 18: twitter -0.5 % / -0.0 %, signed integers +0.2 % / -0.7 %. Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make the index text of items() on arrays lazy basic_json_view::items() on an array formatted every element's index with std::to_string when the item was made, whether or not the loop asks for the key. The item now keeps the position and makes the text on the first key() call. Objects are not affected. Iterating over the items of an array of 3000 numbers and reading only the values: Apple M1: 25.5 us -> 4.6 us (0.45 M -> ~0 net instructions) Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 14.9 M -> 4.1 M / 15.3 M -> 4.6 M Reading key() costs what it did (24.5 us). Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make the default of value() with a C string only if the member is missing basic_json_view::value(key, "text") and value(pointer, "text") built a std::string from the default first and passed it on, so a call allocated for the default even when the member exists, and copied it again when it does not. They now look the member up and make the result directly: from the member if there is one, else from the C string. Behavior, including the exceptions, is the same. Two calls (member found, member missing) with a 40 character default, Apple M1: 1.6 K -> 0.7 K instructions Linux x86-64 Ir (500 rounds), GCC 13 / Clang 18: 2.46 M -> 2.22 M / 2.40 M -> 2.15 M Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Materialize scalars directly and keep the stack of small subtrees local basic_json_view::materialize() (and get<BasicJsonType>()) always set up the SAX handler that parse() uses and a std::vector for the open containers, sized from the document's deepest nesting. For a scalar, which is most of what is materialized one value at a time, neither is needed: the value is made directly (the same constructors the handler ends up calling). For arrays and objects the stack of open containers is now a local array for the first 16 levels, and the vector is only used for deeper documents. Materializing five scalars of a small document, Apple M1: 3.6 K -> 0.9 K instructions Linux x86-64 Ir (500 rounds), GCC 13 / Clang 18: 2.96 M -> 2.37 M / 2.90 M -> 2.29 M Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Copy the text between escapes while scanning it A string that contains an escape is decoded by builder::slow_string(). After each escape it scanned the next run of plain bytes (8 or 16 byte-by-byte steps, then a vector loop), and then copied that run to the decode arena in a second pass. The run is now copied 16 bytes at a time while it is scanned (one vector load, a store of the same 16 bytes, and the mask of the bytes that end the run), and the write position lives in a local variable, not in the builder, since a store of a character may alias the builder's members and made the compiler reload it after every escape. A run that ends at a quote needs no scan, and non-ASCII text goes through the UTF-8 check as before; the portable code is unchanged. Parsing, Apple M1 instructions: jeopardy.json 304.0 M -> 293.8 M (-3.4 %), 3000 strings with escapes 1.72 M -> 1.54 M (-10 %), twitter -0.6 %. Linux x86-64 Ir, GCC 13 / Clang 18: escaped strings -7.6 % / -11.4 %, twitter 0.0 % / 0.0 %. Documents and error messages are identical on 900 k random strings of escapes, UTF-8 and control characters (also under ASan), and on mutants of the test corpus. Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Hash the last bytes of a key of a large object without a loop key_hash() mixed a key eight bytes at a time and then packed the remaining 0-7 bytes one by one, in a loop. The last word of a key of eight or more bytes is now read as the last eight bytes of the key (overlapping the word before), and a key of 2-7 bytes as two overlapping words of four or two bytes. A product only moves bits upwards, and the slot is taken from the low bits: the high bits of the last word are folded down before the multiplication, since keys that differ only in their last characters ("user_17", "user_18") would otherwise share slots. The hash is not stored, so its values may change. Looking up every key of an object of 1000 random keys, Apple M1 instructions per 1000 lookups: keys of 5 bytes 175 K -> 93 K, hex strings of 8 bytes 177 K -> 168 K, keys of 12 bytes 196 K -> 186 K Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 5-byte keys -4.7 % / -8.8 %, 12-byte keys -3.3 % / -7.4 %. Building the tables while parsing such objects is cheaper as well (5-byte keys: 13.3 us -> 10.1 us). Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Compare short keys of a large object without a call of memcmp find_indexed() compared the key of a member the table points to with memcmp, a library call for any length. Up to 16 bytes, the bytes are now compared as two overlapping words (the first and the last 8, 4, or 2 bytes), as find_member() already does for the objects without a table; longer keys still use memcmp. Looking up every key of an object of 1000 random keys, Apple M1 instructions per 1000 lookups (with the previous commit): keys of 5 bytes 93 K -> 72 K, hex strings of 8 bytes 168 K -> 148 K, keys of 12 bytes 186 K -> 170 K, keys of 24 bytes 198 K -> 193 K (unchanged, memcmp). Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 5-byte keys -7.2 % / -7.3 %, 12-byte keys -5.9 % / -6.6 %. Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Read an unedited editable document like a read-only one Views of a json_editable_document followed edits at every step: the lookup, the element access, back(), dump() and materialize() were built for nodes that can be moved or linked, though no edit changes the index until the first one (every edit creates the edit state before it touches a node). Until then these now take the read-only code: one check of the document's edit state per call, not per member or per node. After the first edit nothing changes. Views that were made earlier keep working, as the check is made when they are used. Unedited editable document, Apple M1: dump() of twitter.json 1.40 M -> 1.22 M instructions (-12.6 %); lookups unchanged. Linux x86-64 Ir, GCC 13 / Clang 18: dump -1.7 % / -1.0 %, lookups -0.1 % / 0.0 %. Walking with iterators is unchanged, and so is dump() after an edit. Belongs to the PR "json_view: read-side speedups". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Format the numbered test keys of json_view's index with a stream The keys were made with snprintf() and a format from a list, which Flawfinder reports as an error (CWE-134, a format string that is not a literal). A stream now formats them; the 30000 keys of the test are the same as before. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Put the key number of the json_view index test in parentheses astyle formatted i * step after << as a pointer declaration (i* step), which failed the amalgamation check. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the compiler and lint findings of json_view's read side and tests - find() and contains() dispatch on Editable for the member lookup; for documents that cannot be edited both branches were the same call (GCC -Wduplicated-branches) - json_view.hpp includes <cstdio> for the FILE overload (cpplint) - the arena check macro of the builder is on one line (cpplint read it as an if body without braces) - tests: no nested conditional (also MSVC C4018), no useless cast, parentheses around mixed operators, raw string literals, += instead of temporary concatenations, and NOLINT for the fixed seeds and a find() != npos, as elsewhere in the tests Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Grow the decode arena through a function and store its position back on every return slow_string() kept its write position in locals and grew the arena through the macro NLOHMANN_VIEW_ENSURE. The macro is now the always-inline member arena_ensure() (a lambda is not inlined at every call site by clang, which then keeps the position in memory: +11 % instructions on escape-heavy strings), and a guard stores the position back into m_arena_write on every return instead of by hand on the successful one. Instruction counts are unchanged (clang) or 1 % lower (GCC). Also word the complexity of back() as that of at(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-24 07:41:06 +02:00
Niels Lohmann	e9c4fe3cdf	json_view: indented and ensure_ascii dumps through the fast writer (#5889 ) * Write the indented output of json_view's dump() with the fast writer The compact writer of dump() (cursor in a register, fixed-size copies of tokens) served dump(-1) only; every dump(indent), and everything with ensure_ascii that was indented, went through a generic walk that calls put() for each token and put_repeated() (a memset call) for each line break. The fast writer now writes indented output, too, with or without ensure_ascii, and the generic walk is gone. A line break is a copy of a 64-byte ruler (a line break and 63 indentation characters) when the line is at most 64 bytes, and a store plus memset beyond that. The depth is counted only in the indented instantiations (counting it in the compact one cost an editable document 5 % with Clang). The indented walks are out of line (dump_pretty()): inlined into dump() next to the compact ones, they made compact dumps of integer arrays 7 % slower. A decoded string (one with escape sequences) of a damaged image loaded with image_check::bounds can contain invalid UTF-8. As before, the compact dump writes such bytes unchanged, and the indented dump throws type_error.316; its runs are found by the lexer's validating scan (a scalar validation of every sequence made dump(2) of decoded CJK text twice as slow as before). dump(2) instructions per call, macOS arm64 (before -> after): twitter citm canada jeopardy(4k) json 3.75M -> 1.85M 8.28M -> 3.93M 67.1M -> 56.0M 9.56M -> 4.81M editable 3.82M -> 1.84M 8.48M -> 4.02M 68.1M -> 56.2M 9.66M -> 4.75M Linux instructions (GCC 13 / Clang 18), dump(2) and dump(2, ' ', true): twitter -25 % / -30 % -11 % / -11 % citm -29 % / -28 % -24 % / -24 % jeopardy(4k) -25 % / -32 % -20 % / -24 % CJK text with \u escapes -6 % / -11 % -2 % / -2 % The compact dump is unchanged (within +0.4 % on Linux, within noise on the Mac). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Scan the strings of json_view's dump() with ensure_ascii 16 bytes at a time With ensure_ascii, a string of the source that has a byte >= 0x7F was written by a generic path: put() per piece and a validate-and-decode step for every character beyond ASCII. It now goes through a writer with the cursor in a local variable. The part of the string that has nothing to escape is found 16 bytes at a time (SSE2 or NEON; the existing 8-byte scan without vector support) and copied; strings that are ASCII throughout are still decided by the 8-byte check of dump() with ensure_ascii (a 16-byte scan for them was 2-5 % slower for short strings). The two-byte sequences (lead C2..DF) and the three-byte sequences whose continuation bytes may have any value (lead E1..EF except ED) are decoded with a few compares; E0, ED, four-byte sequences, and damaged input go through validate_one_utf8() as before, so damaged images still throw type_error.316. dump(-1, ' ', true) instructions per call, macOS arm64 (develop -> now); the compact dump without ensure_ascii and dumps of ASCII text are unchanged: twitter 5.42M -> 3.57M citm 3.28M -> 3.24M jeopardy(4k) 5.78M -> 5.65M CJK text, raw 66.5M -> 35.3M escaped \u 32.7M -> 17.0M Latin text with accents 11.5M -> 9.4M ASCII text 4.40M -> 4.40M Linux instructions (GCC 13 / Clang 18), dump(-1, ' ', true): twitter -9 % / -33 % CJK text -14 % / -47 % Latin text -8 % / -18 % citm -0.2 % / -0.4 % ASCII text -0.1 % / -0.3 % The compact dump without ensure_ascii is unchanged (canada 0.0 %). The unit test covers every character of the Basic Multilingual Plane, and a character that needs an escape at each position of strings of 0..40 bytes. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Find the end of the plain run of a decoded string 16 bytes at a time with SSE2 The strings of a document that had escape sequences are written in runs without a quote, backslash, or control character; the compact dump found the end of a run 8 bytes at a time with a SWAR test. With SSE2 (x86-64) it now compares 16 bytes per step. NEON keeps the SWAR code: there the mask of a step costs more than it saves on the short runs that are typical here. Instructions of dump(), Linux (GCC 13 / Clang 18): twitter.json 1.99M -> 1.88M (-5.2 %) / 1.97M -> 1.92M (-3.0 %) CJK text with \u escapes 5.98M -> 4.52M (-24.5 %) / 5.30M -> 4.25M (-19.9 %) ASCII text with a few escapes 4.28M -> 4.09M (-4.4 %) / 4.24M -> 4.11M (-3.2 %) jeopardy (4,000 questions) 5.08M -> 4.78M (-5.8 %) / 4.97M -> 4.80M (-3.4 %) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Rename depth_counter's open() and close() to enter() and leave() Flawfinder took them for the file functions open() and close(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the clang-tidy findings in json_view's dump and its tests - the SSE2 loop of the string scan is within NOLINTBEGIN/END (portability-simd-intrinsics), as in to_chars.hpp - depth_counter<false>'s leave() and get() keep the interface of depth_counter<true> instead of being static - write_ascii() compares the room left as a size_t variable - the test's texts are a std::array, and a JSON fragment with escapes is a raw string literal Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move dump_ascii() and dump_pretty() next to dump() Address a review comment: the two dispatchers that dump() calls now follow it directly. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-24 07:38:17 +02:00
dependabot[bot]	1e9222f955	Bump dompurify from 3.4.16 to 3.4.17 in /docs/mkdocs/scripts/mermaid (#5904 ) Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.4.16 to 3.4.17. - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](https://github.com/cure53/DOMPurify/compare/3.4.16...3.4.17) --- updated-dependencies: - dependency-name: dompurify dependency-version: 3.4.17 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-10-24 07:37:42 +02:00
Niels Lohmann	70d1baa57a	Remove a nested duplicate guard from the json_view image test clang-tidy reported the inner #if !defined(JSON_NOEXCEPTION) as redundant (the test case is in the same #if) and a find() != npos, which gets the NOLINT the other tests use. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 18:53:34 +02:00
Niels Lohmann	536672e927	Dispatch on the compile-time conditions of the json_view comparison MSVC (/W4 /WX) reported C4127 (conditional expression is constant) for the template parameter EA and the object type's ordering in an if condition; both are now tag-dispatched. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 09:44:39 +02:00
Niels Lohmann	ffcd392be0	Fix Semgrep findings in Python tooling (#5899 ) - generate_docset.py: use SHA-256 instead of SHA-1 for the cache file name digest of downloaded images. - generate_docset.py: suppress dynamic-urllib-use-detected; the URL scheme is restricted to http(s) right before the call. - generate_natvis.py: suppress direct-use-of-jinja2; the template renders a local XML file with autoescape enabled. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 07:11:17 +02:00
dependabot[bot]	e28506dd86	Bump jsdom from 30.1.2 to 30.1.3 in /docs/mkdocs/scripts/mermaid (#5898 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:54:23 +02:00
dependabot[bot]	03a0595c1e	Bump actions/upload-artifact from 7.0.1 to 7.0.2 (#5897 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:54:07 +02:00
dependabot[bot]	4e72af9d54	Bump step-security/harden-runner from 2.21.1 to 2.22.1 (#5896 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:53:50 +02:00
dependabot[bot]	0f5d07310c	Bump the codeql-action group with 4 updates (#5895 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:53:32 +02:00
Niels Lohmann	0b924f5375	Split and print JSON pointers in one pass (#5887 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:53:15 +02:00
Niels Lohmann	6fa14d8f95	Write binary formats through reserved sink space and number runs (#5885 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:52:58 +02:00
Niels Lohmann	dec0e9b749	Write strings, scalars and indentation in dump() with fewer calls (#5883 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:52:41 +02:00
Niels Lohmann	86d5c56c54	Speed up flatten(), unflatten(), diff() and patch() further (#5886 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:48:12 +02:00
Niels Lohmann	aa0b371f7c	Faster std::istream input: inline get-area path, binary reads, unbuffered streambufs (#5884 ) Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:48:00 +02:00
Niels Lohmann	55cdc93ff8	Keep the byte comparison helpers in compare.hpp Address review comments: bytes_equal() and bytes_less() move from node.hpp to compare.hpp, their only user; key_less() no longer splits the comment of resolve_indices() from its function; and the loop counters of bytes_equal() and all_digits() are scoped to their loops. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-23 06:47:22 +02:00
Niels Lohmann	ace1030107	Fix develop CI: bit_cast ADL ambiguity and artifact-less comment runs (#5894 ) - unit-stl_helpers.cpp: with a std::array argument, an unqualified bit_cast<...>(bytes) also finds std::bit_cast via ADL in C++20 and MSVC 2019 rejects the call as ambiguous (AppVeyor). Qualify the call. - comment_check_amalgamation.yml: skip, rather than crash, when the triggering run has no "pr" artifact (stale queued runs that complete as failed without ever running a job). Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-22 20:22:53 +02:00
Niels Lohmann	db5bd7fbcc	Fix the compiler and lint failures of the json_view comparison - all_digits(): the lambda declares its constant itself; MSVC does not let a lambda without captures read a constexpr local of the function - equal(): the frame's "in document order" index is a namespace-scope constant (document_order); the local constexpr npos in the default member initializers of the local struct crashed GCC 9 and MinGW GCC and was not found by MSVC - keys_differ_early() keeps the first keys in std::arrays (cpplint: variable-length arrays; clang-tidy: C arrays), and the basic_json side's first_keys() is not noexcept (key() may throw) - the numbers of one type are compared in an if chain instead of a nested conditional operator - the dump test avoids C arrays and temporary string concatenations View comparison instructions are unchanged (Apple clang, twitter, citm, canada: -0.1 % to +0.1 %). Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-22 20:13:22 +02:00
Niels Lohmann	89bd5d63e1	Rename names that Flawfinder takes for the file function open() The comparison cursors' open() is now enter(), and a test's open and close strings are head and tail. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-22 13:27:54 +02:00
Niels Lohmann	3fb7d9c710	Check the float tokens of a json_document image without a sort and a scan of every character load(image_check::full) checked every float token of an image by sorting the token ranges, scanning the token character by character, and comparing the digit layout with the node's. Now: - the ranges are sorted only if they are not sorted already (save() writes them in the order of the nodes, so they are); - a token of the shape [-]digits.digits whose digit counts are the ones the node records (nearly all of them) is accepted by checking its digits eight at a time, without the general scan; - the general scan skips runs of digits eight at a time. Damaged tokens are still rejected as before: the check accepts exactly the tokens that the general scan accepts. Instructions of load(image_check::full) (develop -> now): macOS arm64 canada 44.6M -> 30.9M (-31 %), jeopardy 11.3M -> 10.7M, citm 10.4M -> 10.0M, twitter 5.06M -> 4.94M Linux GCC 13 canada -50 %, jeopardy -43 %, twitter -31 %, citm -28 % Linux Clang 18 canada -42 %, jeopardy -48 %, twitter -35 %, citm -32 % image_check::bounds is unchanged (within 0.4 %). Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-22 09:30:44 +02:00
Niels Lohmann	fa8134ae85	Compare json_view values without a vector per container and a basic_json per number operator== of views (view/view, view/json, and editable documents) built two vectors of members for every object it entered, sorted them with stable_sort for every object, and converted each number to a basic_json value. It now: - keeps the members of all open objects in one vector (reserved with geometric growth, so that nested objects do not reallocate), and orders them by insertion sort for up to 16 members, or not at all when the keys ascend already; - compares numbers of one type directly, and builds a basic_json only for numbers of different types; - compares short texts (keys, numbers, most strings) with two overlapping loads (bytes_equal(), in node.hpp); - first walks both documents in lockstep, in the order of their text, which finds most pairs of equal values without the work of the comparison. The walk only enters an object if the comparison would visit its members in the same order (the object type keeps the order, the keys ascend, or all values are scalars), so a difference is never found later than before, and it records the containers it found different, so that the comparison does not walk them again; - rejects values of different types and arrays of different sizes before any of this, and, for an object type that keeps the order, objects whose first (up to four) members have different keys, which ends a comparison that differs near the start without preparing the members. Results are unchanged: numbers compare by value across types, duplicate keys resolve as parse() resolves them, and objects compare in document order for ordered_json and by key otherwise. Instructions of one comparison of a document with an equal copy, macOS arm64 (develop -> now), json_document / ordered_json_document / editable: twitter 16.1M -> 7.0M 17.7M -> 1.5M 16.6M -> 7.3M citm 45.0M -> 3.4M 47.0M -> 3.2M 45.7M -> 4.3M canada 111.4M -> 13.7M 111.4M -> 13.7M 113.3M -> 15.0M jeopardy 26.2M -> 3.7M 27.3M -> 3.7M 26.6M -> 12.1M and of a view with a basic_json value: twitter 11.4M -> 5.9M 11.2M -> 7.3M 11.6M -> 5.9M citm 33.9M -> 9.7M 32.6M -> 9.1M 34.6M -> 9.8M canada 92.9M -> 32.1M 92.9M -> 32.1M 93.9M -> 32.8M and Linux instruction counts (GCC 13 / Clang 18) of the same comparisons: document with an equal copy json_document ordered_json_document editable twitter -49 % / -52 % -89 % / -89 % -48 % / -50 % citm -90 % / -89 % -91 % / -90 % -88 % / -87 % jeopardy (4,000 questions) -85 % / -85 % -86 % / -86 % -46 % / -49 % view with a basic_json value twitter -41 % / -45 % -28 % / -35 % -45 % / -44 % citm -63 % / -65 % -63 % / -66 % -65 % / -64 % jeopardy -39 % / -43 % -14 % / -24 % -44 % / -43 % A sweep that places one difference at about 2,100 positions of eight documents (a token at an even spacing, the first and last tokens, the first, middle and last member of the root removed, one appended, the document wrapped) in five kinds of comparison (view/view of read-only and of editable documents, view/json, json_document, editable/view) finds the same results as before and no placement slower on the Mac (Clang); on Linux no placement is slower than 3 % (Clang and GCC). Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-22 09:30:42 +02:00
Niels Lohmann	2dc1e54856	Compare json_view with yyjson, simdjson, and Boost.JSON (#5627 ) * Compare json_view with yyjson, simdjson, and Boost.JSON tests/benchmarks/json_view/ holds a comparison with other libraries, answering the question users ask when they pick one. It is not built by CMake and not run by CI. - bench_view.cpp: parse, traverse, select and dump of twitter, citm_catalog, canada, jeopardy, a tweet and an RPC request, against json_view, yyjson, simdjson, Boost.JSON and json::parse, all agreeing and run interleaved. - bench_corpus.cpp: parse, traverse and dump of any file list. - bench_edit.cpp: parses, edits through each library's own API, and serializes; all outputs must describe the same value. - compare.py: builds both programs with system or pinned, SHA-256-checked downloads, and writes results with the commit, CPU, OS, compiler and library versions. - README.md and a workflow that runs on demand or via a PR label. Fairness fixes folded in: each engine runs once untimed before each timed round, so the next one no longer pays for the previous one's cleanup; dump is also compared with source numbers against yyjson's raw-number mode; a "simdjson DOM (fresh)" column and a reused-document column for json_view were added; compare.py's paths are absolute; downloads use a .part file and a failed checksum removes the archive. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make the downloads of json_view's compare.py robust Hash the archives in chunks instead of reading them into memory, download with a timeout, extract into a temporary directory that is renamed into place only after success (a half-extracted directory was trusted forever), and split CXX and CC into arguments so that values like 'ccache g++' work. Note at the pins that a SHA-256 mismatch of a GitHub tag archive means that GitHub regenerated it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add a timeout and a concurrency group to the json_view benchmarks Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Check the CSV file and the round count in the json_view benchmarks fopen() of the CSV output was not checked, so an unwritable directory made fprintf() write to a null FILE*; the round count is parsed with strtol and clamped instead of atoi. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-22 08:40:33 +02:00
Niels Lohmann	9855db81f3	Silence macro-to-enum for the endianness macros (#5881 ) #5876 added JSON_LITTLE_ENDIAN_TARGET and JSON_BIG_ENDIAN_TARGET to macro_scope.hpp. They must stay macros because they select code in #if, so suppress the clang-tidy check as simd_scan.hpp already does. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-22 08:40:05 +02:00
Niels Lohmann	533cbfee51	Let generate_natvis.py cover custom ABI tag combinations (#5880 ) Since #5877, the checked-in nlohmann_json.natvis only covers namespaces with up to two ABI tags. The generator gets options to produce a file for other configurations: - --abi-tags TAGS adds the namespaces of one configuration (repeatable) - --all generates every combination of ABI tags - --list-tags prints the tags with the macros that enable them - --output FILE writes somewhere else than the repository root `make natvis NATVIS_ARGS="..."` passes the options through. The default output is unchanged. The options are documented in the generator's README, the debugging page, and FILES.md, and checked in CI. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 22:57:54 +02:00
Niels Lohmann	9bf531436f	Speed up flatten(), unflatten(), diff() and patch() (#5861 ) * Flatten into a std::map by inserting next to the previous entry flatten() added every path with operator[], so each insertion into the std::map of the result searched the tree from the root and compared long shared path prefixes, which took half the time on canada.json. Paths come out in nearly sorted order, so they are now inserted with emplace_hint next to the entry added last, for object types that have it. ordered_map keeps its append without lookup, and other object types keep operator[]. Path segments are also appended in place, without a temporary string for each escaped key or array index. flatten() of canada.json, citm_catalog.json and twitter.json is 1.4x to 1.9x as fast; ordered_json is unchanged (1.05x). The output is unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Unflatten without splitting keys or walking each path from the root unflatten() split every key into a vector of reference tokens, stored the pointer prefixes in a tree to decide which values become arrays, and then looked up every token of every key from the root again. Keys are now checked by a scan that raises the same parse_error.107/108 as split(), skipping the part a key shares with the previous one, which was checked already. The prefixes that denote arrays are kept in a small hash set of key slices; the array/object decision is unchanged. The walk keeps the values along the previous key's path, so only the part of a key that differs from the previous one is looked up. Keys of a flattened object are usually sorted, so that part is short. unflatten() of canada.json, citm_catalog.json and twitter.json is 1.9x to 2.3x as fast; the output and the exceptions are unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Apply plain JSON Patch operations without building a json_pointer patch() built a json_pointer for every operation, which splits the path into a vector of strings, and for "add" and "remove" a second one for the parent. That took 20% to 45% of the time not spent copying the document. "add", "remove", "replace" and "test" with a path that has no escapes and denotes existing values (the parent, for "add"), as diff() creates them, are now applied by resolving the path in place, reusing one string for the reference tokens. Every other operation, and every operation that would fail, takes the existing code, so the results, the exceptions and the state left after a failing operation are unchanged. patch() of the diffs of canada.json, citm_catalog.json and twitter.json is 1.02x to 1.10x as fast; the rest is the copy of the document. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Build diff operations member by member diff() created every operation from an initializer list, so each {"op", "replace"} pair became a temporary array with two heap strings before it was turned into an object member; an operation took about 25 allocations. The operations are now built as objects whose members are assigned one by one. Paths of members and elements are also built in place, without a temporary string for each escaped key or array index. diff() of canada.json, citm_catalog.json and twitter.json, with every eighth scalar changed, is 1.3x to 1.6x as fast; the output is unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Diff objects with the same keys in lockstep and keep one path buffer diff() looked up every key of the source object in the target object, and built a new string for the path of every value it descended into. Objects of the same size usually have the same keys in the same order, so they are now walked together as long as that holds, without a lookup; objects that differ fall back to the lookup from the first key that does not match. The path is one buffer that is extended before descending and shortened again afterwards. Objects whose members are equal but in a different order (possible with ordered_json) still get no operations. Together with the previous commit, diff() of canada.json, citm_catalog.json and twitter.json is 1.5x to 2.0x as fast; ordered_json 1.65x to 1.85x. The output is unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use std::array for the index buffer in append_index() Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Declare the partner iterator in the loop initializer in diff_object_keys() Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix GCC and clang-tidy warnings in the JSON pointer speedups Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 21:27:33 +02:00
Niels Lohmann	3e031b06a5	Use standard library helpers where available, with C++11 fallbacks (#5876 ) * Use standard library helpers where available, with C++11 fallbacks Add detail helpers that call the standard library function when the library provides it (checked with the __cpp_lib_* feature-test macros, so no new ABI-tag macros) and a C++11 implementation otherwise: - detail::clamp (std::clamp): by value, so that static constexpr members need no out-of-line definition before C++17; used in json_view's dump() buffer sizing, where step() can now be a static constexpr member. - detail::bit_cast (std::bit_cast): replaces dtoa_impl::reinterpret_bits and the memcpy bit copies in the number parser, to_chars, the json_view serializer, and binary_writer::to_char_type. - detail::byteswap (std::byteswap): one implementation instead of the binary writer's own byte_swap() overloads and the bswap chain in to_chars' store_msb_first(). - count_leading_zeros/count_trailing_zeros: two 32-bit _BitScanReverse/ _BitScanForward calls for MSVC x86/ARM32 instead of the loop, and std::countl_zero/countr_zero for compilers without builtins. - conjunction, disjunction, negation, bool_constant, and uncvref_t map to their std counterparts in C++17/C++20. Builtins and intrinsics stay ahead of the <bit> functions, as MSVC's std::countl_zero checks for LZCNT at run time unless AVX2 is enabled. The generated code is unchanged with Clang (C++11/20/23, arm64 and x86-64) and GCC 16 for parsing, dumping, and the binary formats. The binary reader keeps its own byte_swap() and the writer its memcpy calls: the smaller code from detail::byteswap/std::bit_cast there made GCC 16 inline get_number()/write_number() differently, which slowed reading by 10-18 % and writing by about 1 %. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Name the target byte order once (JSON_LITTLE_ENDIAN_TARGET, JSON_BIG_ENDIAN_TARGET) to_chars.hpp and bit_ops.hpp repeated the long __BYTE_ORDER__/_MSC_VER conditions; macro_scope.hpp now defines them once (undefined again in macro_unscope.hpp). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Explain why the serializer growth bound is not a clamp Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 20:14:59 +02:00
Niels Lohmann	7c26e99ac5	Copy ASCII strings in the view's dump with ensure_ascii (#5879 ) With ensure_ascii, the view's dump wrote every string through the escaping path, which decodes UTF-8 byte by byte, even strings of the source that are plain ASCII. A string of the source without escape sequences has no quote, backslash, or control character, so ensure_ascii leaves it unchanged exactly when it has no byte >= 0x7F. The dump now checks that eight bytes at a time (one add and one or per word) and copies such strings like the dump without ensure_ascii. The compact walk (dump_compact) now also serves ensure_ascii, through a separate out-of-line instantiation, so the walk without ensure_ascii keeps its speed. Parsing is unchanged. A flag set by the parser would let the dump skip the check and was about twice as fast for compact output, but it costs every parse of string-heavy input 1-2% with GCC. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 20:01:38 +02:00
Niels Lohmann	6aefd8c662	Copy short printable-ASCII strings without starting the scanner (#5878 ) dump() now checks strings of up to 32 bytes against a table of the bytes that can be written verbatim, four bytes per step, and copies them in one go if all of them can. That is cheaper than setting up the vector scanner, whose setup costs more than such a string takes to scan; SQLite's jsonAppendString() writes strings with the same kind of table loop. Most object keys and many values are this short (citm_catalog averages 8 bytes per string, jeopardy 14). GCC 16 and 13 stopped inlining put_indent() and put_literal() into the pretty-printing loops once dump_escaped() had grown, which made indented output up to 13% slower; both are now always inlined. The new test puts every ASCII character and a 2-byte character at every position of strings of 1 to 40 bytes and compares the output with the reference implementation of the existing UTF-8 tests. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 20:01:28 +02:00
Niels Lohmann	52fecc1fab	Move object keys into the value when parsing (#5855 ) json_sax_dom_parser::key() copied the key from the lexer's buffer into the object. The lexer starts every token with an empty buffer (reset()), so the key can be moved instead, which saves an allocation and a copy for every key that does not fit into the string's inline buffer. json_sax_dom_parser already moved string values the same way. ordered_map has no rvalue operator[], so ordered_json still copies keys. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 20:00:51 +02:00
Niels Lohmann	fa37b952df	Match literals without a stack array and keep parser states in bytes (#5854 ) * Match literals without a stack array and keep parser states in bytes scan_after_whitespace() built the expected text of true, false, and null in local std::array objects. With -fstack-protector-strong (the default of Ubuntu's GCC, among others), a local array gives the function a stack canary check, which then ran on every token. scan_literal() now takes the text after the first character as a string literal, with its length as a template parameter; for contiguous input it compares the rest at once and falls back to the byte loop on a mismatch, so errors are reported at the same character. It is out of line, which keeps the token dispatch small. The parser kept the nesting of arrays and objects in a std::vector<bool>, whose push_back() GCC calls out of line; it is a std::vector<std::uint8_t> now. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Skip the literal fast path test without exceptions The test case compares the error messages of both lexer paths, so it does not compile with JSON_NOEXCEPTION (the caught exception does not exist). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Allow the string literal parameter of scan_literal() for clang-tidy The parameter takes the literal by array reference so that its length is a template parameter; this is the same exemption the C array overloads of to_json() and from_json() have. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 20:00:51 +02:00
Niels Lohmann	e9970066c5	Scan strings with SSE2, SSSE3, or NEON (#5853 ) * Scan strings with SSE2, SSSE3, or NEON The string fast path found the end of a run of plain characters 8 bytes at a time (SWAR) and validated non-ASCII text one sequence at a time, calling validate_one_utf8() out of line for each. The vector code of json_view's scanner, in the new shared header detail/input/simd_scan.hpp, now does both 16 bytes at a time: SSE2 finds the end of a run on x86-64, and the UTF-8 check of Keiser and Lemire runs with NEON on AArch64 and with SSSE3 on x86-64. SSSE3 is not part of x86-64, so the check is compiled for it with a function attribute and used when the CPU has it (cached in a constant-initialized atomic), else the scalar check; code compiled with -mssse3 skips the CPU check. The first word of a run is still checked with SWAR (the first four with NEON, where a vector mask costs more), as most keys and short values end there, and the scalar helpers are always inlined. On anything else, the scalar code is unchanged. The new macro JSON_NO_SIMD selects the portable code everywhere, including the SSE2/NEON digit formatting of the floating-point writer. The lexer tests now also run with JSON_NO_SIMD and, on x86-64, with -mssse3, and a new test places every byte that ends a run at each offset across several 16-byte blocks, after ASCII and after CJK text. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the string scan from slowing down short strings and dump() on Arm Three benchmarks were 7-9% slower than develop on Apple M1 (Apple Clang, -O3 -flto): Accept/jeopardy, Dump/citm_catalog, and Dump/jeopardy. - dump(): the serializer passes a string without its closing quote, so plain_run_end() ran out of input and called the out-of-line NEON loop for every string, with fewer than 16 bytes left. The loop is now only called for 16 bytes or more. - accept(): no string of jeopardy.json reaches the vector UTF-8 check, yet compiling it into the lexer's scan function made it 5% slower: the function runs once per token and now saved more registers and kept a stack slot. lexer_string_bulk_run() checks the first four words inline and continues out of line, and the serializer keeps string_bulk_run() inline, which is faster there. - The byte that ends a run is read from the scanned buffer instead of through get(), which Apple Clang merged with the other get() calls that load the read position again right after it was stored (3%). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Define string_bulk_run_from() only where lexer_string_bulk_run() calls it With NEON and simdutf in C++17, lexer_string_bulk_run() takes the string_bulk_run() path, so the out-of-line helper was unused. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not dispatch the SSSE3 string scan with GCC's C++20 modules GCC does not keep the target("ssse3") attribute on functions it reads back from a module interface, so the module build (ci_module_cpp20, gcc:latest) rejected the intrinsics in vector_utf8_run() with "needs isa option -mssse3". With __cpp_modules, GCC now uses the portable UTF-8 check like other compilers without the target attribute; the SSE2 scan stays, and -mssse3 or more still compiles the SSSE3 check directly. Clang is not affected. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Include <cpuid.h> only once when json.hpp and json_view.hpp are both used The SSSE3 dispatch of the string scan and the one of json_view each include <cpuid.h>. Before GCC 11 the header has no include guard, so a translation unit that includes both headers fails with GCC 10: "redefinition of 'unsigned int __get_cpuid_max(unsigned int, unsigned int)'". Guard both includes with NLOHMANN_JSON_CPUID_INCLUDED. Signed-off-by: Niels Lohmann <mail@nlohmann.me> Add an ABI tag for JSON_NO_SIMD JSON_NO_SIMD selects between two definitions of the internal scanning and formatting functions. Like JSON_VIEW_NO_SIMD (_vns), it now appends _ns to the inline namespace, so translation units built with and without it can be linked into one program without ODR violations. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 20:00:50 +02:00
Niels LohmannandKartikey	3f8677d028	Limit Natvis namespace variants to two ABI tags (#5877 ) The generator wrote a namespace entry for every combination of ABI tags, so nlohmann_json.natvis doubled with each new tag. With the 12 tags on develop it had reached 57.9 MB, and one more tag would push it past GitHub's 100 MB file limit. It now covers the default namespace, every single tag, and every pair of tags (versioned and unversioned), which is 158 entries / 1 MB with 12 tags. Taken over from #5868. Refs #5863. Signed-off-by: Kartikey <kg704925@gmail.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Kartikey <kg704925@gmail.com>	2026-10-21 18:26:07 +02:00
Niels Lohmann	ef3d350639	Split the Visual Studio 2019 AppVeyor job into two shards (#5872 ) The Visual Studio 2019 job (Release, x86, JSON_ImplicitConversions=OFF) builds the tests for all standards and takes 49 to 57 minutes when it passes, close to AppVeyor's 60-minute limit per job. In build 54917118 (#5867), build 54917312 (#5620), and build 54930878 (#5851) it was stopped at the limit while still compiling. Split it with JSON_TestShard like the Visual Studio 2017 jobs. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 16:25:04 +02:00
Niels Lohmann	5e13f834c1	Copy the text of float tokens only when a SAX handler asks for it (#5852 ) After the digits are converted while scanning, copying the token text into token_buffer (an out-of-line std::string::append) was about 10% of parsing canada.json. The only reader is the string argument of json_sax::number_float(), and the library's own handlers (json_sax_dom_parser, json_sax_dom_callback_parser, json_sax_acceptor) ignore it. For contiguous input, the fast path now keeps a pointer to the token in the input, and get_string() copies it on first use. The parser passes an empty string to the handlers marked by the new internal trait sax_ignores_number_text instead of calling get_string(), so json::parse() and json::accept() never copy float text. User handlers, and JSON_DIAGNOSTIC_POSITIONS (which reads get_string().size()), still see the exact text. Keys and strings, which never have pending text, use the unchecked get_string_token(). Input read through a window into a stream (std::istream, FILE*) copies the text right away, because the next read may refill the window. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 12:26:46 +02:00
Niels Lohmann	e13e976801	Accumulate number digits while scanning them (#5851 ) * Accumulate number digits while scanning them The contiguous number fast path validated a token byte by byte, copied it into token_buffer, and then converted it in out-of-line functions that scanned the digits a second time: parse_float_native() called parse_float_significand(), which returned its result through memory and called std::any_of (out of line with GCC) twice per number, and parse_integer_signed() divided by ten on every digit for its overflow check. scan_number_bulk_contiguous() now accumulates the significand while it validates the digits, eight at a time where at least eight bytes remain (is_eight_digits() and parse_eight_digits(), after fast_float). Floats with at most 19 digits go straight to decimal_to_float(); integers with at most 18 digits are stored without an overflow check (a narrower number type still rejects values that do not round-trip). Longer tokens take the existing path. has_nonzero_digit() is a plain loop and parse_float_significand() is always inlined, which helps that path. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Skip the digit accumulation checks without exceptions The section compares error messages from both lexer paths and parses out-of-range numbers, so it does not compile (the caught exception does not exist) and would abort with JSON_NOEXCEPTION. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Share the significant-digit loop of the integer part and the fraction Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 12:26:46 +02:00
Niels Lohmann	898e6ee81d	Add benchmarks for SAX parsing, value operations, JSON Pointer, and conversions (#5845 ) Squashed onto develop `cd6b428ef` (CI fixes from #5860). Original commits: - Add benchmarks for SAX parsing, value operations, JSON Pointer, and conversions - Time copies together with their destruction in the benchmarks - Time destroying the json in ToJsonStrings Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 12:25:42 +02:00
Niels Lohmann	ef1cdb59ac	Update performance docs for fast std::istream and FILE* parsing (#5875 ) performance.md still claimed streams have no fast path (outdated since #5828). Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 12:25:10 +02:00
Niels Lohmann	bd5227c2ec	Fix silent wrong results for custom number types (#5805 ) * Fix silent wrong results for custom number types Squashed onto develop `cd6b428ef` (CI fixes from #5860). Original commits: - Fix silent wrong results for custom number types - Fix CI: build the out_of_range.407 message from string_t and fix a link - Fix CI: diagnostics message, clang-tidy, and MinGW section limit - Format the out_of_range.407 number without dump() - Move the std::deque array test out of unit-regression3.cpp - Fix clang-tidy and clang 3.4/3.5 failures in the custom number type changes - Fix CI: cast the 128-bit unsigned value explicitly in the integer fast path Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Name the range checks of integer_fits_max/min Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 08:36:31 +00:00
Niels Lohmann	0cbe95ffa2	Speed up writing CBOR and MessagePack, above all integer arrays (#5824 ) * Speed up writing CBOR and MessagePack, above all integer arrays Squashed onto develop `cd6b428ef` (CI fixes from #5860). Original commits: - Append to the output vector through a position-tracking sink - Write CBOR integers without branching on their sign or size - Write MessagePack integers without branching on their sign or size - Use std::array for the integer size class table - Fix clang-tidy and GCC warnings in the integer array test - Regenerate the amalgamation Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Convert the streamed bytes explicitly in the binary writer sink tests Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-10-21 10:23:48 +02:00

1 2 3 4 5 ...