mirror of
https://github.com/nlohmann/json.git
synced 2026-11-16 19:20:42 +00:00
b0a4218e384263d285171dc549e97042e0762f63
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
17e3de0fcc |
docs: add version-history notes for fixed start_pos and error positions (#6089)
Two bugs of 3.12.0 are fixed on develop without a note on the pages. With JSON_DIAGNOSTIC_POSITIONS, start_pos() of a string that contains escape sequences was too large; the position is now that of the opening quote. A parse error for an unexpected number that is followed by a line feed reported column 0 (for "1 2" and a line feed, line 1, column 0); for "1", line feed, "2", line feed it now reports line 2, column 1. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
|
33f15727ce |
Parse JSON in one block of memory with a fused pointer-based parser (#6031)
* Parse JSON in one block of memory with a fused pointer-based parser The parser and the lexer exchange one token at a time, and the lexer keeps its state (the current character, a pending unget, where the token starts) in members, which every character pays for: json::accept() of twitter.json took 2.8x as long as json_document::accept(). detail::contiguous_parser reads text in one block of memory through a local pointer, with the grammar, the whitespace and the tokens in one loop (after yyjson's reader), and drives a SAX handler: parse()'s DOM handler, accept()'s handler, or the user's handler of sax_parse(). It is used with the default options (no callback, no comments, no trailing commas; sax_parse() with strict = true). Errors are reported by the library parser, so the exception, its message and its position stay the same. contiguous_parser stops at the token that the library parser would reject, before reporting it, so the events up to the error are the same; then: - parse() runs the library parser with a handler that only reports the error (json_sax_error_reporter), with the lexer of parse(), which is instantiated anyway; - sax_parse() runs it with a handler that forwards only parse_error() to the user's handler (json_sax_error_forwarder), so no event arrives twice; - accept() returns the verdict. A handler that returns false stops the parse as before. An error at the end of a large input costs parse() about 1.2x of what it cost before. On an Apple M1 Max: accept() 1.6-2.0x as fast, parse() 10-16% faster, ordered_json 11-16%, sax_parse() with a user's handler 1.36-1.46x, small messages 12-22% (parse()) and 2.1-2.4x (accept()). On x86-64 (GCC 15, Clang 21): accept() 1.6-1.9x, parse() 5-17%, ordered_json 12-19%, sax_parse() 1.32-1.57x. A translation unit that parses a string compiles 3% (-O0) and 9% (-O2) slower. unit-contiguous_parser.cpp compares the contiguous path with the byte path (std::list iterators) on corner cases, generated documents, their truncations and mutations, deep nesting, and narrow number types: the same values with the same number types and float bits, exception messages, accept() verdicts and sax_parse() events. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix CI: unused parameter without exceptions, cpplint, BUILD.bazel - json_sax_error_reporter::parse_error: mark the exception unused, as JSON_THROW does not use it with JSON_NOEXCEPTION - unit-contiguous_parser.cpp: compare the exceptions of both paths only when exceptions are enabled (invalid input aborted the test otherwise) - silence cpplint's readability/braces false positives on the if statements inside the parser's macros - add contiguous_parser.hpp to BUILD.bazel Signed-off-by: Niels Lohmann <mail@nlohmann.me> * accept(): skip digits without computing their value accept() needs the value of a number only for a float whose exponent may reach the largest value. skip_digits() finds the end of the digits without the multiply-adds of accumulate_digits(), and once an 8-byte word fails the digit test, the remaining loop needs no bounds check: a non-digit is in that word. The value is computed afterwards for the rare floats that need it. accept() of canada.json: -16 % on Apple M1, -22 % with GCC 15 and -21 % with Clang 21 on x86-64; files with many numbers -9 to -15 %. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Tests: mark parser callback in unit-contiguous_parser noexcept Fixes the -Werror=noexcept failure with GCC 26 (ci_test_standards_gcc (26)). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Tests: sweep_allocator::construct() accepts pointers to const Recent libc++ (emsdk latest) probes construct(const T*, const T&) in an unevaluated context while checking for trivial copies; the static_cast to void* in the noexcept specification then fails to compile. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz (cherry picked from commit 794bf70823c12a2a1cb2a8a95f34dfb6357746a8) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Tests: guard sweep_allocator::construct() against const types instead of const_cast The const_cast added in the previous commit is rejected by cppcoreguidelines-pro-type-const-cast (clang-tidy). Make construct() not viable for const U instead, which is what libc++ probes for. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz (cherry picked from commit de099fa607001b710862fd1bbecef3609c54cbc4) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix CI: astyle and clang-tidy for sweep_allocator::construct() astyle 3.4.13 reformats the enable_if<!...> default template argument (Check amalgamation). Delete the const overload instead, which makes the libc++ probe for const objects fail in the SFINAE context just the same and needs no <type_traits>. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Contiguous parser: review fixes (wording, macros, diagnostic positions) - "default parser" instead of "library parser" in comments and tests - the three comments in json.hpp say the same thing - current() replaces the repeated `p != end ? *p : 0` - the backslashes of each macro are aligned - the docs say that JSON_DIAGNOSTIC_POSITIONS turns the fast path off Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
|
ff0bdfdef3 |
Keep the byte range in exceptions about the root value (#5990)
* Keep the byte range in exceptions about the root value (#5987) With JSON_DIAGNOSTICS and JSON_DIAGNOSTIC_POSITIONS both set, exception::diagnostics() and the json_view diagnostics() returned early for a value without a path (the root), dropping "(bytes a-b)". Build the "(path) " prefix only when there are tokens and always append the byte range. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Guard the null value of a value-initialized iterator; drop the unneeded found flag With the root's byte range, diagnostics() reaches get_byte_positions() also with JSON_DIAGNOSTICS when there is no value, as for a value-initialized iterator (invalid_iterator.212). get_byte_positions() now returns no range for a null value, the same change as #6003, so either can be merged first. In the view, a value that value_path() does not find leaves no tokens, so the found flag added nothing to the tokens.empty() test. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
|
54f6b3ef06 |
json_view: source positions in materialize() with JSON_DIAGNOSTIC_POSITIONS (#5945)
* json_view: source positions in materialize() with JSON_DIAGNOSTIC_POSITIONS With JSON_DIAGNOSTIC_POSITIONS, materialize() now gives every value the start_pos()/end_pos() that basic_json::parse() would give it, so exceptions about materialized values name the same bytes (#5864). - json_sax_dom_parser takes the lexer type as a defaulted template parameter; materialize() passes position_source, which reports the positions it derives from the index. diagnostic_positions treats npos as "no position" and keeps ranges all-or-nothing. - Strings with escapes keep the offset of their opening quote plus one in `next` (only with the macro); materialize() scans from there to the closing quote. The end of an array or object is found by scanning from the end of its last element over whitespace, comments and a trailing comma. - In editable documents, values written by edits, arrays and objects whose elements changed, and containers whose last element has no position get none. Images keep the flag is_new, and load() checks that `next` of a string is within the text. Nothing changes on the parse path without the macro. The view's own exceptions still carry no byte range, so the tests that compare them with those of materialized values skip it under the macro, as they skip the path under JSON_DIAGNOSTICS. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Indent comment in unit-json_view_from_json.cpp Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review: clearer position code and docs - position_source gets clear(); set_scalar() returns early for new values, switches on node_kind, and uses no local copies; the one position_source per materialize() call is documented - no npos + 1 for containers without a position; a comment on why the end of a container is reported as its closing bracket - less duplication under #if in materialize() and the builder - diagnostic_positions sets npos explicitly - docs: positions of basic_json values after changes, positions in images depend on how the document was parsed, when next is set, and the cost of positions while parsing Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Share the comment scanners of the view builder with materialize() skip_line_comment() and find_block_comment_end() in scan.hpp are used by the builder's comment() and by position_source::container_end(), which had its own copy. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review: if/else for container positions, clearer next field - the opening bracket of a container sets positions with an if/else instead of a ternary, and the frame takes its start from positions - the closing bracket of a container is found in nested if/else; the parent's last_end is taken from positions, so it is npos whenever no range is set - architecture.md: the next field reads "or 0 if it was not; all other values: 0" instead of "else 0; otherwise 0" Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix GCC unused-result and clang-tidy raw-string findings in unit-json_view_positions.cpp Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
|
a47170346a |
docs: new design for the documentation site (#5934)
* docs: new design for the documentation site Apply the brand to the Material for MkDocs site: Inter and JetBrains Mono served by the site (upright and real italic), brand colours, blue header band with the syntax pattern, wordmark in the header, new home page (hero, cards, animated feature tour), 404 page, polish for tabs, search, footer, focus rings, tables and code blocks, and theme features (code copy, previous/next footer, TOC follow, search suggestions). Draw the number railroad diagram, the parser callback events and the iterator illustrations as inline SVGs that follow the light and dark scheme. The generator scripts are in docs/mkdocs/scripts/figures; the feature slideshow (json-light.svg, json-dark.svg, replacing json.gif on the home page) is built by docs/mkdocs/scripts/slideshow. Decision nodes in five flowcharts become hexagons, which need far less room. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: add figures, table captions and a macro chooser - Figures with numbered captions: SAX stepper, json_view layout and edit stepper, exception map, JSON pointer and patch walkers, QA pipeline, scaling and binary size charts, roadmap lanes, release timeline, integration decision tree and namespace builder - Table captions (nj-table-caption) on 9 pages - Compile times: modules and PCH charts, filesystem section, re-measured Figure 1, links between features/modules.md and compile_times.md - Interactive macro chooser for features/arbitrary_types.md - Link check: exclude URLs that answer the plugin with 400/401/419 or are dead (customers.md) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: macro chooser generates adl_serializer for non-default-constructible types Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: cross-link concept pages and API pages in See also sections Add See also links in both directions between the concept pages (features, home, integration) and the API pages, write API links in code font, give every entry a comment after a dash, and show the callback event figure on the parser callbacks page. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: address review comments on the new design - Home: add a "Latest release" button - Sponsors: update the list and group it by tier - Features: list all feature pages on the index page - Releases: add highlights per release, link the 4.0 roadmap, add the first commit to the timeline - Exceptions: prefix all example messages with the exception name - API: no code font for format names in the See also links of the to_/from_ pages - Merge Patch: show the document before, the patch, and the result - Replace "--" with typographic dashes, mark external links with an icon, center Mermaid diagrams Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
|
63c10a51fc |
Review and extend the documentation, and check it in CI (#5638)
* Review and extend the documentation, and check it in CI A review of all documentation pages found factual errors, dead links, missing cross-references, and gaps in examples. This fixes them and adds checks so the same problems are caught automatically. Fixes: - wrong signatures and version histories (operator!= C++20 member, binary() subtype type, get<PointerType>(), JSON_NO_THREAD_LOCAL, ...) - stale descriptions (number parsing since #5283, UBJSON table, SAX example that no longer compiled, tsl::ordered_map advice) - dead internal and external links; repology.org badges (the domain is suspended) replaced by badges that query the registries directly - deprecation notes link the migration guide; the guide itself fixed Additions: - "See also" sections, cross-references, 25 runnable examples, 12 Mermaid diagrams, new API pages for json_pointer::operator<=> and byte_container_with_subtype::operator==/!= - landing page, guides for untrusted input and performance - "unreleased" badge after versions newer than the latest release Checks: - strict documentation build (broken links/anchors fail it); CI and the publish workflow fetch the full history the build needs - weekly external link check, Mermaid syntax check in CI - check_structure.py: example titles, heading levels, alt texts, header links, docset index coverage; its unused-example check works again - all examples produce the same output on every platform Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the customer links that could not be fixed A dead link on the customers page is still the evidence of where the use of the library was documented. Keep the original URLs of the entries without a working replacement (Marne, Cisco Webex Desk Camera, Philips Hue, CyberArk) and exclude exactly these URLs from the link check. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Correct the duplicate-key recipe's claim about SAX positions The SAX interface's key() receives no position either; only parse_error() does. Also note that the recipe does not report the path to the repeated key (see discussion #5085). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Say the library is available as a single header and mention json_fwd.hpp Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Correct documentation errors found while hunting for bugs - patch/patch_inplace: list the JSON pointer errors parse_error.106-109 and out_of_range.402/404, and quote the actual parse_error.105 message. - unflatten: list parse_error.106/107/108 and out_of_range.404. - to_bson: list out_of_range.415 (binary subtype above 255) and note that 412 and 415 are new in 3.13.0. - to_string: state that string_t must be convertible to std::string, also in the StringType requirements table. - JSON Lines: a `while (input >> j)` loop also throws after the last value for concatenated JSON values; show a loop that works for both. - BON8: a string gets 0xFF only if nothing follows it in the message; a string at the end of an array or object is ended by 0xFE. - custom_string_type.hpp: add operator+=(char), which the "Always required" list asks for (json_pointer::to_string, flatten, unflatten, and diff did not compile), and an ADL int_to_string for diff and items. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Cache the release headers with functools.lru_cache Codacy (Pylint) flagged the mutable default argument that header() used as its cache. functools.lru_cache keeps the same memoization without it. The script's output is unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
|
4424a0fcc1 |
📝 update documentation (#4723)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
|
8fb5d6f92d |
Some documentation updates (#4636)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
|
d23291ba26 |
use diagnostic positions in exceptions (#4585)
* add a ci step for Json_Diagnostic_Positions Signed-off-by: Harinath Nampally <harinath922@gmail.com> * Update ci.cmake to address review comments Signed-off-by: Harinath Nampally <harinath922@gmail.com> * address review comment Signed-off-by: Harinath Nampally <harinath922@gmail.com> * fix typo in the comment Signed-off-by: Harinath Nampally <harinath922@gmail.com> * fix typos in ci.cmake Signed-off-by: Harinath Nampally <harinath922@gmail.com> * invoke the new ci step from ubuntu.yml Signed-off-by: Harinath Nampally <harinath922@gmail.com> * issue4561 - use diagnostic positions for exceptions Signed-off-by: Harinath Nampally <harinath922@gmail.com> * fix ci_test_documentation check Signed-off-by: Harinath Nampally <harinath922@gmail.com> * address review comments Signed-off-by: Harinath Nampally <harinath922@gmail.com> * fix ci check failures for unit-diagnostic-postions.cpp Signed-off-by: Harinath Nampally <harinath922@gmail.com> * improvements based on review comments Signed-off-by: Harinath Nampally <harinath922@gmail.com> * fix const correctness string Signed-off-by: Harinath Nampally <harinath922@gmail.com> * further refinements based on reviews Signed-off-by: Harinath Nampally <harinath922@gmail.com> * add one more test case for full coverage Signed-off-by: Harinath Nampally <harinath922@gmail.com> * ci check fix - add const Signed-off-by: Harinath Nampally <harinath922@gmail.com> * add unit tests for json_diagnostic_postions only Signed-off-by: Harinath Nampally <harinath922@gmail.com> * fix ci_test_diagnostics Signed-off-by: Harinath Nampally <harinath922@gmail.com> * fix ci_test_build_documentation check Signed-off-by: Harinath Nampally <harinath922@gmail.com> --------- Signed-off-by: Harinath Nampally <harinath922@gmail.com> |
||
|
|
26cfec34be | Clean up and document project files (#4560) | ||
|
|
58f5f25968 |
json start/end position implementation (#4517)
* Add implementation to retrieve start and end positions of json during parse * Add more unit tests and add start/stop parsing for arrays * Add raw value for all types * Add more tests and fix compiler warning * Amalgamate * Fix CLang GCC warnings * Fix error in build * Style using astyle 3.1 * Fix whitespace changes * revert * more whitespace reverts * Address PR comments * Fix failing issues * More whitespace reverts * Address remaining PR comments * Address comments * Switch to using custom base class instead of default basic_json * Adding a basic using for a json using the new base class. Also address PR comments and fix CI failures * Address decltype comments * Diagnostic positions macro (#4) Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com> * Fix missed include deletion * Add docs and address other PR comments (#5) * Add docs and address other PR comments --------- Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com> * Address new PR comments and fix CI tests for documentation * Update documentation based on feedback (#6) --------- Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com> * Address std::size_t and other comments * Fix new CI issues * Fix lcov * Improve lcov case with update to handle_diagnostic_positions call for discarded values * Fix indentation of LCOV_EXCL_STOP comments * fix amalgamation astyle issue --------- Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com> |