Compare commits

...
78 Commits
Author SHA1 Message Date
Niels Lohmann 2fb2be036c Check the CSV file and the round count in the json_view benchmarks
fopen() of the CSV output was not checked, so an unwritable directory made
fprintf() write to a null FILE*; the round count is parsed with strtol and
clamped instead of atoi.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:56 +02:00
Niels Lohmann 91125c26a3 Add a timeout and a concurrency group to the json_view benchmarks
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:55 +02:00
Niels Lohmann ffedc9dcb6 Make the downloads of json_view's compare.py robust
Hash the archives in chunks instead of reading them into memory, download
with a timeout, extract into a temporary directory that is renamed into
place only after success (a half-extracted directory was trusted forever),
and split CXX and CC into arguments so that values like 'ccache g++' work.
Note at the pins that a SHA-256 mismatch of a GitHub tag archive means
that GitHub regenerated it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:54 +02:00
Niels Lohmann 2188623289 Compare json_view with yyjson, simdjson, and Boost.JSON
tests/benchmarks/json_view/ holds a comparison with other
libraries, answering the question users ask when they pick one.
It is not built by CMake and not run by CI.

- bench_view.cpp: parse, traverse, select and dump of twitter,
  citm_catalog, canada, jeopardy, a tweet and an RPC request,
  against json_view, yyjson, simdjson, Boost.JSON and json::parse,
  all agreeing and run interleaved.
- bench_corpus.cpp: parse, traverse and dump of any file list.
- bench_edit.cpp: parses, edits through each library's own API,
  and serializes; all outputs must describe the same value.
- compare.py: builds both programs with system or pinned,
  SHA-256-checked downloads, and writes results with the commit,
  CPU, OS, compiler and library versions.
- README.md and a workflow that runs on demand or via a PR label.

Fairness fixes folded in: each engine runs once untimed before
each timed round, so the next one no longer pays for the
previous one's cleanup; dump is also compared with source numbers
against yyjson's raw-number mode; a "simdjson DOM (fresh)" column
and a reused-document column for json_view were added;
compare.py's paths are absolute; downloads use a .part file and a
failed checksum removes the archive.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:53 +02:00
Niels Lohmann 765b9dd790 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:51 +02:00
Niels Lohmann 7985a0b466 Move the dump tests of this branch to unit-json_view_dump.cpp
The dump and comparison tests moved to their own file in the dump pull
request; the tests of the fast dump() join them there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:50 +02:00
Niels Lohmann 4981016097 Split the value and pointer edit tests off unit-json_view_edit.cpp
The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view_edit.cpp has 35409 sections since the fast
dump instantiates more code for editable documents, so its test cases "views
and values", "deeply nested values", "pointers below a null value", and
"strings of other documents are checked" move into
unit-json_view_edit_values.cpp. The helper exception_of_call() that both
files use moves into json_view_test_helpers.hpp.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:49 +02:00
Niels Lohmann 720874a550 Write doubles from their bits only where double is IEEE 754 binary64
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:49 +02:00
Niels Lohmann cf22dc3e7c Fix clang-tidy 22 findings in unit-to_chars.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:48 +02:00
Niels Lohmann 76bb7407f9 Test write_short_decimal and powers_of_ten_16 against to_chars
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:47 +02:00
Niels Lohmann 68e9b48a7c Use a named document in the dump test: root() of a temporary does not compile
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:46 +02:00
Niels Lohmann 6d8e3dbf08 Tidy the view serializer: doxygen block placement, enabled() for a constant
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:45 +02:00
Niels Lohmann 15a865daf2 Fix out-of-bounds read when dumping a float token of an unchecked image
A float node of an image loaded with image_check::bounds can hold a
token whose bytes are not digits, so its value need not have the digit
count recorded in the node. write_double_at() passed that count to
write_short_decimal(), which indexes its table of powers of ten by it:
an assertion failure in debug builds, an out-of-bounds read in release
builds. Use the counted overload only if the value has exactly that many
digits, otherwise count them.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:44 +02:00
Niels Lohmann c53842f7ae Fix MSVC C4127 and Infer finding in the view serializer
Test the template options through enabled() so that conditions combined
with them are not constant, and mark the nav::value() null dereference
as a false positive like materialize.hpp does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:43 +02:00
Niels Lohmann 860a214322 Write json_view's dump() without a call or conversion per token
The default dump() (no indentation, no ensure_ascii) gets its own
writer that makes the same walk and produces the same output:

- the write position stays in a local variable instead of a
  member, so the compiler keeps it in a register across stores
  through aliasing char pointers;
- strings and number tokens are copied with fixed-size 32-byte
  moves wherever enough source bytes remain, instead of one
  memcpy call per token;
- the innermost open container lives in local variables; a stack
  that starts as a local array of 32 entries holds the rest;
- unedited documents are walked through the node array in order,
  and integer tokens are read from the source directly.

On top of that, float tokens of at most 15 significant digits are
written straight from their digits via zmij::to_shortest() and
write_shortest(), without converting to a double and back: such
decimals are farther apart than a double's rounding interval, so
the token's digits are the double's shortest digits. Tokens of
16+ digits, or edited values, still go through decimal_to_float().
The view's own NEON write_decimal() is removed in favor of the
shared writer, and the dump output now grows in 64 KiB steps
instead of being resized to its estimate at once.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:42 +02:00
Niels Lohmann 1a4e4d806c Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:40 +02:00
Niels Lohmann 4c78fbb85f Build the image link test without exceptions
load_result() exists only with exceptions; with JSON_NOEXCEPTION the test
loads the image with the check directly (a failed check aborts).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:39 +02:00
Niels Lohmann d8a11a131d Silence a clang-tidy C++17 finding in the image test
bugprone-suspicious-stringview-data-usage flags data() of the source view
under C++17, although the length is passed and checked by the REQUIRE
before it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:38 +02:00
Niels Lohmann 767a55bad2 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- drop edit_storage.hpp, which edit.hpp includes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:38 +02:00
Niels Lohmann 1c87e8283b Fix the indentation of a loop in the image test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:37 +02:00
Niels Lohmann 5690d776d5 Fix MSVC C4244 and C2017 in the image test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:36 +02:00
Niels Lohmann 01be361251 Fix clang-tidy 22 findings in unit-json_view_image.cpp and image.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:35 +02:00
Niels Lohmann 27d3897e0e Avoid useless casts of header fields in the image tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:34 +02:00
Niels Lohmann 66f880d22a Expect first-wins lookups in the image tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:33 +02:00
Niels Lohmann f58ce0bf6d Document that editable documents refuse invalid UTF-8 copied from a loaded image
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:32 +02:00
Niels Lohmann a1de9a56f6 Test that invalid UTF-8 of a loaded image is not copied into an editable document
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:31 +02:00
Niels Lohmann 2d6ead2eff Release the list of large objects once load() has built the indexes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:31 +02:00
Niels Lohmann 70136a68ec Test lookups in loaded images with duplicate and colliding keys
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:30 +02:00
Niels Lohmann 43a53a8abf Test that the targets of links never reach an image
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:29 +02:00
Niels Lohmann e4869ae29d Adapt the image tests to named documents and last-wins lookups
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:28 +02:00
Niels Lohmann 84e36d5f6f Check each distinct string and float range once in the full image check
The full check scanned the contents of every string node and every float token over its own range, so many nodes pointing to one large range made load() quadratic. The structural walk now only collects the ranges; after it, each kind is sorted and checked once per distinct range. Ranges of one kind that overlap without being identical are rejected, as save() never writes them (nodes that share a value share the whole range). The cost is linear in the size of the image plus sorting the ranges.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:27 +02:00
Niels Lohmann 4430b636d5 Fix MSVC errors in image test and add image_check doc anchor
Move the raw string literal out of the CHECK macro (MSVC preprocessor),
cast 64-bit header fields to std::size_t (C4244 on 32-bit), and give the
image_check section in load.md a real anchor for the mkdocs strict build.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:26 +02:00
Niels Lohmann 176eeedd07 Add images of json_documents: save() and load()
An image is a document stored so that loading it needs no
parsing: save() writes the node index, the text and the decoded
strings; the static load() reads an image written by save().

load() takes a pointer and size, a borrowed vector, or an owned
rvalue vector; the nodes are copied so they are aligned and can
be edited, while the text and decoded strings stay in the image.

image_check controls how much load() trusts the input: full
checks structure, bounds, strings and numbers, the parser's own
guarantees; bounds checks structure and bounds only; none skips
all checks, for images from a trusted source.

Layout is little-endian only ("NJVI" header, nodes, text, decoded
strings), following the idea of zero-copy formats such as
FlatBuffers and YaFF; the check follows FlatBuffers' Verifier.

New errors: parse_error.116 for a malformed image or a failed
check, type_error.320 for a discarded document or a big-endian
target.

A dedicated fuzzer and 6,000 seeded corruptions, checked under
ASan/UBSan, found and fixed two gaps: unchecked reserved header
fields, and unbounded null/boolean offsets that could make
dump() throw std::length_error.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:25 +02:00
Niels Lohmann ce31f2aecf Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:23 +02:00
Niels Lohmann e642e5507d Keep the view's edit headers clean for clang-tidy in C++17 mode
clang-tidy 22 checks the headers with -std=gnu++17 as well, where it asks
for a transparent comparator and std::make_unique. The regions map uses its
default comparator, and the allocation of the edit state keeps new (the
library is C++11) with a NOLINT.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:22 +02:00
Niels Lohmann befbafca02 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), and the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), plus <version> for
std::nullptr_t, and removing <cstddef> and edit_storage.hpp.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- drop edit_storage.hpp, which edit.hpp includes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:21 +02:00
Niels Lohmann 279009e7c3 Keep a raw string with a backslash out of a CHECK (MSVC C2017)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:20 +02:00
Niels Lohmann e4c23e568e Fix clang-tidy 22 findings in unit-json_view_edit.cpp and edit.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:20 +02:00
Niels Lohmann 070396235f Fix clang-tidy 22 findings in the json_view headers
Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast
would trip -Wcast-align=strict), use auto for a cast initialiser, add
parentheses to a mixed expression, and name bugprone-std-namespace-modification
in the NOLINTs of the tuple_size/tuple_element specialisations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
(cherry picked from commit 0effbaa7a2)
2026-10-11 12:45:19 +02:00
Niels Lohmann 17c0db7b07 Fix MSVC C4127 in the lookups of editable json documents
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:18 +02:00
Niels Lohmann 347de0a8d0 Assign the first member when set() meets duplicate keys again
Lookups find the first member of a repeated key again, so set(object, key,
value) assigns that member (keeping the key where it is) and drops the
others, as documented before, and a view taken from object[key] shows the
new value. This undoes the code and documentation changes of 5cbb8e6d6.
Lookups in edited objects (navigation of editable documents) stop at the
first match, like those of parsed objects.

The tests that commit added expect the first member from lookups now. In the
seeded differential test, the documents with repeated keys repeat them after
the real members; the reference holds the first members, which the edits
address and the lookups are compared with, while materialize() is compared
with what parse() makes of the document's dump.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:17 +02:00
Niels Lohmann e2c2fe9364 Assign the member that lookups find when set() meets duplicate keys
An editable document reads the last member of a repeated key (as the
read-only document does), but set(object, key, value) assigned the first
one: a view taken from object[key] before the call did not show the new
value. It now assigns the last member's value, keeps the key at the
position of its first occurrence (where materialize() puts it), and drops
the other members.

The seeded differential test now also edits documents whose objects
repeat keys and compares every lookup (operator[], at, find, value,
contains, count, and JSON pointers) with the parsed basic_json value.
A targeted test covers duplicates before and after edits that move the
object, in large objects with an index, in moved arrays, and in values
copied from other documents.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:16 +02:00
Niels Lohmann cb448406f6 Update the json_view banner and type_error.319 for editable documents
The banner still described a read-only view, and the documentation of
type_error.319 listed set and push_back only, although insert rejects
binary values as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:15 +02:00
Niels Lohmann 7d49247447 Replace a container by a scalar in constant time once it is linked
Replacing an array or object that spans several nodes by a scalar looks
up its parent from the root (find_parent), which is linear in the size
of the document: setting every element of a 20,000 element array took
more than half a second. The parent only has to switch to links once;
afterward the extent of the replaced value no longer matters. Nodes that
an entry of a moved sequence links to are now marked (node_flags::linked),
and assign() skips the lookup for them.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:14 +02:00
Niels Lohmann 07d8536bf1 Set a null value below a pointer as basic_json does
set(json_pointer, value) turned a null parent into an object for every
last reference token, so "/a/0" and "/a/-" below {"a":null} made
{"a":{"0":1}}, where basic_json's operator[](json_pointer) makes an
array. A null parent now becomes an array for "-" and for digit tokens
(filled with nulls up to the index) and an object otherwise. An invalid
index is reported before the parent changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:13 +02:00
Niels Lohmann 2247b0b728 Allow the edit arena to grow up to 4 GiB - 1 bytes
append_text() doubled the capacity of the arena and rejected the result
if it exceeded 4 GiB, so that an arena of more than 2 GiB could not grow
even though the 32-bit offsets of nodes address 4 GiB - 1 bytes. The
capacity is now clamped to that limit (text_capacity()), and an append
is rejected only if the bytes themselves do not fit.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:13 +02:00
Niels Lohmann 07b2891eda Check strings copied from another document for valid UTF-8
An editable document only holds valid UTF-8, but copy_scalar() copied the
strings and keys of a view of another document unchecked. A document of
a weaker check (or a borrowed text that changed after parsing) could
therefore bring ill-formed UTF-8 into it. The copy is checked now, with
the error that dump() reports for the string; copies within the same
document stay unchecked.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:12 +02:00
Niels Lohmann aa695d683b Do not leave a half-assigned container when set_moved() throws
assign() rewrote the kind, length and extent of a slot before set_moved()
registered its new element sequence. set_moved() can throw
(std::bad_alloc from reserving the bookkeeping vectors) for a slot that
is not moved yet, which left a container without the moved flag that
showed its old children. The bookkeeping is now reserved first
(reserve_moved()), so that nothing after the first write to the slot can
throw. block_of() reserves before it marks anything for the same reason.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:11 +02:00
Niels Lohmann d361cff07a Copy nested values into a document without recursion
Counting and copying the nodes of a view or a basic_json value into an
editable document recursed once per nesting level, so that a deeply
nested value overflowed the stack. The four functions now walk the value
with an explicit stack, as materialize() does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:10 +02:00
Niels Lohmann c5798e64b2 Fix old clang and MSVC C4127 in editable json documents
Value-initialize the const std::less in find_parent (clang 3.4/3.6 do not
implement DR 253), and test the Editable template argument through a
function to avoid MSVC C4127.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:09 +02:00
Niels Lohmann fd4a29e636 Add editable json_documents: set, push_back, insert, and erase
basic_json_document gets a second template parameter, Editable
(false by default), plus the aliases json_editable_document,
json_editable_view, ordered_json_editable_document and
ordered_json_editable_view.

Editable documents can change values and structure without
rewriting the source text: set()/push_back() on values, keys,
array indices and JSON pointers; insert() before an array
element; erase() of an object key, array index or JSON pointer.

New values and element sequences go into edit storage that the
document owns and never moves, so views keep referring to their
value across edits and a parsed node never moves. Read-only
documents walk the plain node array and are unaffected.

Strings are checked for UTF-8 on entry, so dump() of an editable
document never throws type_error.316. Binary values cannot be
stored (type_error.319).

A seeded differential test applies random edits to an editable
document and to the equivalent ordered_json and compares both
after every step.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:08 +02:00
Niels Lohmann 2714283deb Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:06 +02:00
Niels Lohmann 369e348378 Move the dump, comparison, and large-object tests to unit-json_view_dump.cpp
The dump and comparison tests moved to their own file in the dump pull
request; the hash index tests of this branch join them there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:05 +02:00
Niels Lohmann 231c7c1f2f Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:04 +02:00
Niels Lohmann f04142b28a Index strings with size_t in the colliding-keys test (MSVC C4244 on Win32)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:03 +02:00
Niels Lohmann 46be79d11c Avoid GCC useless-cast and strict-overflow warnings in unit-json_view.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:03 +02:00
Niels Lohmann 68898f12e4 Fix clang-tidy 22 findings in unit-json_view.cpp and unit-json_view_builder.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:02 +02:00
Niels Lohmann c7ff87b21f Fix clang-tidy 22 findings in the json_view headers
Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast
would trip -Wcast-align=strict), use auto for a cast initialiser, add
parentheses to a mixed expression, and name bugprone-std-namespace-modification
in the NOLINTs of the tuple_size/tuple_element specialisations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:01 +02:00
Niels Lohmann ef070668b8 Keep the first of duplicate keys in the object hash index again
Lookups in objects with 128 members or more return the first member of a
repeated key again, like the linear search of smaller objects. This undoes
the code change of a69542046; its test now expects the first member from
lookups (with and without a table) and the last value from materialize()
and basic_json::parse().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:45:00 +02:00
Niels Lohmann 556fdc75e2 Keep the last of duplicate keys in the object hash index
Lookups in objects with 128 members or more now return the last member of
a repeated key, like the linear search of smaller objects and like
materialize() and basic_json::parse(). build_object_index let the first
occurrence win, so the same text gave different results depending on the
size of the object.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:59 +02:00
Niels Lohmann ba811fc7bb Document the hash index number in node::extra
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:58 +02:00
Niels Lohmann 932ca7814f Use the portable string scan on non-x86 targets that define __SSE2__
WebAssembly with -msse2 defines __SSE2__, but has no <cpuid.h> or x86 intrinsics headers.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:57 +02:00
Niels Lohmann 8390420040 Release the spare capacity of the hash index in shrink_to_fit
shrink_to_fit() now trims the tables of large objects like the node array and the decoded strings, and the list of large objects is released as soon as the tables are built.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:56 +02:00
Niels Lohmann dbd773908f Bound the probe length of the large-object hash index
The key hash is not seeded, so keys chosen to collide made building the table quadratic (20,000 colliding keys took 470 ms to parse). A key may now sit at most 64 slots from its home slot; if a key would sit further away, the table is dropped and the object is searched linearly. Lookups stop after the same distance.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:56 +02:00
Niels Lohmann 02d4ede9b4 Remove an accidentally committed GCC module cache
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:55 +02:00
Niels Lohmann aeba45f128 Fix GCC module build and MSVC C4127 in the json_view SIMD code
GCC ignores the target attribute in modules, so the SSSE3 dispatch is
disabled for the module interface (the check stays portable, SSE2 is kept).
Make the 8-vs-16 byte unrolling condition in scan_string_run a
preprocessor/template split to avoid a constant condition (C4127).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:54 +02:00
Niels Lohmann 59694a5e3e Scan json_view strings with SIMD and index large objects
Speed up json_view's parser with SIMD scanning and a hash table
for large objects.

Long runs of string bytes are scanned 16 bytes at a time with NEON
(AArch64, GCC and Clang) and SSE2 (x86-64), both baseline
instruction sets. Keys keep 16 table checks before the vector
loop, because their lengths repeat from record to record; string
values get 8, because their lengths vary more. Non-ASCII text is
validated 16 bytes at a time with simdjson's "lookup4" check
(Keiser and Lemire, 2021), with NEON on AArch64 and, on x86-64,
with SSSE3. SSSE3 is not part of baseline x86-64, so the check is
compiled for SSSE3 with a function attribute and used only where
CPUID reports it, which all x86-64 CPUs since about 2011 do; the
answer is cached in a statically initialized atomic, so there is
no guard of a local static and no global constructor. The same
input is accepted either way. JSON_VIEW_NO_SIMD selects the
portable code.

On x86-64, string runs are now checked vector-first: one SSE2
compare from the first byte finds the end of most keys and short
values, instead of a branch per byte for the first 8-16 bytes.
AArch64 keeps the byte-wise steps, where a NEON mask costs more and
the branches predict well. Entering an object or array no longer
stalls: open() stores the parent's frame field by field instead of
building it on the stack and reading it back with wider loads,
which waited for the narrower stores to retire.

Objects with 128 members or more get an open-addressing hash table
built when the object closes, so operator[], at(), find(),
contains(), count(), value(), and JSON pointers take constant time
on average in such objects; of duplicate keys, the first is kept,
as for the linear search. The idea comes from Boost.JSON.

simdjson is credited in simd.hpp's SPDX block, the README, and
license.md.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:53 +02:00
Niels Lohmann b92242d2e7 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:51 +02:00
Niels Lohmann 078cfa95d4 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:50 +02:00
Niels Lohmann 5770d02a78 Split the dump and comparison tests off unit-json_view.cpp
The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view.cpp reaches that limit as the stack
grows, so its "json_view dump" and "json_view comparison" test cases move
into unit-json_view_dump.cpp. The test generator and has_duplicate_keys()
that both files use move into json_view_test_helpers.hpp.

The new file mentions JSON_HAS_CPP_17, so it is built for C++17 like the file
it was split from.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:49 +02:00
Niels Lohmann 74301297c8 Fix the clang-tidy 22 findings in json_view and its tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:48 +02:00
Niels Lohmann 95c4044c52 Document and test first-wins lookups next to dump() and ==
dump() writes every member and == resolves duplicate keys as parse()
does, whereas lookups find the first member of a duplicate key.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:47 +02:00
Niels Lohmann 4edf9bca4a Do not shrink the output of dump()
finish() copied the whole output when the buffer was more than twice as
large as the result (citm dump +11%). The tighter source_extent()
estimate already keeps the buffer of a small value small.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:46 +02:00
Niels Lohmann ab0d330468 Hold the documents of the dump and comparison tests in names
root() of a temporary document is deleted: its views would dangle.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:45 +02:00
Niels Lohmann a431c030b0 Size the dump buffer of a small value by its own extent
The source extent of a value was read from the next node, falling back to the rest of the document when that node held a decoded string. dump() of a small value could thus allocate a buffer as large as the document. Skip a few such nodes, cap the fallback estimate, and shrink a buffer that is much larger than its output.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:45 +02:00
Niels Lohmann 20d7b21b76 Avoid raw string with escapes inside CHECK macro (MSVC C2017)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:44 +02:00
Niels Lohmann 05fea2e972 Add dump() and comparisons to json_view
Add basic_json_view::dump() and the comparison operators, and read
floats from the parser's digit layout instead of rescanning the
token.

dump(indent, indent_char, ensure_ascii, number_format) writes a
value the way ordered_json::parse(text).dump() writes it for the
same arguments: members in document order, all of them should a
key occur more than once; strings escaped by the same rules, using
the library's scanning kernels; floats written with the library's
to_chars conversion, so the output equals basic_json's byte for
byte; integers copied from the source, where they are already
canonical, except -0, which parse() reads as 0. There is no
error_handler argument, because the view only holds valid UTF-8.
number_format::source copies numbers exactly as they appear in the
source (e.g. "1.50", "1E2", "-0"), which basic_json cannot provide.
operator<< takes the indentation from the stream width, as for
basic_json. The writer walks iteratively, so nesting depth is
limited by memory only.

operator== and operator!= compare two views, or a view and a
basic_json value in either order, by the rules basic_json's
operator== uses: numbers compare by value across their types,
objects compare by their members with duplicate keys resolved as
parse() resolves them, member order matters only where the object
type keeps one, and discarded views compare as discarded basic_json
values do, including under JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON.
Nothing is materialized except single scalars.

While parsing, the view now records where the integer digits, the
fraction digits, and the exponent of a float token are, so floats
and doubles with at most 19 digits are read from that layout with
the library's decimal_to_float() instead of rescanning the token.
Both round correctly, so the values are those of parse(). get<double>(),
materialize(), dump(), and the comparisons all use it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:43 +02:00
Niels Lohmann d1d0e303a0 Report one-character non-numeric array indices like basic_json in the view
Develop's #5800 made basic_json report a JSON pointer array index such as
"/a" as parse_error.109 instead of out_of_range.404, as longer ones were;
the view's array_index() follows, with "-" and the empty token still
unresolved.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 12:44:42 +02:00
113 changed files with 17739 additions and 524 deletions

No files matched your search

+5 -3
View File
@@ -62,12 +62,14 @@ labels:
- "include/nlohmann/detail/view/.*"
- "single_include/nlohmann/json_view\\.hpp"
- "tests/src/unit-json_view.*"
- "tests/src/fuzzer-parse_json_view\\.cpp"
- "tests/src/json_view_test_helpers\\.hpp"
- "tests/src/fuzzer-(parse_json_view|json_view_image)\\.cpp"
- "tests/benchmarks/json_view/.*"
- "tools/amalgamate/config_json_view\\.json"
- "docs/mkdocs/docs/features/json_view\\.md"
- "docs/mkdocs/docs/api/basic_json_(document|view)/.*"
- "docs/mkdocs/docs/api/(ordered_)?json_(document|view)\\.md"
- "docs/mkdocs/docs/examples/(basic_json_(document|view)__|(ordered_)?json_(document|view)).*"
- "docs/mkdocs/docs/api/(ordered_)?json_(editable_)?(document|view)\\.md"
- "docs/mkdocs/docs/examples/(basic_json_(document|view)__|(ordered_)?json_(editable_)?(document|view)).*"
- "tests/benchmarks/src/benchmarks_view\\.cpp"
- label: "aspect: json_view"
@@ -0,0 +1,84 @@
name: "json_view benchmarks"
# On demand only: runs the comparison of json_view with yyjson, simdjson, and
# Boost.JSON (tests/benchmarks/json_view/compare.py) on GitHub-hosted runners,
# for numbers from x86-64 and AArch64 Linux. It runs when started by hand, or
# when a pull request gets the label "benchmark" (on both architectures, with
# GCC and the default settings). Shared runners are noisy: the results show
# where json_view stands, but published numbers need a quiet machine (see
# tests/benchmarks/json_view/README.md).
on:
pull_request:
types: [labeled]
workflow_dispatch:
inputs:
runner:
description: "Runner image"
type: choice
options:
- ubuntu-24.04
- ubuntu-24.04-arm
default: ubuntu-24.04
compiler:
description: "Compiler"
type: choice
options:
- g++
- clang++
default: g++
native:
description: "Compile for the runner's CPU (-march=native)"
type: boolean
default: false
rounds:
description: "Rounds of bench_view"
type: number
default: 30
permissions:
contents: read
concurrency:
# (the label is part of the group: adding an unrelated label must not cancel a running benchmark)
group: json-view-benchmarks-${{ github.workflow }}-${{ github.ref }}-${{ github.event.label.name }}
cancel-in-progress: true
jobs:
compare:
if: github.event_name == 'workflow_dispatch' || github.event.label.name == 'benchmark'
strategy:
matrix:
runner: ${{ fromJSON(github.event_name == 'workflow_dispatch' && format('["{0}"]', inputs.runner) || '["ubuntu-24.04", "ubuntu-24.04-arm"]') }}
runs-on: ${{ matrix.runner }}
timeout-minutes: 60
steps:
- name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Download test data
run: |
cmake -S . -B build -DJSON_BuildTests=On
cmake --build build --target download_test_data
- name: Run the comparison
env:
CXX: ${{ inputs.compiler || 'g++' }}
CC: ${{ inputs.compiler == 'clang++' && 'clang' || 'gcc' }}
ROUNDS: ${{ inputs.rounds || 30 }}
NATIVE: ${{ inputs.native && '--native' || '' }}
run: python3 tests/benchmarks/json_view/compare.py --data build/test_files --download --rounds "$ROUNDS" $NATIVE
- name: Summary
run: cat tests/benchmarks/json_view/results/*.md >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: json_view-benchmarks-${{ matrix.runner }}-${{ inputs.compiler || 'g++' }}
path: tests/benchmarks/json_view/results/
+3
View File
@@ -60,3 +60,6 @@ MODULE.bazel.lock
/json.tar.xz.asc
/release_files/
/single_include/nlohmann/*.asc
# GCC module cache
/gcm.cache/
+7
View File
@@ -68,8 +68,12 @@ cc_library(
"include/nlohmann/detail/string_utils.hpp",
"include/nlohmann/detail/value_t.hpp",
"include/nlohmann/detail/view/builder.hpp",
"include/nlohmann/detail/view/compare.hpp",
"include/nlohmann/detail/view/document_data.hpp",
"include/nlohmann/detail/view/edit.hpp",
"include/nlohmann/detail/view/edit_storage.hpp",
"include/nlohmann/detail/view/errors.hpp",
"include/nlohmann/detail/view/image.hpp",
"include/nlohmann/detail/view/input.hpp",
"include/nlohmann/detail/view/iterator.hpp",
"include/nlohmann/detail/view/lookup.hpp",
@@ -78,8 +82,11 @@ cc_library(
"include/nlohmann/detail/view/materialize.hpp",
"include/nlohmann/detail/view/node.hpp",
"include/nlohmann/detail/view/number.hpp",
"include/nlohmann/detail/view/object_index.hpp",
"include/nlohmann/detail/view/pointer.hpp",
"include/nlohmann/detail/view/scan.hpp",
"include/nlohmann/detail/view/serializer.hpp",
"include/nlohmann/detail/view/simd.hpp",
"include/nlohmann/detail/view/string_ref.hpp",
"include/nlohmann/detail/view/value.hpp",
"include/nlohmann/json.hpp",
+1
View File
@@ -1398,6 +1398,7 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I
- The class contains parts of [Google Abseil](https://github.com/abseil/abseil-cpp) which is licensed under the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0).
- The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
- The view's parser (`<nlohmann/json_view.hpp>`) contains techniques and code adapted from [yyjson](https://github.com/ibireme/yyjson) by YaoYuan, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above): table-driven decoding of `\u` escapes and fixed-offset unrolled checks.
- The view's parser (`<nlohmann/json_view.hpp>`) validates non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
<img align="right" src="https://git.fsfe.org/reuse/reuse-ci/raw/branch/master/reuse-horizontal.png" alt="REUSE Software">
+2
View File
@@ -91,6 +91,8 @@ Binary values are serialized as an object containing two keys:
- [to_string](to_string.md) returns a string representation of a JSON value
- [operator<<](../operator_ltlt.md) serialize to stream
- [`basic_json_view::dump`](../basic_json_view/dump.md) the corresponding function of `basic_json_view`, serializing
directly from a flat index without building a `basic_json` value
- [Serialization](../../features/serialization.md) - the serialization article
## Version history
@@ -171,6 +171,8 @@ Linear.
- [operator!=](operator_ne.md) compare for inequality
- [operator<=>](operator_spaceship.md) comparison: 3-way (C++20)
- [basic_json_view::operator==](../basic_json_view/operator_eq.md) - the same comparison on a zero-copy view, without
building a `basic_json` value for it
## Version history
@@ -95,6 +95,8 @@ Linear.
- [operator==](operator_eq.md) comparison: equal
- [operator<=>](operator_spaceship.md) comparison: 3-way (C++20)
- [basic_json_view::operator!=](../basic_json_view/operator_ne.md) - the same comparison on a zero-copy view, without
building a `basic_json` value for it
## Version history
@@ -0,0 +1,128 @@
# <small>nlohmann::basic_json_document::</small>erase
```cpp
// (1)
std::size_t erase(view_type object, string_view_t key);
// (2)
template<typename I>
void erase(view_type array, I idx);
// (3)
std::size_t erase(const json_pointer& ptr);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `erase`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
1. Removes every member of `object` whose key is `key` (see [Notes](#notes) on duplicate keys) and returns how many
were removed; `#!cpp 0` if `object` has no member with this key.
2. Removes the element at index `idx` of `array`, which must already exist (`#!cpp idx < array.size()`).
3. Removes the value the JSON pointer `ptr` refers to, relative to [`root()`](root.md), and returns how many values
were removed: the *parent* of the target must already exist, and the target itself is removed as in 1. (an object
member; `#!cpp 0` or more) or 2. (an array element; always `#!cpp 1`). `ptr` must not be empty -- [`root()`](root.md)
itself cannot be erased.
## Template parameters
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
## Parameters
`object` (in)
: the object to remove a member of
`array` (in)
: the array to remove an element of
`key` (in)
: the key of the member(s) to remove
`idx` (in)
: the index of the element to remove; a negative value throws (see [Exceptions](#exceptions))
`ptr` (in)
: a JSON pointer to the value to remove, relative to `root()`
## Return value
1. the number of removed members (`#!cpp 0` if `object` had none with this `key`)
2. (nothing)
3. the number of removed values (`#!cpp 0` or more for an object member, always `#!cpp 1` for an array element)
## Exceptions
1. Throws [`type_error.307`](../../home/exceptions.md#jsonexceptiontype_error307) if `object` is not an object -- the
same message [`BasicJsonType::erase`](../basic_json/erase.md) throws for the same type.
2. Throws `type_error.307` if `array` is not an array. Throws
[`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is negative, or if
`#!cpp idx >= array.size()`.
3. Throws [`out_of_range.405`](../../home/exceptions.md#jsonexceptionout_of_range405) ("JSON pointer has no parent")
if `ptr` is empty. Throws what [`at`](../basic_json_view/at.md) throws (overload 3) for resolving `ptr`'s parent.
For the last reference token itself: if the parent is an array, throws what 2. throws for an index that is out of
range, or, for a token that is not a valid array index,
[`parse_error.106`](../../home/exceptions.md#jsonexceptionparse_error106) (a leading `#!cpp '0'`),
[`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) (not a number),
[`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) (too large for `size_type`), or
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) (an empty token); otherwise (an
object, or a primitive value the pointer's parent resolves to) throws what 1. throws.
Every overload also throws [`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view
does not belong to this document") if `object`/`array` is a [discarded](../basic_json_view/is_discarded.md) view or a
view of a *different* document (overloads 1-2 only; overload 3 always starts from this document's own
[`root()`](root.md)).
## Complexity
1. Linear in the number of members of `object`.
2. Linear in the number of elements of `array` at or after `idx` (they move one slot over).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array (as [`at`](../basic_json_view/at.md)), plus the complexity of 1. or 2. for
the last token.
## Notes
!!! info "Duplicate keys"
Overload 1. removes *every* member with `key`, not just the first -- unlike [`set`](set.md), which assigns the
first occurrence and drops the rest. This is why it returns a count rather than a single view: there may be
more than one member removed, or none.
Like [`set`](set.md) and [`push_back`](push_back.md), `erase` never moves an element's *value*: a view still
referring to a removed member or element keeps showing what it last held (see [Edits](index.md#edits)) -- it just no
longer appears when `array`/`object` is read, dumped, or iterated. Removing an element of `array` (2.) does shift the
*links* to the elements after it, the same way `insert`, `set`, or `push_back` on the same array would; any iterator
already taken over `array`/`object` is invalidated by an erase, since it was walking the old layout.
## Examples
??? example
The example below drops a deprecated field and a decommissioned entry from a configuration document -- using all
three overloads -- and shows what stays intact that would not with a plain `json`/`ordered_json` value: the order
of the fields around the ones removed, and the exact spelling of a number that was never touched.
```cpp
--8<-- "examples/basic_json_document__erase.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__erase.output"
```
## See also
- [insert](insert.md) - insert an element into an array
- [set](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [push_back](push_back.md) - append to an array
- [root](root.md) - the view of the root value, the starting point of overload 3
- [`BasicJsonType::erase`](../basic_json/erase.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
## Version history
- Added in version 3.13.0.
@@ -3,7 +3,7 @@
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable = false>
class basic_json_document;
```
@@ -19,6 +19,11 @@ it (a copy, or an rvalue `#!cpp std::string` that was moved in); see [`owns_sour
is move-only: copying a document would either duplicate a potentially large index and text, or leave two documents
claiming to borrow the same buffer, so it is disabled.
With `#!cpp Editable == true`, the document also offers [`set`](set.md), [`push_back`](push_back.md),
[`insert`](insert.md), and [`erase`](erase.md) to change values in place, see [Edits](#edits) below. The source text
itself is never written; a read-only document (`#!cpp Editable == false`, the default) does not carry any of the
bookkeeping edits need, and calling any of them on one fails to compile (`#!cpp static_assert`).
## Template parameters
`BasicJsonType`
@@ -26,23 +31,38 @@ claiming to borrow the same buffer, so it is disabled.
[`ordered_json`](../ordered_json.md). Only 64-bit `number_integer_t`/`number_unsigned_t` types are supported; this
is checked with a `static_assert`.
`Editable`
: whether the document supports [`set`](set.md), [`push_back`](push_back.md), [`insert`](insert.md), and
[`erase`](erase.md) (optional, `#!cpp false` by default). See [Edits](#edits) below.
## Specializations
- [**json_document**](../json_document.md) - documents of the default specialization [`json`](../json.md)
- [**ordered_json_document**](../ordered_json_document.md) - documents of [`ordered_json`](../ordered_json.md)
- [**json_document**](../json_document.md) - read-only documents of the default specialization [`json`](../json.md)
- [**ordered_json_document**](../ordered_json_document.md) - read-only documents of
[`ordered_json`](../ordered_json.md)
- [**json_editable_document**](../json_editable_document.md) - editable documents of [`json`](../json.md)
- [**ordered_json_editable_document**](../ordered_json_editable_document.md) - editable documents of
[`ordered_json`](../ordered_json.md)
## Member types
- **view_type** - the type of view returned by [`root()`](root.md) (`#!cpp basic_json_view<BasicJsonType>`)
- **view_type** - the type of view returned by [`root()`](root.md) (`#!cpp basic_json_view<BasicJsonType, Editable>`)
- **value_t** - the JSON type enumeration, see [`basic_json::value_t`](../basic_json/value_t.md)
- **image_check** - how [`load()`](load.md) validates an image (`full`, `bounds`, `none`), see [`image_check`](load.md#image_check)
## Member functions
- [(constructor)](basic_json_document.md)
### Parsing
- [**parse**](parse.md) (_static_) - deserialize from a compatible input, borrowing or owning it as appropriate
- [**parse_copy**](parse_copy.md) (_static_) - deserialize a copy of a compatible input
- [**accept**](accept.md) (_static_) - check whether the input is valid JSON
- [**read**](read.md) - (re-)parse into this document, reusing its memory
### Access
- [**root**](root.md) - the view of the root value
- [**is_discarded**](is_discarded.md) - return whether the last parse failed
- [**source**](source.md) - the parsed text
@@ -51,6 +71,49 @@ claiming to borrow the same buffer, so it is disabled.
- [**memory_usage**](memory_usage.md) - the number of bytes held by the document
- [**shrink_to_fit**](shrink_to_fit.md) - release unused index capacity
### Images
- [**save**](save.md) - the document as an image that `load()` reads without parsing
- [**load**](load.md) (_static_) - read an image written by `save()`
### Edits
- [**set**](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers
to (`#!cpp Editable` documents only)
- [**push_back**](push_back.md) - append to an array (`#!cpp Editable` documents only)
- [**insert**](insert.md) - insert an element into an array before a given position (`#!cpp Editable` documents only)
- [**erase**](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
(`#!cpp Editable` documents only)
## Edits
An editable document (`#!cpp Editable == true`) can be changed after parsing, with [`set`](set.md),
[`push_back`](push_back.md), [`insert`](insert.md), and [`erase`](erase.md);
[`json_editable_document`](../json_editable_document.md) and
[`ordered_json_editable_document`](../ordered_json_editable_document.md) are the corresponding specializations. A few
points apply to every edit:
- **The source text is never written**, and the parsed index never moves: every value keeps the node it was parsed
into, so [views](../basic_json_view/index.md) taken before an edit stay valid, including
[`root()`](root.md). New values (and the element sequences of an edited array/object) go to storage owned by the
document, allocated on demand.
- **A view keeps referring to the same value.** After [`set`](set.md) replaces the value a view refers to, that view
sees the new value; a view of a value that a later edit replaces or drops keeps showing what it last held. An edit
of an array or object, however, **invalidates the iterators taken over it** (its members may now live in a
different sequence), and a string obtained with [`get_string()`](../basic_json_view/get_string.md) stays valid even
as further edits happen (earlier buffers of edited text are kept alive, not overwritten).
- **Values are accepted three ways:** a [`basic_json_view`](../basic_json_view/index.md) of *any* document
(read-only or editable; it is copied, nothing is shared with the source document), a `BasicJsonType` value, or
anything `BasicJsonType` can be constructed from (numbers, strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
- [`dump()`](../basic_json_view/dump.md) writes an edited document with members in document order, new members at
the end, and, with [`number_format::source`](../basic_json_view/number_format.md), keeps the spelling of every
number that was not itself edited -- see [Editing a document](../../features/json_view.md#editing-a-document) for
why this matters.
- [`read()`](read.md) discards all edits, [`shrink_to_fit()`](shrink_to_fit.md) does not move the node index once
there are edits, and [`memory_usage()`](memory_usage.md) includes the memory edits use.
[`source_offset()`](../basic_json_view/source_offset.md) of a value introduced by an edit is
`#!cpp static_cast<std::size_t>(-1)`, the same value it reports for a decoded string.
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,114 @@
# <small>nlohmann::basic_json_document::</small>insert
```cpp
template<typename I, typename V>
view_type insert(view_type array, I idx, V&& value);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `insert`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
Inserts `value` into `array` as a new element before position `idx`, which must not be past the end
(`#!cpp idx <= array.size()`; `#!cpp idx == array.size()` appends, like [`push_back`](push_back.md)). Unlike
[`push_back`](push_back.md), a [null](../basic_json_view/is_null.md) `array` does *not* first become an empty array:
`array` must already be an array.
`value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any* document -- read-only or
editable, and it does not have to be `array`'s own document -- which is copied so that nothing is shared with the
source document afterward; a `BasicJsonType` value; or anything `BasicJsonType` can be constructed from (numbers,
strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
## Template parameters
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
`V`
: the type of `value`, deduced; see above for what is accepted.
## Parameters
`array` (in)
: the array to insert into
`idx` (in)
: the position to insert `value` before; a negative value throws (see [Exceptions](#exceptions))
`value` (in)
: the value to insert
## Return value
a view of the new element, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while `array` switches from its parsed
layout to a growable block, or while that block grows, see [Notes](#notes) -- can still leave `array` already
switched to that layout even though `value` itself was not inserted.
## Exceptions
Throws [`type_error.309`](../../home/exceptions.md#jsonexceptiontype_error309) if `array` is not an array -- the same
message [`BasicJsonType::insert`](../basic_json/insert.md) throws for the same type; a null `array` throws this too
(see above). Throws [`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is negative,
or if `#!cpp idx > array.size()`. Throws
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a *different* document.
Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view or a [discarded](../basic_json/is_discarded.md) `BasicJsonType`
value, and [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or contains) a
binary value -- `BasicJsonType` can hold one, but a `json_document` cannot. Throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `value` is (or contains) a string that is
not valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
## Complexity
Linear in the number of elements of `array` at or after `idx` (they move one slot over), plus time linear in the
size of `value` to encode it into the document's storage (constant for a scalar, linear in the number of nested
values for an array or object): like [`push_back`](push_back.md), the elements of `array` move to a growable block
of links the first time it is inserted into (or [`set`](set.md)/[`push_back`](push_back.md) on), and that block
grows in amortized constant time; inserting before the end within that block still shifts every later element.
## Notes
Like [`set`](set.md) on a member or an element, `insert` never moves an existing *element's value* -- only where
`array`'s *links* to its elements live -- so a view of an existing element of `array` stays valid across an
`insert`, and keeps referring to the same element even though its index shifts. Any iterator already taken over
`array` is invalidated, since it was walking the old layout. See [Edits](index.md#edits) for what stays valid across
an edit in general.
## Examples
??? example
The example below inserts a step into the middle of a deployment plan, without touching the steps that come
after it, and shows that a view taken before the insert keeps referring to the same element even though its
index shifts -- something a plain `json`/`ordered_json` array, or its `std::vector`-based storage, has no
equivalent for.
```cpp
--8<-- "examples/basic_json_document__insert.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__insert.output"
```
## See also
- [push_back](push_back.md) - append to an array
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [set](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [`BasicJsonType::insert`](../basic_json/insert.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,176 @@
# <small>nlohmann::basic_json_document::</small>load
```cpp
// (1)
static basic_json_document load(const std::uint8_t* image, std::size_t size,
const image_check check = image_check::full);
// (2)
static basic_json_document load(const std::vector<std::uint8_t>& image,
const image_check check = image_check::full);
// (3)
static basic_json_document load(std::vector<std::uint8_t>&& image,
const image_check check = image_check::full);
```
1. Reads an image [`save()`](save.md) wrote, from a pointer and a byte count. The image is **borrowed**: `image`
must stay alive and unchanged for as long as the returned document, and any view taken from it, is used.
2. Reads an image from a `#!cpp std::vector`. Also **borrowed** -- equivalent to overload 1 called with
`#!cpp image.data()` and `#!cpp image.size()`.
3. Reads an image, keeping the vector instead of copying it: `image` is moved into the document (no copy), which
then owns it for as long as it needs the text and the decoded strings. [`owns_source()`](owns_source.md) is
`#!cpp true` afterward.
In every overload, the node index is copied into storage the document itself owns -- so that it is properly aligned,
and, for an [editable](index.md#edits) document, can be edited -- while the text and the decoded strings stay in
`image`. The hash indexes [large objects](../../features/json_view.md) use for lookup are rebuilt, exactly as after
parsing.
## Parameters
`image` (in)
: the image [`save()`](save.md) wrote (overloads 1 and 2), or one to take ownership of (overload 3)
`size` (in)
: the number of bytes at `image` (overload 1)
`check` (in)
: how thoroughly to validate `image` before trusting it; see [`image_check`](#image_check) below (optional,
`#!cpp image_check::full` by default)
## Return value
The document read from the image.
## Exception safety
Overloads 1 and 2 give the strong guarantee: `image` is only read, never written, so a thrown exception leaves the
caller's buffer untouched.
Overload 3 moves `image` into the document *before* validating it, so that a good image is kept without a copy. If
loading then fails, the partially built document -- and the vector now inside it -- is discarded along with the
exception, and `image` itself is left **empty**, not restored to what was passed in. Move a copy in instead, or
validate with overload 2 first, if the original vector must survive a failed load.
## Exceptions
On a big-endian target, throws [`type_error.320`](../../home/exceptions.md#jsonexceptiontype_error320) -- the same
exception [`save()`](save.md#exceptions) throws there, since the image format is little-endian only.
Otherwise throws [`parse_error.116`](../../home/exceptions.md#jsonexceptionparse_error116) if `image` is not one
`save()` could have written, or fails the requested `check`:
| message | when |
|------------------------|------------------------------------------------------------------------------------------------------|
| `too short` | `image` is `#!cpp nullptr`, or `size` is smaller than the 64-byte header |
| `unknown format` | the header's magic bytes or version do not match, or a reserved header field is not zero |
| `sizes out of range` | the node count, text size, or decoded-string size the header describes does not fit `size`, or the `#!cpp '\0'` after the text or after the decoded strings is missing |
| `the check failed` | `check` is not `#!cpp image_check::none`, and the image fails it -- see [`image_check`](#image_check) |
!!! failure "Example messages"
```
[json.exception.parse_error.116] parse error: invalid json_document image: too short
```
```
[json.exception.parse_error.116] parse error: invalid json_document image: unknown format
```
```
[json.exception.parse_error.116] parse error: invalid json_document image: sizes out of range
```
```
[json.exception.parse_error.116] parse error: invalid json_document image: the check failed
```
## Complexity
Linear in the number of nodes, which are always copied into the document. `#!cpp image_check::none` does not read
the text or the decoded strings. `#!cpp image_check::bounds` reads one byte of the text for each float token (its sign,
to check the recorded digits against the token's length), and nothing else of the text or the decoded strings.
With `#!cpp image_check::full`, linear in the size of the image (the nodes, the text, and the decoded strings), plus
sorting the ranges of the strings and of the float tokens (at most one per node): the contents of each distinct range
are checked once, however many nodes refer to it.
## Notes
### `image_check` { #image_check }
```cpp
using image_check = detail::view::image_check;
enum class image_check
{
full,
bounds,
none
};
```
How thoroughly `load()` validates `image` before trusting it.
| value | checks | guarantees |
|----------|--------------------------------------------------------------------------------------------------------------|------------|
| `full` | everything the parser itself guarantees: structure and bounds; that every string is valid UTF-8 (and, for a string still in the source text, that it contains no quote, backslash, or control character); and that every number token is well-formed and matches the value stored for it; strings and float tokens may share a range only if the ranges are identical (as nodes that share a value do), and never overlap otherwise | reading and serializing a checked image is safe and always produces valid JSON, exactly as for a parsed document |
| `bounds` | structure and bounds only -- that every offset and count in the node index stays inside the image | reading and serializing stay memory-safe, but a crafted image can hold strings that are not valid UTF-8 or that serialize to invalid JSON ([`dump()`](../basic_json_view/dump.md) writes them unchanged or throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316)), and numbers whose values differ from their text |
| `none` | nothing | images from a trusted source only -- reading a damaged image is undefined behavior |
`full` is the default and the right choice for an image from anything you do not fully control -- a file, a cache
shared with other processes, a peer on the network. `bounds` skips scanning the text and the decoded strings, so it
fits a cache your own process just wrote and reads straight back, where damage would mean a bug or a hardware fault
rather than adversarial input; it still cannot crash or read out of bounds. `none` skips validation entirely and
should only be used for an image you trust as much as your own memory.
**Lifetime.** Overloads 1 and 2 borrow `image`: it must stay alive and byte-for-byte unchanged for as long as the
returned document, and any [view](../basic_json_view/index.md) taken from it, is used -- exactly like a document
[`parse()`](parse.md) borrowed its input for. Overload 3 avoids this by keeping the vector itself; see
[`owns_source`](owns_source.md).
!!! warning "Experimental"
The image format is versioned but not yet stable, and may change in an incompatible way before it is declared
stable; `load()` already rejects an image written by a different format version with `parse_error.116`
("unknown format"). Use images to cache a document within one build of the library, or to hand one to another
process running the *same* build on the *same* (little-endian) machine -- not as a long-term storage format.
**What `image_check::bounds` does not guarantee.** A bounds-checked image can never make `load()`,
[`root()`](root.md), element access, or [`materialize()`](../basic_json_view/materialize.md) read outside the image,
so those stay safe on a damaged one. It does *not* guarantee that the image describes valid JSON: a string
that a `full` check would have rejected can make [`dump()`](../basic_json_view/dump.md) write invalid UTF-8 or invalid
JSON, or throw `type_error.316`, and a number can read back with a value that does not match how it is spelled.
Reserve `bounds` for images you already trust to be well-formed, and use it only to skip the extra scan.
An [editable document](../json_editable_document.md) does not take such a string over either:
[`set`](set.md), [`push_back`](push_back.md) and [`insert`](insert.md) throw `type_error.316` when they copy it from
a view of the loaded document, and leave the editable document unchanged.
## Examples
??? example "Caching a document, ownership, and a rejected image"
The example below saves a parsed document as an image, checks that `load()` reproduces the original
[`dump()`](../basic_json_view/dump.md) without parsing, and shows the difference between
`load(std::move(image))` (owned) and `load(image)` (borrowed). It then damages one byte of the image and shows
`image_check::full` rejecting it with `parse_error.116`, while `image_check::bounds` -- meant for a cache the
process already trusts -- still reads it without going out of bounds.
```cpp
--8<-- "examples/basic_json_document__load.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__load.output"
```
## See also
- [save](save.md) - write the document as an image
- [owns_source](owns_source.md) - return whether the document holds its own copy of the text
- [parse](parse.md) - deserialize from JSON text instead of an image
- [Images](../../features/json_view.md#images) - why and when to use images
## Version history
- Added in version 3.13.0.
@@ -147,6 +147,7 @@ buffer of a given length, pass a pair of pointers: `#!cpp parse(ptr, ptr + len)`
- [accept](accept.md) - check whether the input is valid JSON
- [read](read.md) - (re-)parse into this document, reusing its memory
- [owns_source](owns_source.md) - return whether the document holds its own copy of the text
- [load](load.md) - read a document from an image instead of parsing JSON text
- [`BasicJsonType::parse`](../basic_json/parse.md) - the corresponding function of `basic_json`
## Version history
@@ -0,0 +1,102 @@
# <small>nlohmann::basic_json_document::</small>push_back
```cpp
template<typename V>
view_type push_back(view_type array, V&& value);
```
Appends `value` as a new last element of `array`. A [null](../basic_json_view/is_null.md) `array` first becomes an
empty array, the same way [`set`](set.md) turns a null `object` into an empty object.
`value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any* document -- read-only or
editable, and it does not have to be `array`'s own document -- which is copied so that nothing is shared with the
source document afterward; a `BasicJsonType` value; or anything `BasicJsonType` can be constructed from (numbers,
strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `push_back`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
## Template parameters
`V`
: the type of `value`, deduced; see above for what is accepted.
## Parameters
`array` (in)
: the array (or null value) to append to
`value` (in)
: the value to append
## Return value
a view of the new last element of `array`, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while `array` switches from its parsed
layout to a growable block, or while that block grows, see [Notes](#notes) -- can still leave a partial effect, such
as a null `array` argument already turned into an empty array even though `value` itself was not appended.
## Exceptions
Throws [`type_error.308`](../../home/exceptions.md#jsonexceptiontype_error308) if `array` is neither an array nor
null -- the same message [`BasicJsonType::push_back`](../basic_json/push_back.md) throws for the same type. Throws
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a *different* document.
Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view or a [discarded](../basic_json/is_discarded.md) `BasicJsonType`
value, and [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or contains) a
binary value -- `BasicJsonType` can hold one, but a `json_document` cannot. Throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `value` is (or contains) a string that is
not valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
## Complexity
Amortized constant, plus time linear in the size of `value` to encode it into the document's storage (constant for
a scalar, linear in the number of nested values for an array or object): the elements of `array` move to a growable
block of links the first time it is appended to (or [`set`](set.md) on), and that block itself grows -- doubling its
capacity, so the cost of growing it amortizes to constant per element -- only once it runs out of room. See
[Notes](#notes).
## Notes
Like [`set`](set.md) on a member or an element, `push_back` never moves an existing element itself -- only where
`array`'s *links* to its elements live -- so a view of an existing element of `array` stays valid across a
`push_back`, but any iterator already taken over `array` is invalidated, since it was walking the old layout. See
[Edits](index.md#edits) for what stays valid across an edit in general.
## Examples
??? example
The example below appends records to an array one at a time, as they might arrive from a stream of events,
without ever building a `BasicJsonType` value for the array or for the records already in it, and shows that a
view taken from an earlier `push_back` still refers to the same element once later ones have run.
```cpp
--8<-- "examples/basic_json_document__push_back.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__push_back.output"
```
## See also
- [set](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [insert](insert.md) - insert an element into an array before a given position
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [root](root.md) - the view of the root value
- [`BasicJsonType::push_back`](../basic_json/push_back.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
## Version history
- Added in version 3.13.0.
@@ -49,6 +49,11 @@ whether or not the new parse succeeds; take fresh views from [`root()`](root.md)
`input` is borrowed or owned by the same rules as [`parse()`](parse.md#notes); a document can borrow on one call and
own on the next, since ownership is decided freshly each time.
Reusing a document matters most for large inputs: the operating system provides the memory of a fresh node index one
page at a time, and every page costs a page fault the first time it is written. On x86-64 Linux (4 KiB pages), parsing
a 55 MB document into a reused document took about 40 % less time than parsing it into a fresh one. Programs that parse
many documents of similar size should therefore keep one document and call `read()`.
An integer argument that is not a `#!cpp bool` where the flags are expected, such as `#!cpp read(ptr, len)`, does not
compile; see [`parse`](parse.md#notes).
@@ -73,6 +78,7 @@ compile; see [`parse`](parse.md#notes).
- [parse](parse.md) - deserialize from a compatible input
- [root](root.md) - the view of the root value
- [load](load.md) - read a document from an image instead of parsing JSON text
## Version history
@@ -0,0 +1,104 @@
# <small>nlohmann::basic_json_document::</small>save
```cpp
std::vector<std::uint8_t> save() const;
```
Writes the document as an *image*: a byte buffer that [`load`](load.md) reads back without parsing. The image holds
the node index, the source text (plus, for an edited document, the number tokens edits wrote), and the decoded
strings (plus the strings edits wrote) -- everything [`root()`](root.md) needs, with nothing left to parse.
An edited document is written in its *current* state, with its values in document order, the way the library's own
parser would have produced them for that JSON text: a member [`set`](set.md) added goes at the end, an
[`erase`](erase.md)d member leaves no trace, and a float that is not finite (NaN or positive/negative infinity)
becomes null, the same substitution [`dump()`](../basic_json_view/dump.md) makes. The same document always saves to
the same bytes -- also across `BasicJsonType` and `#!cpp Editable`, since the image reflects document order and
values only, not which specialization produced them.
## Return value
The image, as a `#!cpp std::vector<std::uint8_t>`. Pass it, or a pointer to its data together with its size, to
[`load`](load.md) to read the document back.
## Exception safety
Strong guarantee: `save()` does not modify `#!cpp *this` (it is `#!cpp const`), so if it throws, the document is left
exactly as it was, and the partially built image is discarded with the exception.
## Exceptions
Throws [`type_error.320`](../../home/exceptions.md#jsonexceptiontype_error320) if the document is
[discarded](is_discarded.md) -- a default-constructed document, or one a failed [`parse()`](parse.md)/
[`read()`](read.md) with `allow_exceptions == false` left discarded.
On a big-endian target, throws `type_error.320` with a different message instead: the image format is little-endian
only (see [Notes](#notes)).
Throws [`out_of_range.416`](../../home/exceptions.md#jsonexceptionout_of_range416) if the node count, the text, or
the decoded strings of the image would individually reach 4 GiB -- the same 32-bit offsets
[`parse()`](parse.md#exceptions) and, for edits, [`set`](set.md)/[`push_back`](push_back.md) are already limited to.
!!! failure "Example messages"
```
[json.exception.type_error.320] cannot save a discarded json_document
```
```
[json.exception.type_error.320] json_document images need a little-endian target
```
```
[json.exception.out_of_range.416] images of 4 GiB or more are not supported by json_document
```
## Complexity
Linear in the size of the document: the number of nodes, plus the length of the text and the decoded strings that end
up in the image.
## Notes
**Format.** The image begins with a 64-byte header (the magic bytes `#!cpp "NJVI"`, a version number, the node count,
and the sizes of the text and the decoded strings, all little-endian), followed by the nodes
([16 bytes each](../../home/architecture.md#node-index-of-json-views)), the text and a `#!cpp '\0'`, and the decoded
strings and a `#!cpp '\0'`. [`load`](load.md) checks the header, and the sizes it describes, before reading anything
else -- see [`load`'s Exceptions](load.md#exceptions).
!!! warning "Experimental"
The image format is versioned but not yet stable: it may change in an incompatible way before it is declared
stable. Use images to cache a document within one build of the library, or to hand one to another process running
the *same* build on the *same* (little-endian) machine -- not as a long-term storage format. Keep the original
JSON text if you need to read a saved document back with a future library version.
**Little-endian only.** The image is written as raw little-endian bytes, with no byte-swapping. `save()` (and
[`load`](load.md)) throw `type_error.320` on a big-endian target rather than silently produce bytes a big-endian
reader could not interpret correctly.
## Examples
??? example "Caching a parsed document as an image"
The example below saves a parsed configuration as an image -- the way a service might cache one to answer later
requests without parsing the text again -- and confirms that loading it back gives exactly the same result as
parsing did, and that saving is deterministic.
```cpp
--8<-- "examples/basic_json_document__save.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__save.output"
```
## See also
- [load](load.md) - read an image written by `save()`
- [owns_source](owns_source.md) - return whether the document holds its own copy of the text
- [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document to JSON text instead of an image
- [Images](../../features/json_view.md#images) - why and when to use images
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,203 @@
# <small>nlohmann::basic_json_document::</small>set
```cpp
// (1)
template<typename V>
view_type set(view_type target, V&& value);
// (2)
template<typename V>
view_type set(view_type object, string_view_t key, V&& value);
// (3)
template<typename I, typename V>
view_type set(view_type array, I idx, V&& value);
// (4)
template<typename V>
view_type set(const json_pointer& ptr, V&& value);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `set`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
1. Replaces the value `target` refers to with `value`.
2. Sets the member `key` of the object `object` to `value`: assigns it if `object` already has a member with this
key -- the first one, should the key occur more than once, and the later duplicates are then dropped (see the
[Notes](#notes) below) -- or appends a new member at the end otherwise. A [null](../basic_json_view/is_null.md)
`object` first becomes an empty object.
3. Assigns `value` to the element at index `idx` of the array `array`, which must already exist (`#!cpp idx <
array.size()`).
4. Sets the value the JSON pointer `ptr` refers to, relative to [`root()`](root.md), to `value`. The *parent* of the
target must already exist: an object member is set as in 2. (added if it does not exist yet), an array element is
assigned as in 3., and a last reference token of `#!cpp "-"`, or equal to the size of the array, appends `value`
instead, exactly as [`push_back`](push_back.md) would. A [null](../basic_json_view/is_null.md) parent becomes what
[`basic_json::operator[]`](../basic_json/operator%5B%5D.md) with a JSON pointer makes of it: an array if the last
reference token is `#!cpp "-"` or consists of digits only (for an index beyond 0, the array is first filled with
null values up to that index), an object otherwise. An empty `ptr` sets [`root()`](root.md) itself, as in 1.
In every overload, `value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any*
document -- read-only or editable, and it does not have to be `target`'s/`object`'s/`array`'s own document -- which
is copied so that nothing is shared with the source document afterward; a `BasicJsonType` value; or anything
`BasicJsonType` can be constructed from (numbers, strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
## Template parameters
`V`
: the type of `value`, deduced; see above for what is accepted.
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
## Parameters
`target` (in)
: the value to replace
`object` (in)
: the object (or null value) whose member to set
`array` (in)
: the array whose element to assign
`key` (in)
: the key of the member to set
`idx` (in)
: the index of the element to assign; a negative value throws (see [Exceptions](#exceptions))
`ptr` (in)
: a JSON pointer to the value to set, relative to `root()`
`value` (in)
: the new value
## Return value
1. a view of `target`, now holding `value`
2. a view of the member `key` of `object`, now holding `value`
3. a view of the element `idx` of `array`, now holding `value`
4. a view of the value `ptr` refers to, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while an edited array or object switches
from its parsed layout to a growable block, see [Notes](#notes) -- can still leave a partial effect, such as a
[null](../basic_json_view/is_null.md) `object`/`array` argument already turned into an empty object/array even
though `value` itself was not linked in. Likewise, a failure of `value` in 4. leaves a null parent that is set with an
index beyond 0 already filled with the null values before the index.
## Exceptions
1. Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view, or a [discarded](../basic_json/is_discarded.md)
`BasicJsonType` value (e.g. `#!cpp BasicJsonType(value_t::discarded)`) -- an object or array `value`, of either
kind, is fine and is encoded as a whole subtree.
2. Throws [`type_error.305`](../../home/exceptions.md#jsonexceptiontype_error305) if `object` is neither an object
nor null -- the same message [`operator[]`](../basic_json_view/operator%5B%5D.md) throws for a string argument on
such a value. Throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `key` is not
valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
Also throws what 1. throws for `value`.
3. Throws `type_error.305` if `array` is not an array -- the same message `operator[]` throws for a numeric argument
on such a value. Throws [`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is
negative, or if `#!cpp idx >= array.size()`. Also throws what 1. throws for `value`.
4. Throws what [`at`](../basic_json_view/at.md) throws (overload 3) for resolving `ptr`'s parent, except that a
missing object member or an array index equal to the array's size at the very last reference token is not an
error there (it becomes a new member or an appended element) instead of
[`out_of_range.403`](../../home/exceptions.md#jsonexceptionout_of_range403)/[`out_of_range.402`](../../home/exceptions.md#jsonexceptionout_of_range402).
For the last reference token itself: if the parent is an object (or a primitive value, where it throws
`type_error.305`), throws what 2. throws; if the parent is an array, throws what 3. throws for an index that is
out of range, or, for a token that is not a valid array index,
[`parse_error.106`](../../home/exceptions.md#jsonexceptionparse_error106) (a leading `#!cpp '0'`),
[`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) (not a number),
[`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) (too large for `size_type`), or
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) (an empty token); the same errors are
thrown for a null parent and a token of digits (the parent is not changed then), and
[`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if the index is 4294967295 or more.
Also throws what 1. throws for `value`.
Every overload also throws [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or
contains) a binary value -- `BasicJsonType` can hold one, but a `json_document` cannot -- and
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `target`/`object`/`array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a
*different* document (overloads 1-3 only; overload 4 always starts from this document's own [`root()`](root.md)).
## Complexity
1. Linear in the size of `value` (encoding it into the document's storage): constant for a scalar, linear in the
number of nested values for an array or object. If `target` is itself an array or object that spans more than one
node in its parent's original, unedited layout, and `value` is a scalar, replacing it additionally costs time
linear in the size of the document, the *first* time (the parent of `target` is looked up from
[`root()`](root.md), and then switches to links) -- see [Notes](#notes). Once the parent has links, `target` is
replaced in constant time: setting every element of a large array one after the other is linear overall. To avoid
the lookup altogether, use 3. (or 2. for an object), which know the parent.
2. Linear in the number of members of `object`, to find an existing member with `key`, plus the complexity of 1. for
`value`.
3. Constant, plus the complexity of 1. for `value`.
4. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array (as [`at`](../basic_json_view/at.md)), plus the complexity of 2. or 3. for
the last token; for a null parent and an index, linear in the index.
## Notes
!!! info "Duplicate keys"
If `object` already has more than one member with `key` (2.), the *first* one is assigned `value` and every
later member with the same key is removed -- so that a lookup, an iteration, and
[`materialize()`](../basic_json_view/materialize.md) of `object` afterward all agree on a single value for
`key`, the same way [`operator[]`](../basic_json_view/operator%5B%5D.md) already picks the first occurrence of a
duplicate key for reading. See the [Notes on duplicate keys](../basic_json_view/operator%5B%5D.md#notes) of
`operator[]`.
Setting a member (2.) or an element (3., through 4.) of an array or object whose elements have not been edited
before switches it from its parsed layout to a growable block holding links to its elements; a later
[`push_back`](push_back.md) or `set` on the same container reuses that block, growing it (amortized constant time)
only once it runs out of room. This never moves an element itself -- only where the container's *links* to its
elements live -- so a view of an element stays valid, but any iterator already taken over the container is
invalidated, since it was walking the old layout. See [Edits](index.md#edits) for what stays valid across an edit in
general.
The same switch happens, for the same reason, when overload 1. replaces a multi-node array/object value with a
scalar: the *parent's* element sequence is what has to switch to links, not `target` itself, because the parent
originally stepped over `target`'s whole subtree by its node count, which no longer applies once `target` is a
one-node scalar.
## Examples
??? example "Example: (1)/(2)/(3)/(4) replace a value, set a member, assign an element, set via a JSON pointer"
The example below edits a small configuration document -- replacing a value, adding an object member, assigning
an array element, and reaching a field through a JSON pointer -- and shows what
[`dump()`](../basic_json_view/dump.md) preserves that is lost once the same edits are made on a `BasicJsonType`
value instead: the order object members were written in, and the exact spelling of a number that was never
touched.
```cpp
--8<-- "examples/basic_json_document__set.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__set.output"
```
## See also
- [push_back](push_back.md) - append to an array
- [insert](insert.md) - insert an element into an array
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [root](root.md) - the view of the root value, the starting point of overload 4
- [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document, keeping an untouched number's
spelling with `#!cpp number_format::source`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
- [Editing a document](../../features/json_view.md#editing-a-document) - why editable documents keep the source
text's order and number spelling
## Version history
- Added in version 3.13.0.
@@ -77,6 +77,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
1. Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after
another, in document order, stopping at the first match. Each comparison first checks the
key's length -- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index while parsing, so that a lookup in them takes constant time
on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
@@ -35,6 +35,8 @@ No-throw guarantee: this function never throws exceptions.
1. Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after
another, in document order, stopping at the first match. Each comparison first checks the
key's length -- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index while parsing, so that a lookup in them takes constant time
on average.
2. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array -- as for [`operator[]`](operator[].md#complexity) and
[`at`](at.md#complexity) with a JSON pointer.
@@ -26,6 +26,8 @@ No-throw guarantee: this function never throws exceptions.
Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after another,
in document order, stopping at the first match. Each comparison first checks the key's length
-- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index while parsing, so that a lookup in them takes constant time on
average.
## Notes
@@ -0,0 +1,102 @@
# <small>nlohmann::basic_json_view::</small>dump
```cpp
string_t dump(const int indent = -1,
const char indent_char = ' ',
const bool ensure_ascii = false,
const number_format numbers = number_format::shortest) const;
```
Serializes this value (and its subtree) directly from the flat index, without ever building a `BasicJsonType` value
first. With the default `#!cpp numbers == number_format::shortest`, the result is the same string
[`BasicJsonType::dump`](../basic_json/dump.md) would produce for the value
[`BasicJsonType::parse()`](../basic_json/parse.md) builds from the same source text, called with the same `indent`,
`indent_char`, and `ensure_ascii` -- except that members of an object appear in document order rather than sorted by
key, and *every* occurrence of a repeated key is written rather than only the last one (see
[Notes on duplicate keys](operator[].md#notes)). For a `json_view` (whose `BasicJsonType` is not ordered), this means
`dump()` can print an object's members in a different order than [`materialize()`](materialize.md)`.dump()` of the
same subtree.
## Parameters
`indent` (in)
: If `indent` is nonnegative, array elements and object members are pretty-printed with that indent level. An
indent level of `0` only inserts newlines. `-1` (the default) selects the most compact representation.
`indent_char` (in)
: The character used for indentation if `indent` is greater than `0`. The default is ` ` (space).
`ensure_ascii` (in)
: If `ensure_ascii` is `#!cpp true`, all non-ASCII characters in the output are escaped with `\uXXXX` sequences, and
the result consists of ASCII characters only.
`numbers` (in)
: how to write numbers, see [`number_format`](number_format.md): `shortest` (the default) writes them the way
[`BasicJsonType::dump`](../basic_json/dump.md) would; `source` copies every number exactly as it appears in the
source text.
## Return value
string containing the serialization of this value, or `#!cpp "<discarded>"` if the view is
[discarded](is_discarded.md).
## Exception safety
Strong exception safety: if an exception is thrown, there are no changes to the view or the document it refers to.
## Exceptions
May throw `#!cpp std::bad_alloc` if allocating the output string fails. Unlike
[`BasicJsonType::dump`](../basic_json/dump.md), there is no `error_handler` parameter and no
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316): the view only ever holds text the parser
already validated as UTF-8, so there is nothing to replace or ignore.
## Complexity
Linear in the size of the output text.
## Notes
The walk over the subtree is iterative, so the nesting depth it can write is limited by available memory only, not by
the call stack -- as for [`materialize()`](materialize.md).
Strings are escaped by the same rules as [`BasicJsonType::dump`](../basic_json/dump.md). With
`#!cpp numbers == number_format::shortest`, floats are written with the library's shortest round-trip conversion,
exactly as [`BasicJsonType::dump`](../basic_json/dump.md) would (e.g. `#!cpp 1.5`, `#!cpp 100.0`, `#!cpp 1e+100`), and
integers are copied from the source text -- already canonical in JSON, so this matches their shortest form too --
except that `#!cpp -0` is written as `#!cpp 0`, the way [`BasicJsonType::parse()`](../basic_json/parse.md) reads it.
`#!cpp number_format::source` copies every number exactly as written in the source text instead, with no exception
for `#!cpp -0` -- `#!cpp 1.50`, `#!cpp 1E2`, `#!cpp -0.0`, `#!cpp -0`, or all digits of an integer literal with more
digits than any number type holds (such a literal is itself classified as a float, see
[What is different](../../features/json_view.md#what-is-different)) -- something `BasicJsonType` cannot do, since
parsing already reduces every number to its parsed value.
## Examples
??? example
The example below forwards a single record out of a larger batch, and re-serializes a configuration file, both
without ever building a `BasicJsonType` value for the surrounding array or for the parts of it that were not
needed. It also shows that [`materialize()`](materialize.md)`.dump()` of the configuration sorts its keys, where
`dump()` on the view keeps the order they appear in the source text.
```cpp
--8<-- "examples/basic_json_view__dump.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__dump.output"
```
## See also
- [`number_format`](number_format.md) - how `dump()` writes numbers
- [operator<<](operator_ltlt.md) - serialize this value to a stream
- [materialize](materialize.md) - build a `BasicJsonType` value, e.g. to use `BasicJsonType::dump`'s `error_handler`
- [`BasicJsonType::dump`](../basic_json/dump.md) - the corresponding function of `basic_json`
## Version history
- Added in version 3.13.0.
@@ -28,6 +28,8 @@ No-throw guarantee: this function never throws exceptions.
Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after another,
in document order, stopping at the first match. Each comparison first checks the key's length
-- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index while parsing, so that a lookup in them takes constant time on
average.
## Notes
+30 -4
View File
@@ -3,7 +3,7 @@
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable = false>
class basic_json_view;
```
@@ -20,11 +20,19 @@ Moving the document itself does not invalidate its views: the index is heap-allo
`basic_json_document` object.
`basic_json_view` provides the read-only part of the `BasicJsonType` interface: the type-inspection functions, element
access, lookup, iteration, and conversion -- [`get<T>()`](get.md), [`get_string()`](get_string.md),
access, lookup, iteration, conversion, and comparison -- [`get<T>()`](get.md), [`get_string()`](get_string.md),
[`number_token()`](number_token.md), and [`materialize()`](materialize.md) to build the `BasicJsonType` value of a
subtree on demand. [`operator[]`](operator%5B%5D.md), [`at`](at.md), [`contains`](contains.md), and
[`value`](value.md) also accept a [`json_pointer`](../json_pointer/index.md). It does not (yet) provide `dump()` or
comparison.
[`value`](value.md) also accept a [`json_pointer`](../json_pointer/index.md). [`operator==`](operator_eq.md) and
[`operator!=`](operator_ne.md) compare two views, or a view and a `BasicJsonType` value, without ever building a
`BasicJsonType` value for a view; no ordering comparison (`#!cpp operator<`) is provided.
`basic_json_view` itself is always read-only -- it never has a `set` or `push_back` of its own. A view of an
**editable** document (`#!cpp Editable == true`) sees every edit made through
[`basic_json_document::set`](../basic_json_document/set.md) and
[`basic_json_document::push_back`](../basic_json_document/push_back.md): once a value is changed, every view that
still refers to it -- including ones taken before the change -- reads the new value. See
[Edits](../basic_json_document/index.md#edits).
## Template parameters
@@ -32,10 +40,17 @@ comparison.
: a specialization of [`basic_json`](../basic_json/index.md), matching the
[`basic_json_document`](../basic_json_document/index.md) the view was taken from.
`Editable`
: whether the view is of an editable document, matching the [`basic_json_document`](../basic_json_document/index.md)
it was taken from (optional, `#!cpp false` by default). See [Edits](../basic_json_document/index.md#edits).
## Specializations
- [**json_view**](../json_view.md) - views of a [`json_document`](../json_document.md)
- [**ordered_json_view**](../ordered_json_view.md) - views of an [`ordered_json_document`](../ordered_json_document.md)
- [**json_editable_view**](../json_editable_view.md) - views of a [`json_editable_document`](../json_editable_document.md)
- [**ordered_json_editable_view**](../ordered_json_editable_view.md) - views of an
[`ordered_json_editable_document`](../ordered_json_editable_document.md)
## Member types
@@ -47,6 +62,7 @@ comparison.
- **iterator**, **const_iterator** - a forward iterator over the elements of an array or the member values of an
object, in document order; both names refer to the same type, since a view is always read-only
- **item** - a (key, value) pair produced by [`items()`](items.md)
- [**number_format**](number_format.md) - how [`dump()`](dump.md) writes numbers
## Member functions
@@ -105,6 +121,16 @@ comparison.
- [**number_token**](number_token.md) - get a number's token text without a copy
- [**materialize**](materialize.md) - build the `BasicJsonType` value of this subtree
### Comparison
- [**operator==**](operator_eq.md) - comparison: equal
- [**operator!=**](operator_ne.md) - comparison: not equal
### Serialization
- [**dump**](dump.md) - serialize to a JSON-formatted string
- [**operator<<**](operator_ltlt.md) - serialize to stream
### Source access
- [**source_offset**](source_offset.md) - byte offset of this value in the document's source text
@@ -0,0 +1,51 @@
# <small>nlohmann::basic_json_view::</small>number_format
```cpp
enum class number_format {
shortest,
source
};
```
This enumeration is used in [`dump`](dump.md) to choose how numbers are written. Two values are differentiated:
shortest
: integers are copied from the source text -- already canonical in JSON -- except that `#!cpp -0` becomes
`#!cpp 0`, the way [`BasicJsonType::parse()`](../basic_json/parse.md) reads it; floats are written with the
library's shortest round-trip conversion, exactly as [`BasicJsonType::dump()`](../basic_json/dump.md) would (e.g.
`#!cpp 1.5`, `#!cpp 100.0`, `#!cpp 1e+100`)
source
: every number is copied exactly as it appears in the source text -- `#!cpp 1.50`, `#!cpp 1E2`, `#!cpp -0`, all
digits of an integer literal with more digits than any number type holds -- something `BasicJsonType` cannot do,
since parsing already reduces every number to its parsed value
## Examples
??? example
The example below writes back a price list received from a supplier: with `number_format::shortest` (the
default), a trailing zero and scientific notation are normalized away and a long account number that overflows
every number type is rounded, the same way `#!cpp materialize().dump()` (or `basic_json::dump()`) would;
`number_format::source` keeps every number exactly as it was written in the source text instead.
```cpp
--8<-- "examples/basic_json_view__number_format.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__number_format.output"
```
## See also
- [dump](dump.md) - serialize to a JSON-formatted string
- [number_token](number_token.md) - get a single number's token text without dumping the whole value
- [`BasicJsonType::error_handler_t`](../basic_json/error_handler_t.md) - the analogous enumeration for
`BasicJsonType::dump`'s decoding-error behavior
## Version history
- Added in version 3.13.0.
@@ -83,6 +83,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
another, in document order, stopping at the first match. Each comparison first checks the
key's length -- already known from the index, without reading the key bytes -- before comparing its content, so a
key of a different length than `key` is rejected without touching the source text.
Objects with 128 or more members get a hash index while parsing, so that a lookup in them takes constant time
on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
@@ -114,6 +116,7 @@ document.
member in order and so keep the *last* value for a repeated key, so `#!cpp v["a"]` and
`#!cpp v.materialize()["a"]` can differ. To get the value `parse()` would give, use
[`materialize()`](materialize.md) or iterate the members with [`items()`](items.md) and keep the last match.
The hash index of a larger object (128 members or more) leads to the first member of a key as well.
[`begin()`](begin.md)/[`end()`](end.md) and [`items()`](items.md) iterate over *all* members, including
duplicates, in document order. See [Duplicate keys](../../features/json_view.md#duplicate-keys) and
[`size()`](size.md#notes).
@@ -0,0 +1,109 @@
# <small>nlohmann::basic_json_view::</small>operator==
```cpp
// (1)
bool operator==(const basic_json_view& lhs, const basic_json_view& rhs);
// (2)
bool operator==(const basic_json_view& lhs, const BasicJsonType& rhs);
bool operator==(const BasicJsonType& lhs, const basic_json_view& rhs);
```
1. Compares two views for equality: whether the values [`BasicJsonType::parse()`](../basic_json/parse.md) would
produce for `lhs` and `rhs` are equal, according to `BasicJsonType`'s [`operator==`](../basic_json/operator_eq.md).
2. Compares a view and a `BasicJsonType` value for equality, in either order: whether the value `parse()` would
produce for the view and the other operand are equal, according to `BasicJsonType`'s
[`operator==`](../basic_json/operator_eq.md).
Neither overload builds a `BasicJsonType` value for a view to do the comparison (see [Notes](#notes) below). Numbers
compare by value across their types (`#!cpp 1 == 1.0`), and an object compares by its members, with duplicate keys
resolved exactly as `parse()` resolves them -- the last value, at the position of the first occurrence of the key.
This is not what [`operator[]`](operator[].md) and the other lookups return for a duplicate key: they find the *first*
member (see [Duplicate keys](../../features/json_view.md#duplicate-keys)).
## Parameters
`lhs` (in)
: first value to consider
`rhs` (in)
: second value to consider
## Return value
whether the values `lhs` and `rhs` are equal
## Exception safety
Strong exception safety: if an exception is thrown, there are no changes to either operand, or to the document(s) a
view refers to.
## Exceptions
May throw `#!cpp std::bad_alloc`. Unlike the other comparison and most other `basic_json_view` functions,
`operator==` is not `#!cpp noexcept`: resolving an object's members needs a temporary array to sort them by key (see
[Complexity](#complexity) below), and that allocation can fail.
## Complexity
Linear in the size of the compared values: every number, string, array element, and object member is visited at most
once, and the walk is iterative, so the nesting depth it can compare is limited by available memory only, not by the
call stack (as for [`materialize()`](materialize.md)). Resolving an object's members takes an additional O(n log n)
in the number of members at that level, since they are sorted by key to detect and resolve duplicates before being
compared. Two arrays of different [`size()`](size.md) are rejected without visiting either one's elements.
## Notes
Only a single number, boolean, or `#!cpp null` value is ever materialized into a `BasicJsonType`, to reuse its
`operator==` -- for numbers, so that values written differently in the source text but equal in value (e.g. an
integer and a floating-point literal) still compare equal, following the same rules `BasicJsonType` does for special
values such as `#!cpp NaN`. Constructing one of these scalars never allocates. Strings are compared directly, without
allocating, either from the source text on both sides or, for overload 2, against `BasicJsonType`'s own string.
Arrays and objects are never materialized at all; only their elements or members are visited, one pair at a time.
!!! info "How objects are compared"
For a [`json_view`](../json_view.md) (`BasicJsonType::object_t` is `#!cpp std::map`), members are compared by
key, regardless of the order they appear in the source text. For an
[`ordered_json_view`](../ordered_json_view.md) (`object_t` is `ordered_map`), they are compared in the order
they occur, so the very same two objects with their members reordered can compare equal as `json_view`s but not
as `ordered_json_view`s. This is exactly how [`json`](../json.md) and [`ordered_json`](../ordered_json.md)
compare, see ["Comparing different `basic_json` specializations"](../basic_json/operator_eq.md#notes).
!!! info "Discarded views"
A [discarded](is_discarded.md) view compares the same way a discarded `BasicJsonType` value does, which is
governed by
[`JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON`](../macros/json_use_legacy_discarded_value_comparison.md): by
default, a discarded view is never equal to anything, not even another discarded view.
No ordering comparison (`#!cpp operator<`) is provided for `basic_json_view`; [`materialize()`](materialize.md) is
the way to get a `BasicJsonType` value that supports it.
## Examples
??? example
The example below checks whether a newly received configuration differs from the previous one, and whether a
received document matches what a test expects -- directly on views, without ever materializing a `BasicJsonType`
value for either side.
```cpp
--8<-- "examples/basic_json_view__operator_eq.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__operator_eq.output"
```
## See also
- [operator!=](operator_ne.md) - compare for inequality
- [materialize](materialize.md) - build a `BasicJsonType` value, e.g. to keep comparing after the document is gone
- [`BasicJsonType::operator==`](../basic_json/operator_eq.md) - the corresponding function of `basic_json`
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,74 @@
# <small>nlohmann::basic_json_view::</small>operator<<
```cpp
std::ostream& operator<<(std::ostream& o, const basic_json_view& v);
```
Not available when [`JSON_NO_IO`](../macros/json_no_io.md) is defined.
Serializes the given view `v` to the output stream `o`, using [`dump`](dump.md) -- exactly as
`#!cpp operator<<(std::ostream&, const basic_json&)` does for a `basic_json` value.
- The indentation of the output can be controlled with the member variable `width` of the output stream `o`. For
instance, using the manipulator `std::setw(4)` on `o` sets the indentation level to `4`, and the serialization
result is the same as calling `#!cpp v.dump(4)`. A `width` of `0` or less (the default) selects the most compact
representation, as `#!cpp v.dump(-1)` does.
- The indentation character can be controlled with the member variable `fill` of the output stream `o`. For instance,
the manipulator `std::setfill('\t')` sets indentation to use a tab character rather than the default space
character.
- As for `basic_json`, `o`'s `width` is reset to `0` after this call, whether or not it was greater than `0` before.
Numbers are always written as `#!cpp v.dump()` writes them by default, i.e. as with
[`number_format::shortest`](number_format.md); there is no way to select `#!cpp number_format::source` through the
stream.
## Parameters
`o` (in, out)
: stream to write to
`v` (in)
: view to serialize
## Return value
the stream `o`
## Exceptions
May throw `#!cpp std::bad_alloc`, propagated from [`dump`](dump.md#exceptions). Unlike
`#!cpp operator<<(std::ostream&, const basic_json&)`, there is no UTF-8 decoding step that could throw
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316), and no `error_handler` to choose between --
see the [Exceptions](dump.md#exceptions) of `dump`.
## Complexity
Linear, as [`dump`](dump.md#complexity).
## Examples
??? example
The example below writes one record out of a larger batch straight to a log stream -- compact for a one-line
entry, and pretty-printed with `std::setw`/`std::setfill` for a readable dump -- without ever building a
`BasicJsonType` value for the record, or for the rest of the batch.
```cpp
--8<-- "examples/basic_json_view__operator_ltlt.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__operator_ltlt.output"
```
## See also
- [dump](dump.md) - serialize to a JSON-formatted string
- [`operator<<(std::ostream&)`](../operator_ltlt.md) - the corresponding operator for `basic_json`
- [`JSON_NO_IO`](../macros/json_no_io.md) - switch off functions relying on certain C++ I/O headers
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,82 @@
# <small>nlohmann::basic_json_view::</small>operator!=
```cpp
// (1)
bool operator!=(const basic_json_view& lhs, const basic_json_view& rhs);
// (2)
bool operator!=(const basic_json_view& lhs, const BasicJsonType& rhs);
bool operator!=(const BasicJsonType& lhs, const basic_json_view& rhs);
```
1. Compares two views for inequality. Returns `#!cpp !(lhs == rhs)`, see [operator==](operator_eq.md).
2. Compares a view and a `BasicJsonType` value for inequality, in either order. Returns `#!cpp !(lhs == rhs)` (or,
for the reversed order, `#!cpp !(rhs == lhs)`), see [operator==](operator_eq.md).
Since `operator!=` is defined as the negation of [`operator==`](operator_eq.md), it follows the same rules for
special cases: for instance, since a [discarded](is_discarded.md) view is never equal to anything by default (see
[operator=='s Notes](operator_eq.md#notes)), it is never *unequal* to anything either -- `#!cpp discarded != discarded`
is also `#!cpp false`, exactly as for a discarded `BasicJsonType` value.
## Parameters
`lhs` (in)
: first value to consider
`rhs` (in)
: second value to consider
## Return value
whether the values `lhs` and `rhs` are not equal
## Exception safety
Strong exception safety: if an exception is thrown, there are no changes to either operand, or to the document(s) a
view refers to.
## Exceptions
May throw `#!cpp std::bad_alloc`, propagated from [`operator==`](operator_eq.md#exceptions). Unlike most other
`basic_json_view` functions, `operator!=` is not `#!cpp noexcept`.
## Complexity
Linear, as [`operator==`](operator_eq.md#complexity).
## Notes
See the [Notes](operator_eq.md#notes) of `operator==` -- in particular for how an object's members are compared
(order matters for [`ordered_json_view`](../ordered_json_view.md) but not for [`json_view`](../json_view.md)) and
for how discarded views compare.
No ordering comparison (`#!cpp operator<`) is provided for `basic_json_view`; [`materialize()`](materialize.md) is
the way to get a `BasicJsonType` value that supports it.
## Examples
??? example
The example below asserts, as a test would, that a received document differs from an unwanted value, and shows
that -- as for [`json`](../json.md)/[`ordered_json`](../ordered_json.md) -- reordering an object's members is
detected as a difference for an `ordered_json_view` but not for a `json_view`.
```cpp
--8<-- "examples/basic_json_view__operator_ne.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__operator_ne.output"
```
## See also
- [operator==](operator_eq.md) - compare for equality
- [materialize](materialize.md) - build a `BasicJsonType` value, e.g. to keep comparing after the document is gone
- [`BasicJsonType::operator!=`](../basic_json/operator_ne.md) - the corresponding function of `basic_json`
## Version history
- Added in version 3.13.0.
@@ -70,6 +70,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
1. Linear in the number of members: as for [`operator[]`](operator[].md#complexity), members are compared one after
another, in document order, stopping at the first match. Plus the complexity of converting
the found member to `T` (see [`get`](get.md)).
Objects with 128 or more members get a hash index while parsing, so that a lookup in them takes constant time
on average.
2. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array -- as for the [`operator[]`](operator[].md#complexity) and
[`at`](at.md#complexity) overloads that take a JSON pointer. Plus the complexity of converting the resolved value
@@ -0,0 +1,43 @@
# <small>nlohmann::</small>json_editable_document
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using json_editable_document = basic_json_document<json, true>;
```
This type is an **editable** [`basic_json_document`](basic_json_document/index.md) of the default
[`json`](json.md) specialization: in addition to everything [`json_document`](json_document.md) offers,
[`set`](basic_json_document/set.md) and [`push_back`](basic_json_document/push_back.md) change values after
parsing, without ever rewriting the source text -- see [Edits](basic_json_document/index.md#edits) and
[Editing a document](../features/json_view.md#editing-a-document).
## Examples
??? example
The example below patches two fields of a small configuration document -- changing one and adding another --
and dumps it back out with the member order and the spelling of the untouched number preserved, something a
plain [`json`](json.md) value cannot do.
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
## See also
- [json_editable_view](json_editable_view.md) - a view of a value of a `json_editable_document`
- [json_document](json_document.md) - the read-only document this type adds edits to
- [ordered_json_editable_document](ordered_json_editable_document.md) - the corresponding editable document for
`ordered_json`
- [Edits](basic_json_document/index.md#edits) - what an edit guarantees
## Version history
Since version 3.13.0.
@@ -0,0 +1,41 @@
# <small>nlohmann::</small>json_editable_view
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using json_editable_view = basic_json_view<json, true>;
```
This type is a [`basic_json_view`](basic_json_view/index.md) of a value of a
[`json_editable_document`](json_editable_document.md). It offers the same read-only interface as
[`json_view`](json_view.md); what is different is what it can be a view *of* -- a value that
[`set`](basic_json_document/set.md) and [`push_back`](basic_json_document/push_back.md) can change, with every view
still referring to it seeing the change, see [Edits](basic_json_document/index.md#edits).
## Examples
??? example
The example below is the same as [`json_editable_document`'s](json_editable_document.md): every view read back
out of the document -- `#!cpp doc.root()` and the views nested under it -- sees the edits made through `set`.
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
## See also
- [json_editable_document](json_editable_document.md) - the document type this view refers into
- [json_view](json_view.md) - the corresponding read-only view
- [ordered_json_editable_view](ordered_json_editable_view.md) - the corresponding view for
`ordered_json_editable_document`
## Version history
Since version 3.13.0.
+2
View File
@@ -38,6 +38,8 @@ header. See also the [macro overview page](../../features/macros.md).
- [**JSON_SKIP_UNSUPPORTED_COMPILER_CHECK**](json_skip_unsupported_compiler_check.md) - do not warn about unsupported compilers
- [**JSON_USE_GLOBAL_UDLS**](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace
- [**JSON_USE_SIMDUTF**](json_use_simdutf.md) - use the simdutf library to accelerate UTF-8 validation
- [**JSON_VIEW_NO_SIMD**](json_view_no_simd.md) - use only portable code in the parser of `json_view.hpp`
- [**JSON_VIEW_USE_SSSE3**](json_view_use_ssse3.md) - validate non-ASCII strings with SSSE3 in the parser of `json_view.hpp`
## Library version
@@ -0,0 +1,50 @@
# JSON_VIEW_NO_SIMD
```cpp
#define JSON_VIEW_NO_SIMD
```
When defined, the parser of [`basic_json_document`](../basic_json_document/index.md) (`<nlohmann/json_view.hpp>`)
uses only portable C++ to scan strings. By default, it scans long runs of string bytes 16 at a time with NEON on
AArch64 (with GCC and Clang) and SSE2 on x86-64, which are part of the baseline instruction sets of these
architectures, and validates non-ASCII text with NEON (or SSSE3, see
[`JSON_VIEW_USE_SSSE3`](json_view_use_ssse3.md)).
The same input is accepted or rejected either way, with the same values, and errors are reported the same way; only
the speed differs. The macro exists for platforms whose compilers lack the intrinsics headers, and to test the portable
code.
!!! warning "Define consistently"
The macro selects between two definitions of the same inline functions. It must therefore be defined identically for
**every** translation unit that includes `<nlohmann/json_view.hpp>`; prefer a compile definition on the target.
## Default definition
By default, `#!cpp JSON_VIEW_NO_SIMD` is not defined, and the vector code is used where available.
```cpp
#undef JSON_VIEW_NO_SIMD
```
## Examples
??? example
The code below uses the portable string scanning of the view.
```cpp
#define JSON_VIEW_NO_SIMD
#include <nlohmann/json_view.hpp>
...
```
## See also
- [JSON_VIEW_USE_SSSE3](json_view_use_ssse3.md) - validate non-ASCII strings with SSSE3 on x86-64
- [json_view](../../features/json_view.md) - the zero-copy view
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,53 @@
# JSON_VIEW_USE_SSSE3
```cpp
#define JSON_VIEW_USE_SSSE3
```
When defined on x86-64, the parser of [`basic_json_document`](../basic_json_document/index.md)
(`<nlohmann/json_view.hpp>`) validates non-ASCII text in strings with SSSE3 without asking the CPU first.
By default, the parser checks once at run time whether the CPU has SSSE3 (all x86-64 CPUs since about 2011 have it)
and then validates non-ASCII text 16 bytes at a time, using the "lookup4" algorithm of
[simdjson](https://github.com/simdjson/simdjson); on CPUs without SSSE3, it validates one UTF-8 sequence at a time.
The vector check is compiled for SSSE3 with a function attribute (GCC 4.9 and later, Clang), so this needs no compiler
option. With MSVC, the check uses `__cpuid`. On AArch64, the vector check uses NEON and is always on.
Define the macro only together with a compiler option that enables SSSE3 (e.g. `-mssse3`, or `-march=` with a CPU that
has it), and only for programs that run on such CPUs. It saves the check of the CPU, which costs little. The same
input is accepted or rejected either way; only the speed of non-ASCII text differs.
!!! warning "Define consistently"
The macro selects between two definitions of the same inline functions. It must therefore be defined identically,
with the same compiler options, for **every** translation unit that includes `<nlohmann/json_view.hpp>`; mixing
translation units that define it with ones that do not is an ODR violation. Prefer a compile definition on the
target.
## Default definition
By default, `#!cpp JSON_VIEW_USE_SSSE3` is not defined.
```cpp
#undef JSON_VIEW_USE_SSSE3
```
## Examples
??? example
With CMake, for a program that only runs on CPUs with SSSE3:
```cmake
target_compile_definitions(your_target PRIVATE JSON_VIEW_USE_SSSE3)
target_compile_options(your_target PRIVATE -mssse3)
```
## See also
- [JSON_VIEW_NO_SIMD](json_view_no_simd.md) - use only portable code in the view's parser
- [JSON_USE_SIMDUTF](json_use_simdutf.md) - validate UTF-8 with simdutf in `basic_json`'s parser
## Version history
- Added in version 3.13.0.
+2
View File
@@ -86,6 +86,8 @@ Linear.
## See also
- [dump](basic_json/dump.md) - serialize to a JSON-formatted string
- [`basic_json_view::operator<<`](basic_json_view/operator_ltlt.md) - the corresponding operator for
`basic_json_view`
- [Serialization](../features/serialization.md) - the serialization article
## Version history
@@ -0,0 +1,47 @@
# <small>nlohmann::</small>ordered_json_editable_document
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using ordered_json_editable_document = basic_json_document<ordered_json, true>;
```
This type is an **editable** [`basic_json_document`](basic_json_document/index.md) of the
[`ordered_json`](ordered_json.md) specialization: [`set`](basic_json_document/set.md) and
[`push_back`](basic_json_document/push_back.md) change values after parsing, as for
[`json_editable_document`](json_editable_document.md), and
[`materialize()`](basic_json_view/materialize.md) preserves the document order of object members -- including
members [`set`](basic_json_document/set.md) added -- instead of sorting them like
[`json_editable_document`](json_editable_document.md) does.
## Examples
??? example
The example below edits a document with `set`, then shows that `materialize()` keeps the member order of the
source text (with the new member at the end) for `ordered_json_editable_document`, where it would sort the
members alphabetically for [`json_editable_document`](json_editable_document.md).
```cpp
--8<-- "examples/ordered_json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/ordered_json_editable_document.output"
```
## See also
- [ordered_json_editable_view](ordered_json_editable_view.md) - a view of a value of an
`ordered_json_editable_document`
- [ordered_json_document](ordered_json_document.md) - the read-only document this type adds edits to
- [json_editable_document](json_editable_document.md) - the corresponding editable document for the default `json`
specialization
- [Object Order](../features/object_order.md)
- [Edits](basic_json_document/index.md#edits) - what an edit guarantees
## Version history
Since version 3.13.0.
@@ -0,0 +1,40 @@
# <small>nlohmann::</small>ordered_json_editable_view
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using ordered_json_editable_view = basic_json_view<ordered_json, true>;
```
This type is a [`basic_json_view`](basic_json_view/index.md) of a value of an
[`ordered_json_editable_document`](ordered_json_editable_document.md), the corresponding view for
[`ordered_json_view`](ordered_json_view.md) the way [`json_editable_view`](json_editable_view.md) is for
[`json_view`](json_view.md).
## Examples
??? example
The example below is the same as [`ordered_json_editable_document`'s](ordered_json_editable_document.md): the
views `set` returns see the document's member order preserved on `materialize()`, unlike for a
[`json_editable_document`](json_editable_document.md).
```cpp
--8<-- "examples/ordered_json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/ordered_json_editable_document.output"
```
## See also
- [ordered_json_editable_document](ordered_json_editable_document.md) - the document type this view refers into
- [ordered_json_view](ordered_json_view.md) - the corresponding read-only view
- [json_editable_view](json_editable_view.md) - the corresponding view for `json_editable_document`
## Version history
Since version 3.13.0.
@@ -0,0 +1,38 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a deprecated field is dropped from a configuration file, and a
// decommissioned replica is removed from the list -- "price" keeps its
// trailing zero, and the fields around the removed ones keep their order
const std::string text = R"({
"name": "cache",
"legacy_host": "db0",
"host": "db1",
"price": 19.90,
"replicas": ["db2", "db3", "db4"]
})";
json_editable_document doc = json_editable_document::parse(text);
doc.erase(doc.root(), "legacy_host"); // (1) an object member
doc.erase(doc.root()["replicas"], 1); // (2) an array element ("db3")
const std::size_t removed = doc.erase(json::json_pointer("/replicas/0")); // (3) via a JSON pointer
std::cout << removed << '\n';
std::cout << doc.root().dump(2, ' ', false, json_editable_view::number_format::source) << "\n\n";
// the same edits on a plain json value: object_t is a std::map, so
// parsing already sorted the keys, and dump() rewrites every number to
// its shortest form, even "price", which was never touched
json plain = json::parse(text);
plain.erase("legacy_host");
plain["replicas"].erase(1);
plain["replicas"].erase(0);
std::cout << plain.dump(2) << '\n';
}
@@ -0,0 +1,18 @@
1
{
"name": "cache",
"host": "db1",
"price": 19.90,
"replicas": [
"db4"
]
}
{
"host": "db1",
"name": "cache",
"price": 19.9,
"replicas": [
"db4"
]
}
@@ -0,0 +1,38 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a deployment plan -- "budget" is written with a trailing zero that has
// no effect on its value
const std::string text = R"({
"release": "2026.09",
"steps": ["build", "test", "deploy"],
"budget": 19.90
})";
json_editable_document doc = json_editable_document::parse(text);
const std::size_t deploy_index = 2;
const auto deploy = doc.root()["steps"][deploy_index]; // held across the insert
doc.insert(doc.root()["steps"], deploy_index, "smoke-test"); // insert before "deploy"
// the held view still refers to "deploy", even though its index moved
// from 2 to 3, and nothing else in the document was touched
std::cout << deploy.dump() << '\n';
std::cout << doc.root().dump(2, ' ', false, json_editable_view::number_format::source) << "\n\n";
// the same edit on a plain json value: an index held from before the
// insert now refers to whatever moved into that slot, and dump()
// rewrites "budget" to its shortest form even though it was never
// touched
json plain = json::parse(text);
plain["steps"].insert(plain["steps"].begin() + static_cast<std::ptrdiff_t>(deploy_index), "smoke-test");
std::cout << plain["steps"][deploy_index].dump() << '\n';
std::cout << plain.dump(2) << '\n';
}
@@ -0,0 +1,23 @@
"deploy"
{
"release": "2026.09",
"steps": [
"build",
"test",
"smoke-test",
"deploy"
],
"budget": 19.90
}
"smoke-test"
{
"budget": 19.9,
"release": "2026.09",
"steps": [
"build",
"test",
"smoke-test",
"deploy"
]
}
@@ -0,0 +1,52 @@
#include <cstdint>
#include <iostream>
#include <string>
#include <vector>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_document = nlohmann::json_document;
using image_check = json_document::image_check;
int main()
{
std::cout << std::boolalpha;
// the image of a parsed document -- as if read back from a cache file or
// received from another process running the same build of the library
const std::string text = R"({"name": "cache", "note": "caf\u00e9", "replicas": ["db2", "db3"]})";
const json_document parsed = json_document::parse(text);
const std::vector<std::uint8_t> image = parsed.save();
// (1)/(2) load() needs no parsing, yet dumps exactly what parsing did
const json_document borrowed = json_document::load(image);
std::cout << (borrowed.root().dump() == parsed.root().dump()) << '\n';
std::cout << borrowed.owns_source() << '\n'; // borrowed: still points into `image`
// (3) load(std::move(image)) keeps the vector instead of copying it
std::vector<std::uint8_t> to_move = image;
const json_document owned = json_document::load(std::move(to_move));
std::cout << owned.owns_source() << '\n';
// a damaged image -- the last byte of the decoded string "note" holds
// (an escape sequence, so it was unescaped into the document's own
// buffer), flipped, as storage or transport corruption might do
std::vector<std::uint8_t> damaged = image;
damaged[damaged.size() - 2] = 0xFF;
// image_check::full inspects strings and numbers, so it catches the damage
try
{
static_cast<void>(json_document::load(damaged, image_check::full));
}
catch (const json::parse_error& e)
{
std::cout << e.id << '\n';
}
// image_check::bounds only checks structure and bounds, so a cache the
// process already trusts loads without the extra scan -- reading a value
// the damage did not touch is still safe
const json_document trusted = json_document::load(damaged, image_check::bounds);
std::cout << trusted.root()["name"].get<std::string>() << '\n';
}
@@ -0,0 +1,5 @@
true
false
true
116
cache
@@ -0,0 +1,24 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
int main()
{
// "events" starts out null -- the first push_back() turns it into an
// array, exactly like set() turns a null object member into an object
json_editable_document doc = json_editable_document::parse(R"({"source": "sensor-1", "events": null})");
const auto first = doc.push_back(doc.root()["events"], json{{"type", "start"}, {"t", 0}});
for (int t = 1; t <= 3; ++t)
{
doc.push_back(doc.root()["events"], json{{"type", "tick"}, {"t", t}});
}
// push_back() never moves an existing element: a view taken from an
// earlier call still refers to the same element after later ones
std::cout << first.dump() << '\n';
std::cout << doc.root()["events"].size() << '\n';
std::cout << doc.root().dump(2) << '\n';
}
@@ -0,0 +1,23 @@
{"t":0,"type":"start"}
4
{
"source": "sensor-1",
"events": [
{
"t": 0,
"type": "start"
},
{
"t": 1,
"type": "tick"
},
{
"t": 2,
"type": "tick"
},
{
"t": 3,
"type": "tick"
}
]
}
@@ -0,0 +1,30 @@
#include <cstdint>
#include <iostream>
#include <string>
#include <vector>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
int main()
{
std::cout << std::boolalpha;
// a configuration a service parses once and then caches as an image, so
// that later requests can load() it instead of parsing the text again
const std::string text = R"({"name": "cache", "host": "db1", "port": 6379, "replicas": ["db2", "db3"]})";
const json_document config = json_document::parse(text);
// save() turns the parsed document into a byte buffer: a 64-byte header,
// the node index, the source text, and the decoded strings
const std::vector<std::uint8_t> image = config.save();
std::cout << image.size() << '\n';
// the same document always saves to the same bytes
std::cout << (image == json_document::parse(text).save()) << '\n';
// loading the image back needs no parsing, yet dumps exactly what
// parsing the text produced
const json_document reloaded = json_document::load(image);
std::cout << (reloaded.root().dump() == config.root().dump()) << '\n';
}
@@ -0,0 +1,3 @@
316
true
true
@@ -0,0 +1,41 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a configuration file, as it might be read from disk -- "price" is
// written with a trailing zero that has no effect on its value
const std::string text = R"({
"name": "cache",
"host": "db1",
"port": 6379,
"price": 19.90,
"replicas": ["db2", "db3"],
"timeout": 30
})";
json_editable_document doc = json_editable_document::parse(text);
doc.set(doc.root()["port"], 6380); // (1) replace a value
doc.set(doc.root(), "region", "us-east"); // (2) add a member
doc.set(doc.root()["replicas"], 0, "db4"); // (3) assign an element
doc.set(json::json_pointer("/timeout"), 45); // (4) via a JSON pointer
// members stay in document order (the new one at the end), and a number
// that was not itself edited keeps its exact spelling
std::cout << doc.root().dump(2, ' ', false, json_editable_view::number_format::source) << "\n\n";
// the same edits on a plain json value: object_t is a std::map, so
// parsing already sorted the keys, and dump() rewrites every number to
// its shortest form, even "price", which was never touched
json plain = json::parse(text);
plain["port"] = 6380;
plain["region"] = "us-east";
plain["replicas"][0] = "db4";
plain[json::json_pointer("/timeout")] = 45;
std::cout << plain.dump(2) << '\n';
}
@@ -0,0 +1,25 @@
{
"name": "cache",
"host": "db1",
"port": 6380,
"price": 19.90,
"replicas": [
"db4",
"db3"
],
"timeout": 45,
"region": "us-east"
}
{
"host": "db1",
"name": "cache",
"port": 6380,
"price": 19.9,
"region": "us-east",
"replicas": [
"db4",
"db3"
],
"timeout": 45
}
@@ -0,0 +1,25 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using json_view = nlohmann::json_view;
int main()
{
// a large batch of sensor readings -- forward just the one that changed,
// without ever building a basic_json value for the batch or for the
// readings that are not needed
const json_document batch = json_document::parse(R"(
[{"id": 1, "temp": 21.5}, {"id": 2, "temp": 87.3}, {"id": 3, "temp": 21.7}]
)");
const json_view readings = batch.root();
std::cout << readings[1].dump() << '\n';
// a configuration file -- dump() on the view keeps the member order of
// the source text; a json value's object_t is std::map, so
// materialize().dump() of the very same view sorts the keys instead
const json_document config = json_document::parse(
R"({"name": "cache", "host": "db1", "port": 6379, "timeout": 30})");
std::cout << config.root().dump(2) << "\n\n";
std::cout << config.root().materialize().dump(2) << '\n';
}
@@ -0,0 +1,14 @@
{"id":2,"temp":87.3}
{
"name": "cache",
"host": "db1",
"port": 6379,
"timeout": 30
}
{
"host": "db1",
"name": "cache",
"port": 6379,
"timeout": 30
}
@@ -0,0 +1,28 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using json_view = nlohmann::json_view;
int main()
{
// a price list received from a supplier feed -- prices and account
// numbers must be forwarded exactly, e.g. into an invoice
const json_document doc = json_document::parse(R"(
[{"sku": "A1", "price": 19.90, "account_id": 12345678901234567890123456},
{"sku": "A2", "price": 1E2, "account_id": 98765432109876543210987654}]
)");
const json_view list = doc.root();
// number_format::shortest (the default) writes numbers the way
// basic_json::dump() would: "19.90" becomes "19.9", "1E2" becomes
// "100.0", and each account number -- far beyond any 64-bit integer --
// is rounded to the nearest double, exactly as materialize().dump()
// (or a plain nlohmann::json) would round it
std::cout << list.dump() << '\n';
// number_format::source copies every number exactly as it was written
// in the source text instead -- something basic_json cannot do at all,
// since parsing already reduces every number to its parsed value
std::cout << list.dump(-1, ' ', false, json_view::number_format::source) << '\n';
}
@@ -0,0 +1,2 @@
[{"sku":"A1","price":19.9,"account_id":1.2345678901234568e+25},{"sku":"A2","price":100.0,"account_id":9.876543210987655e+25}]
[{"sku":"A1","price":19.90,"account_id":12345678901234567890123456},{"sku":"A2","price":1E2,"account_id":98765432109876543210987654}]
@@ -0,0 +1,30 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using json = nlohmann::json;
int main()
{
// two snapshots of a polled configuration endpoint -- compare them
// directly as views, without ever building a nlohmann::json value for
// either one
const json_document previous = json_document::parse(
R"({"name": "cache", "port": 6379, "timeout": 30})");
const json_document current = json_document::parse(
R"({"port": 6379.0, "timeout": 30, "name": "cache"})");
// same members, reordered, and 6379 written as a float -- operator==
// treats them the same way BasicJsonType::operator== would
std::cout << std::boolalpha << (previous.root() == current.root()) << '\n';
// an actually changed value is detected the same way
const json_document changed = json_document::parse(
R"({"name": "cache", "port": 6380, "timeout": 30})");
std::cout << (previous.root() == changed.root()) << '\n';
// comparing a view directly against an expected json value -- handy in a
// test, without materializing the received document at all
const json expected = {{"name", "cache"}, {"port", 6379}, {"timeout", 30}};
std::cout << (previous.root() == expected) << '\n';
}
@@ -0,0 +1,3 @@
true
false
true
@@ -0,0 +1,26 @@
#include <iostream>
#include <iomanip>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using json_view = nlohmann::json_view;
int main()
{
// one order out of a large incoming batch -- write it straight to a log
// stream without ever building a basic_json value for it, or for the
// rest of the batch
const json_document doc = json_document::parse(R"(
[{"id": 1, "item": "cable"}, {"id": 2, "item": "adapter"}]
)");
const json_view orders = doc.root();
// compact, for a one-line log entry
std::cout << orders[1] << '\n';
// std::setw sets the indentation level, exactly as for basic_json
std::cout << std::setw(2) << orders[1] << "\n\n";
// std::setfill changes the indentation character
std::cout << std::setw(1) << std::setfill('\t') << orders[1] << '\n';
}
@@ -0,0 +1,10 @@
{"id":2,"item":"adapter"}
{
"id": 2,
"item": "adapter"
}
{
"id": 2,
"item": "adapter"
}
@@ -0,0 +1,28 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using ordered_json_document = nlohmann::ordered_json_document;
using json = nlohmann::json;
int main()
{
// assert, as a test would, that a received document differs from an
// unwanted shape -- without ever materializing it into a json value just
// to compare
const json_document received = json_document::parse(
R"({"status": "ok", "code": 200})");
const json unwanted = {{"status", "error"}, {"code", 500}};
std::cout << std::boolalpha << (received.root() != unwanted) << '\n';
// json (std::map) compares object members regardless of order ...
const json_document a = json_document::parse(R"({"a": 1, "b": 2})");
const json_document b = json_document::parse(R"({"b": 2, "a": 1})");
std::cout << (a.root() != b.root()) << '\n';
// ... but ordered_json (ordered_map) compares them in the order they
// appear, so the very same reordering is detected as a difference
const ordered_json_document oa = ordered_json_document::parse(R"({"a": 1, "b": 2})");
const ordered_json_document ob = ordered_json_document::parse(R"({"b": 2, "a": 1})");
std::cout << (oa.root() != ob.root()) << '\n';
}
@@ -0,0 +1,3 @@
true
false
true
@@ -0,0 +1,30 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a configuration file, as it might be read from disk
const std::string text = R"({"name": "cache", "host": "db1", "port": 6379, "price": 19.90})";
std::cout << text << "\n\n";
// patch two fields -- "price" is never touched
json_editable_document doc = json_editable_document::parse(text);
doc.set(doc.root(), "host", "db2");
doc.set(doc.root(), "retries", 3);
// member order (the new member at the end) and the untouched number's
// exact spelling survive
std::cout << doc.root().dump(-1, ' ', false, json_editable_view::number_format::source) << '\n';
// the same patch on a plain json value: keys are sorted (object_t is a
// std::map), and "price" is rewritten even though the patch never
// touched it
json plain = json::parse(text);
plain["host"] = "db2";
plain["retries"] = 3;
std::cout << plain.dump() << '\n';
}
@@ -0,0 +1,4 @@
{"name": "cache", "host": "db1", "port": 6379, "price": 19.90}
{"name":"cache","host":"db2","port":6379,"price":19.90,"retries":3}
{"host":"db2","name":"cache","port":6379,"price":19.9,"retries":3}
@@ -0,0 +1,15 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using ordered_json_editable_document = nlohmann::ordered_json_editable_document;
int main()
{
// ordered_json_editable_document is basic_json_document<nlohmann::ordered_json, true>
ordered_json_editable_document doc = ordered_json_editable_document::parse(R"({"z": 1, "a": 2, "m": 3})");
doc.set(doc.root(), "b", 4); // set() always appends a new member at the end
// materialize() preserves the document order (with "b" at the end),
// instead of sorting the keys the way json_editable_document does
std::cout << doc.root().materialize().dump() << '\n';
}
@@ -0,0 +1 @@
{"z":1,"a":2,"m":3,"b":4}
+2
View File
@@ -14,6 +14,8 @@ C++ types, and finally serialize it again.
[parsing untrusted input](parsing/untrusted_input.md).
- [Zero-copy JSON views](json_view.md) — read a JSON text through a flat index instead of building a `json` tree;
strings and numbers stay in the input and are only decoded when needed.
[Editable documents](json_view.md#editing-a-document) can also be modified, and [images](json_view.md#images) load a
parsed document again without parsing it.
- [Comments](comments.md) and [trailing commas](trailing_commas.md) — opt-in relaxations of the JSON grammar.
## Accessing and modifying values
+148 -11
View File
@@ -140,8 +140,12 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
element access and lookup functions never carry the JSON Pointer path `JSON_DIAGNOSTICS` would otherwise add: the
view has no `basic_json` value to point at, so the exception is created without one, regardless of how
`BasicJsonType` was built.
- **`dump()` and comparison are not (yet) provided** by `basic_json_view`. For now,
[`materialize()`](../api/basic_json_view/materialize.md) is the way to get a value you can do those things with.
- **Ordering comparisons are not provided** by `basic_json_view` -- there is no `#!cpp operator<`.
[`operator==`](../api/basic_json_view/operator_eq.md) and [`operator!=`](../api/basic_json_view/operator_ne.md) are
provided, though: two views, or a view and a `BasicJsonType` value, compare equal exactly when
[`materialize()`](../api/basic_json_view/materialize.md) or [`parse()`](../api/basic_json/parse.md) would produce
equal values for them, without ever building a tree to do it. For ordering, too,
[`materialize()`](../api/basic_json_view/materialize.md) is the way to get a value you can compare.
## Duplicate keys
@@ -161,9 +165,10 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
**Why the first?** A lookup can stop as soon as it finds a match. Returning the last member would force every lookup
to scan all members of the object, even when the key is found at the very first one: this made lookups in small
objects 1.6 to 3.4 times slower. Other zero-copy parsers that index the source text, such as yyjson and simdjson, also
return the first member. RFC 8259 only says that names within an object SHOULD be unique and that the behavior of a
receiver that sees duplicates is unpredictable, so neither choice is wrong.
objects 1.6 to 3.4 times slower. The hash index of objects with 128 members or more leads to the first member of a
key as well. Other zero-copy parsers that index the source text, such as yyjson and simdjson, also return the first
member. RFC 8259 only says that names within an object SHOULD be unique and that the behavior of a receiver that sees
duplicates is unpredictable, so neither choice is wrong.
**What stays the same as `parse()`?** [`materialize()`](../api/basic_json_view/materialize.md) and
[`get<std::map<...>>()`](../api/basic_json_view/get.md) replay every member in order, so they keep the *last* value
@@ -212,14 +217,146 @@ Two conversions never copy at all:
Both results are only valid as long as the view -- and, for a string with no escapes, the borrowed source text -- is.
## Writing a view back
[`dump()`](../api/basic_json_view/dump.md) serializes a view directly from the flat index, without ever building a
`basic_json` value. An object's members are written in document order, not sorted by key, and *every* occurrence of a
repeated key is written, not only the last one -- the same two ways [iteration](#what-is-different) already differs
from a [`materialize()`](../api/basic_json_view/materialize.md)d value, see [Duplicate keys](#duplicate-keys).
`#!cpp materialize().dump()` gives a different result in both respects for a `json_view`.
By default, numbers are written the way [`basic_json::dump()`](../api/basic_json/dump.md) would.
[`number_format::source`](../api/basic_json_view/number_format.md) instead copies every number exactly as it was
written in the source text -- a price like `#!cpp 19.90`, a long order or account ID with more digits than any number
type holds, or a high-precision coordinate -- something `basic_json` cannot do at all, since parsing already reduces
a number to its parsed `#!cpp double`/`#!cpp int64_t` value.
[`operator<<`](../api/basic_json_view/operator_ltlt.md) writes a view to a stream the way `basic_json`'s does, using
the stream's `width`/`fill` for indentation.
## Editing a document
Everything above is read-only: a `json_document`/`json_view` lets you look at a parsed text without copying it, but
not change it. [`basic_json_document<BasicJsonType, true>`](../api/basic_json_document/index.md) -- more conveniently
spelled [`json_editable_document`](../api/json_editable_document.md) or
[`ordered_json_editable_document`](../api/ordered_json_editable_document.md) -- also lets you
[`set`](../api/basic_json_document/set.md) a value, [`push_back`](../api/basic_json_document/push_back.md) onto or
[`insert`](../api/basic_json_document/insert.md) into an array, and [`erase`](../api/basic_json_document/erase.md)
an object member or an array element, still without ever building a `basic_json` tree for parts you do not touch.
`#!cpp Editable` defaults to `#!cpp false`, so `json_document`/`ordered_json_document` are unaffected -- they carry
none of the bookkeeping edits need, and calling `set`/`push_back`/`insert`/`erase` on one is a compile error, not a
runtime one.
### Why: editing without reformatting
The `#!cpp 19.90` price from [above](#writing-a-view-back) is exactly the kind of value that makes editing a `json`
or `ordered_json` value in place lossy. Say you parse a configuration file, patch one field, and write it back:
- **`json`** re-sorts every key on the way in (`object_t` is a `#!cpp std::map`) and rewrites every number to its
shortest round-trip form on the way out -- a one-field patch turns into a diff that reorders the whole file and
rewrites `#!cpp 19.90` to `#!cpp 19.9`.
- **`ordered_json`** keeps the key order, but still rewrites every number the same way: parsing has already reduced
it to a `#!cpp double`/`#!cpp int64_t`, and there is no way back to how it was spelled in the source text.
An editable document keeps both. [`dump()`](../api/basic_json_view/dump.md) of an edited document writes members in
document order -- a member [`set`](../api/basic_json_document/set.md) added goes at the end, exactly where it was
inserted, and an [`erase`](../api/basic_json_document/erase.md)d member simply leaves a gap: everything around it
keeps its place -- and [`number_format::source`](../api/basic_json_view/number_format.md) keeps the exact spelling
of every number an edit did not itself touch; a number an edit *did* touch is written the way
[`BasicJsonType::dump()`](../api/basic_json/dump.md) would write it, since there is no source spelling for a brand
new value.
??? example "Example: patch a configuration, keeping member order and number spellings"
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
### What stays valid, and what an edit costs
The source text itself is **never written**, and the parsed index never moves -- a value keeps the node it was
parsed into for as long as it is not itself replaced. So every [view](../api/basic_json_view/index.md) taken before
an edit, including a previously obtained [`root()`](../api/basic_json_document/root.md), stays valid and, if it
still refers to the edited value, sees the edit; a view of a value a later edit drops or replaces just keeps showing
what it last held. New values go to storage the document allocates and owns on demand. The one thing an edit does
invalidate is the **iterators** taken over an edited array or object: the first time one of its elements is set,
appended to, inserted into, or erased, its elements move from the parsed, fixed layout to a growable block of links
so that [`push_back`](../api/basic_json_document/push_back.md) can later grow it in amortized constant time --
existing elements are not touched, but an iterator that was walking the old layout no longer matches. A string
obtained with
[`get_string()`](../api/basic_json_view/get_string.md) is unaffected either way and stays valid across further
edits. See [`basic_json_document`'s Edits](../api/basic_json_document/index.md#edits) for the details, and
[`set`'s Exception safety](../api/basic_json_document/set.md#exception-safety) for what an edit guarantees if it
throws (the *basic* guarantee, not the strong one `dump()` and the read-only functions provide). How edits are kept in
the index is described in the [architecture overview](../home/architecture.md#node-index-of-json-views).
## Images
[`save()`](../api/basic_json_document/save.md) writes a document as an *image*: a byte buffer that
[`load()`](../api/basic_json_document/load.md) reads back into a document without parsing -- no lexing, no building
the node index, nothing but copying the nodes and pointing the text and the decoded strings at the image. Where
[`parse_copy()`](../api/basic_json_document/parse_copy.md) still has to scan the whole input,
[`load()`](../api/basic_json_document/load.md) turns that scan into a copy of the node index alone.
**Why.** A document that is parsed once and then read many times -- a configuration loaded at startup, a template
rendered on every request, a large reference dataset a worker process needs in memory -- pays for parsing once but
can amortize [`save()`](../api/basic_json_document/save.md)'s cost across every later load. That makes images useful
for a cache: save a document the first time it is parsed (to a file, a shared-memory segment, an in-process cache),
and [`load()`](../api/basic_json_document/load.md) it on every later use instead of parsing the source text again.
They are just as useful for handing a parsed document to another process (or a forked worker) running the same build
of the library, since [`load()`](../api/basic_json_document/load.md) turns the transfer into a copy of the node index
plus pointers into the received bytes, not a re-parse.
**Choosing a check.** [`load()`](../api/basic_json_document/load.md) takes an
[`image_check`](../api/basic_json_document/load.md#image_check) that trades validation against speed:
`image_check::full` (the default) checks everything the parser itself guarantees, so a checked image is exactly as
safe to read and serialize as a freshly parsed document -- the right choice whenever the image did not come straight
from this process's own [`save()`](../api/basic_json_document/save.md), such as a file or a network peer.
`image_check::bounds` only checks structure and bounds -- cheaper, since it skips scanning the text and the decoded
strings -- and fits a cache the process trusts, one it wrote and reads back itself. `image_check::none` skips
validation entirely, for an image trusted as much as the process's own memory. See
[`load()`'s Notes](../api/basic_json_document/load.md#notes) for exactly what each level does and does not guarantee.
??? example "Example: cache a parsed configuration as an image"
```cpp
--8<-- "examples/basic_json_document__save.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__save.output"
```
!!! warning "Experimental"
The image format is versioned but not yet stable, and may change in an incompatible way before it is declared
stable. It is little-endian only, and tied to the library build that wrote it -- use it to cache a document or to
hand one to another process running the *same* build, not as a long-term storage format; keep the original JSON
text if a saved document needs to be readable by a future library version.
The idea of a document you can read without parsing comes from zero-copy formats such as
[FlatBuffers](https://github.com/google/flatbuffers) and [YaFF](https://github.com/yandex/yaff); the check
[`load()`](../api/basic_json_document/load.md) runs follows the idea of FlatBuffers' Verifier. No code is taken from
either.
## Choosing between `json`, `ordered_json`, the SAX interface, and `json_view`
| | [`json`](../api/json.md) / [`ordered_json`](../api/ordered_json.md) | [SAX interface](parsing/sax_interface.md) | [`json_document`](../api/json_document.md) / [`json_view`](../api/json_view.md) |
|---|---|---|---|
| **Ownership** | owns every value | owns nothing; you decide what to keep, in your handler | borrows or owns the *text*; the index is always owned by the document |
| **Mutability** | freely mutable | not applicable (a one-shot event stream) | read-only |
| **What you get** | a full tree you can read, write, and keep as long as you like | a sequence of callbacks; whatever your handler builds from them | a flat index plus, on demand, [`materialize()`](../api/basic_json_view/materialize.md)d `json`/`ordered_json` values for the parts you actually use |
| **Typical use** | general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto | validating or projecting a text into your own data structure without ever holding the whole thing as JSON | large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives |
| | [`json`](../api/json.md) / [`ordered_json`](../api/ordered_json.md) | [SAX interface](parsing/sax_interface.md) | [`json_document`](../api/json_document.md) / [`json_view`](../api/json_view.md) | [`json_editable_document`](../api/json_editable_document.md) / [`json_editable_view`](../api/json_editable_view.md) |
|---|---|---|---|---|
| **Ownership** | owns every value | owns nothing; you decide what to keep, in your handler | borrows or owns the *text*; the index is always owned by the document | same as `json_document`; edits go to storage the document owns |
| **Mutability** | freely mutable | not applicable (a one-shot event stream) | read-only | [`set`](../api/basic_json_document/set.md)/[`push_back`](../api/basic_json_document/push_back.md)/[`insert`](../api/basic_json_document/insert.md)/[`erase`](../api/basic_json_document/erase.md) edit in place; the source text is never rewritten |
| **What you get** | a full tree you can read, write, and keep as long as you like | a sequence of callbacks; whatever your handler builds from them | a flat index plus, on demand, [`materialize()`](../api/basic_json_view/materialize.md)d `json`/`ordered_json` values for the parts you actually use | the same, plus [`dump()`](../api/basic_json_view/dump.md) of an edited document that keeps the member order and, with [`number_format::source`](../api/basic_json_view/number_format.md), the spelling of every untouched number |
| **Typical use** | general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto | validating or projecting a text into your own data structure without ever holding the whole thing as JSON | large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives | a document you read, patch a few fields of, and write back -- a configuration file, for instance -- where the rest of it should come back exactly as it was |
| **Caching/reload** | not applicable -- re-parse, or roll your own serialization | not applicable | [`save()`](../api/basic_json_document/save.md)/[`load()`](../api/basic_json_document/load.md): cache the parsed index as an image and reload it without parsing | same, saving the document's current -- possibly edited -- state |
## Version history
+31 -3
View File
@@ -194,9 +194,9 @@ packet-beta
| Bytes | Field | Type | Contents |
|-------|---------|------------|-------------------------------------------------------------------------------------------------------------------------------|
| 0 | `kind` | `uint8_t` | the type, numbered as [`value_t`](../api/basic_json/value_t.md): 0 null, 1 object, 2 array, 3 string, 4 boolean, 5 signed integer, 6 unsigned integer, 7 float |
| 1 | `flags` | `uint8_t` | bits 0-1: where a string's bytes are (0: the source text, 1: the buffer of decoded strings, for strings with escapes); bit 2: the value of a boolean |
| 2-3 | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; otherwise 0 |
| 0 | `kind` | `uint8_t` | the type, numbered as [`value_t`](../api/basic_json/value_t.md): 0 null, 1 object, 2 array, 3 string, 4 boolean, 5 signed integer, 6 unsigned integer, 7 float; 10 for a link (see below) |
| 1 | `flags` | `uint8_t` | bits 0-1: where a string's bytes (or a number's token) are (0: the source text, 1: the buffer of decoded strings, for strings with escapes, 2: the edit buffer); bit 2: the value of a boolean; bits 3 and 4: moved and new (see below) |
| 2-3 | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; objects: the number of their hash index (1-based), or 0; otherwise 0 |
| 4-7 | `off` | `uint32_t` | where the value starts: the first byte after a string's opening quote (or its position in the buffer of decoded strings), the first byte of a number or literal, the bracket of an array or object |
| 8-11 | `len` | `uint32_t` | strings: the length after decoding; floats and literals: the length of the token; arrays and objects: the number of elements |
| 12-15 | `next` | `uint32_t` | arrays and objects: the number of nodes of the subtree, including the node itself |
@@ -212,6 +212,11 @@ packet-beta
step from element to element this way and skip whole subtrees in constant time.
- **Offsets** are 32 bits wide, so a document is limited to 4294967279 bytes, 4 GiB minus 16 bytes (a margin below
2^32 for positions one scanner step past the end of the text; `out_of_range.416`).
- **Large objects** (128 members or more) get a hash index after parsing
([`detail/view/object_index.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/object_index.hpp)):
an open-addressing table whose slots hold the distance from the object's node to a key's node, so that a lookup does
not compare every key. Of duplicate keys the table leads to the first, as a linear search does. The object's `extra`
holds the number of its table. Only 65,535 tables fit into `extra`; objects beyond them are searched linearly.
For example, `#!json {"a": [1, 2.5]}` becomes five nodes. Each node's elements follow it, and `next` leads from an
array or object past its subtree:
@@ -240,6 +245,29 @@ flowchart LR
All `flags` are 0. The integer's bytes 8-15 hold its value, 1; its `extra` says it has one digit. The float's `extra`
says it has one integer and one fraction digit, and its `len` is that of the token `2.5`.
Editable documents ([`json_editable_document`](../api/json_editable_document.md),
[`detail/view/edit.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/edit.hpp) and
[`detail/view/edit_storage.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/edit_storage.hpp))
never write the source text and never move or resize the parsed index, so views stay valid while the document is
edited:
- A new scalar is written over its node. Its text (a string, or the token of a number as `dump()` writes it) goes to
the edit buffer, which `flags` bits 0-1 then name.
- An array or object whose elements change gets the flag *moved* (bit 3): its elements then live in a separate
sequence (a header node, then the entries), whose number is in `off`. The entries are links (`kind` 10), whose bytes
8-15 hold the address of the value's node, so values never move.
- A node written by an edit gets the flag *new* (bit 4): it has no position in the source text.
- Views of read-only documents compile without any of this: how views walk the index is a template parameter
(`navigation<Editable>`).
Images ([`save`](../api/basic_json_document/save.md) and [`load`](../api/basic_json_document/load.md),
[`detail/view/image.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/image.hpp)) store
the nodes as they are: a 64-byte header (the magic bytes `NJVI`, a format version, the sizes, and reserved bytes that
must be zero), the nodes, the text, and the decoded strings. An edited document is first written in document order, as
the parser would have written it (without links), and the numbers of the hash indexes are cleared, since `load`
rebuilds the indexes. So a change of the node layout is a change of the image format: it must raise `image_version`,
and `load` then rejects images of other versions (`parse_error.116`) instead of misreading them.
## Input adapters
Input is read via **input adapters** that abstract a source. Every input adapter provides this interface:
+69 -2
View File
@@ -395,6 +395,23 @@ A UBJSON high-precision number could not be parsed.
[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing UBJSON high-precision number: invalid number text: 1A
```
### json.exception.parse_error.116
[`basic_json_document::load()`](../api/basic_json_document/load.md) rejected an
[image](../features/json_view.md#images): either the bytes are not one [`save()`](../api/basic_json_document/save.md)
could have written (too short, an unknown magic number or format version, or sizes that do not fit the buffer), or
they are, but fail the requested [`image_check`](../api/basic_json_document/load.md#image_check).
!!! failure "Example message"
```
[json.exception.parse_error.116] parse error: invalid json_document image: the check failed
```
!!! note
This exception was added in version 3.13.0, together with [images](../features/json_view.md#images).
## Iterator errors
This exception is thrown if iterators passed to a library function do not match
@@ -812,6 +829,45 @@ does not list an enumerator and it is therefore converted like the first listed
[json.exception.type_error.318] duplicate object key 'red'
```
### json.exception.type_error.319
[`basic_json_document::set`](../api/basic_json_document/set.md),
[`basic_json_document::push_back`](../api/basic_json_document/push_back.md), and
[`basic_json_document::insert`](../api/basic_json_document/insert.md) can store any `basic_json` value except
a binary one: a `json_document` has no representation for [binary values](../features/binary_values.md), which only
ever arise from parsing a binary format or from an explicit [`json::binary`](../api/basic_json/binary.md) value, not
from JSON text.
!!! failure "Example message"
```
[json.exception.type_error.319] cannot store a binary value in a json_document
```
!!! note
This exception was added in version 3.13.0, together with editable [`json_document`s](../features/json_view.md).
### json.exception.type_error.320
[`basic_json_document::save()`](../api/basic_json_document/save.md) cannot write an
[image](../features/json_view.md#images) of a [discarded](../api/basic_json_document/is_discarded.md) document.
[`save()`](../api/basic_json_document/save.md) and [`load()`](../api/basic_json_document/load.md) also throw this
exception on a big-endian target, since the image format is little-endian only.
!!! failure "Example messages"
```
[json.exception.type_error.320] cannot save a discarded json_document
```
```
[json.exception.type_error.320] json_document images need a little-endian target
```
!!! note
This exception was added in version 3.13.0, together with [images](../features/json_view.md#images).
### json.exception.type_error.321
A discarded value (one created by [`parse()`](../api/basic_json/parse.md) with a callback that returns `false` for the
@@ -1056,13 +1112,24 @@ MessagePack's ext type and BSON's binary subtype are each stored in a single byt
[`basic_json_document::parse()`](../api/basic_json_document/parse.md) and the other parsing functions of
[`basic_json_document`](../api/basic_json_document/index.md) index a value's position in the source text in 32 bits,
so they do not support an input of 4294967280 bytes (4 GiB minus 16 bytes) or more.
so they do not support an input of 4294967280 bytes (4 GiB minus 16 bytes) or more. The same 32-bit limit applies to an **editable** document's own
storage: [`set`](../api/basic_json_document/set.md) and [`push_back`](../api/basic_json_document/push_back.md) throw
this exception once the strings and number tokens written by edits reach 4 GiB in total, or once more than
4294967295 arrays/objects have had an element set or appended to them. The same limit applies to an
[image](../features/json_view.md#images): [`save()`](../api/basic_json_document/save.md) throws it if the node
count, the text, or the decoded strings it would write would individually reach 4 GiB.
!!! failure "Example message"
!!! failure "Example messages"
```
[json.exception.out_of_range.416] input of 4294967280 bytes or more is not supported by json_document
```
```
[json.exception.out_of_range.416] edits of 4 GiB or more are not supported by json_document
```
```
[json.exception.out_of_range.416] images of 4 GiB or more are not supported by json_document
```
!!! note
+2
View File
@@ -25,3 +25,5 @@ The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Eva
The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
The view's parser (`<nlohmann/json_view.hpp>`) contains techniques and code adapted from [yyjson](https://github.com/ibireme/yyjson) by YaoYuan, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above): table-driven decoding of `\u` escapes and fixed-offset unrolled checks.
The view's parser (`<nlohmann/json_view.hpp>`) validates non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
+17
View File
@@ -238,14 +238,20 @@ nav:
- 'Overview': api/basic_json_document/index.md
- '(Constructor)': api/basic_json_document/basic_json_document.md
- 'accept': api/basic_json_document/accept.md
- 'erase': api/basic_json_document/erase.md
- 'insert': api/basic_json_document/insert.md
- 'is_discarded': api/basic_json_document/is_discarded.md
- 'load': api/basic_json_document/load.md
- 'memory_usage': api/basic_json_document/memory_usage.md
- 'node_count': api/basic_json_document/node_count.md
- 'owns_source': api/basic_json_document/owns_source.md
- 'parse': api/basic_json_document/parse.md
- 'parse_copy': api/basic_json_document/parse_copy.md
- 'push_back': api/basic_json_document/push_back.md
- 'read': api/basic_json_document/read.md
- 'root': api/basic_json_document/root.md
- 'save': api/basic_json_document/save.md
- 'set': api/basic_json_document/set.md
- 'shrink_to_fit': api/basic_json_document/shrink_to_fit.md
- 'source': api/basic_json_document/source.md
- basic_json_view:
@@ -258,6 +264,7 @@ nav:
- 'cend': api/basic_json_view/cend.md
- 'contains': api/basic_json_view/contains.md
- 'count': api/basic_json_view/count.md
- 'dump': api/basic_json_view/dump.md
- 'empty': api/basic_json_view/empty.md
- 'end': api/basic_json_view/end.md
- 'find': api/basic_json_view/find.md
@@ -280,8 +287,12 @@ nav:
- 'is_structured': api/basic_json_view/is_structured.md
- 'items': api/basic_json_view/items.md
- 'materialize': api/basic_json_view/materialize.md
- 'number_format': api/basic_json_view/number_format.md
- 'number_token': api/basic_json_view/number_token.md
- 'operator<<': api/basic_json_view/operator_ltlt.md
- 'operator[]': api/basic_json_view/operator[].md
- 'operator==': api/basic_json_view/operator_eq.md
- 'operator!=': api/basic_json_view/operator_ne.md
- 'size': api/basic_json_view/size.md
- 'source_offset': api/basic_json_view/source_offset.md
- 'type': api/basic_json_view/type.md
@@ -302,6 +313,8 @@ nav:
- 'to_json': api/adl_serializer/to_json.md
- 'json': api/json.md
- 'json_document': api/json_document.md
- 'json_editable_document': api/json_editable_document.md
- 'json_editable_view': api/json_editable_view.md
- json_pointer:
- 'Overview': api/json_pointer/index.md
- '(Constructor)': api/json_pointer/json_pointer.md
@@ -343,6 +356,8 @@ nav:
- 'operator""_json_pointer': api/operator_literal_json_pointer.md
- 'ordered_json': api/ordered_json.md
- 'ordered_json_document': api/ordered_json_document.md
- 'ordered_json_editable_document': api/ordered_json_editable_document.md
- 'ordered_json_editable_view': api/ordered_json_editable_view.md
- 'ordered_json_view': api/ordered_json_view.md
- 'ordered_map': api/ordered_map.md
- macros:
@@ -376,6 +391,8 @@ nav:
- 'JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON': api/macros/json_use_legacy_discarded_value_comparison.md
- 'JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS': api/macros/json_use_objects_for_enum_keyed_maps.md
- 'JSON_USE_SIMDUTF': api/macros/json_use_simdutf.md
- 'JSON_VIEW_NO_SIMD': api/macros/json_view_no_simd.md
- 'JSON_VIEW_USE_SSSE3': api/macros/json_view_use_ssse3.md
- 'NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_derived_type.md
- 'NLOHMANN_DEFINE_TYPE_INTRUSIVE, NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_intrusive.md
- 'NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_non_intrusive.md
@@ -1342,6 +1342,48 @@ inline char* write_shortest(char* first, const zmij::shortest_decimal d) noexcep
return end + (three ? 5 : 4);
}
/// the powers of ten up to 10^16
inline const std::array<std::uint64_t, 17>& powers_of_ten_16() noexcept
{
static const std::array<std::uint64_t, 17> powers =
{
{
1u, 10u, 100u, 1000u, 10000u, 100000u, 1000000u, 10000000u, 100000000u, 1000000000u, 10000000000u,
100000000000u, 1000000000000u, 10000000000000u, 100000000000000u, 1000000000000000u, 10000000000000000u
}
};
return powers;
}
/*!
@brief digits * 10^exp, as write_decimal() writes it, for the digits of a
double that need no conversion (count digits, at most 15, the first not 0;
trailing zeros allowed): extended to 16 digits and written by write_shortest()
@return a pointer past the text; up to 41 bytes at @a first are written
(some beyond the returned end)
*/
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_short_decimal(char* first, std::uint64_t digits, int count, int exp) noexcept
{
JSON_ASSERT(digits >= powers_of_ten_16()[static_cast<std::size_t>(count - 1)] && count <= 15);
const int scale = 16 - count;
return write_shortest(first, zmij::shortest_decimal{digits * powers_of_ten_16()[static_cast<std::size_t>(scale)], exp - scale - 1, 0, false});
}
/// as write_short_decimal(), counting the digits (not 0, less than 10^15)
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_short_decimal(char* first, std::uint64_t digits, int exp) noexcept
{
JSON_ASSERT(digits != 0 && digits < 1000000000000000u);
// floor(log10(2^bits)) + 1 digits, or one less
const int log2_bound = ((64 - count_leading_zeros(digits)) * 1233) >> 12;
const int count = log2_bound + (digits >= powers_of_ten_16()[static_cast<std::size_t>(log2_bound)] ? 1 : 0);
return write_short_decimal(first, digits, count, exp);
}
/*!
@brief whether FloatType is an IEEE 754 binary64 type (a double, or a long double
that has the same format, as with MSVC and on Apple's Arm CPUs)
+39 -16
View File
@@ -116,6 +116,13 @@ class builder
frame shallow[64]; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays): not initialized on purpose; filled as containers open
std::vector<frame> deep{};
/// remember an object to index after parsing (out of line, so that the
/// parse loop only has a call for it)
NLOHMANN_VIEW_NOINLINE void note_large_object(std::uint32_t idx)
{
doc.large_objects.push_back(idx);
}
NLOHMANN_VIEW_NOINLINE bool fail(error_code c, const unsigned char* at) noexcept
{
m_failure.code = c;
@@ -511,7 +518,7 @@ class builder
switch (cur()) \
{ \
case '"': \
if (NLOHMANN_VIEW_UNLIKELY(!string())) { return false; } \
if (NLOHMANN_VIEW_UNLIKELY(!string<true>())) { return false; } \
goto NEXT; \
case '{': \
open(value_t::object); \
@@ -595,7 +602,7 @@ obj_key:
{
return fail(error_code::expected_key);
}
if (NLOHMANN_VIEW_UNLIKELY(!string()))
if (NLOHMANN_VIEW_UNLIKELY(!string<false>()))
{
return false;
}
@@ -636,19 +643,27 @@ obj_next:
if (enabled(TrailingCommas) && cur() == '}')
{
++p;
goto close_container;
goto close_object;
}
goto obj_key;
}
if (cur() == '}')
{
++p;
goto close_container;
goto close_object;
}
return fail(error_code::expected_object_end);
#undef NLOHMANN_VIEW_VALUE
close_object:
// a large object gets a hash index (objects only, so that closing
// an array pays nothing for this)
if (NLOHMANN_VIEW_UNLIKELY(cur_count >= document_data::index_min_members))
{
cold.note_large_object(cur_idx);
}
close_container:
close();
if (NLOHMANN_VIEW_UNLIKELY(depth == 0))
@@ -698,7 +713,7 @@ root_done:
switch (cur())
{
case '"':
return string();
return string<true>();
case 't':
return literal("true", 4, value_t::boolean, node_flags::is_true);
case 'f':
@@ -827,14 +842,19 @@ indent_done:
const auto idx = static_cast<std::uint32_t>(n - base);
if (depth != 0)
{
const frame f = {cur_idx, cur_count, cur_is_object};
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
{
cold.shallow[depth - 1] = f;
// field by field: a frame put together on the stack and
// copied would be read back wider than it was written,
// and that load waits until the stores are done
frame& f = cold.shallow[depth - 1];
f.idx = cur_idx;
f.count = cur_count;
f.is_object = cur_is_object;
}
else
{
cold.deep.push_back(f);
cold.deep.push_back(frame{cur_idx, cur_count, cur_is_object});
}
}
++depth;
@@ -850,19 +870,21 @@ indent_done:
n.next = static_cast<std::uint32_t>(out - base) - cur_idx;
if (--depth != 0)
{
frame f{};
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
{
f = cold.shallow[depth - 1];
const frame& f = cold.shallow[depth - 1];
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
}
else
{
f = cold.deep.back();
const frame f = cold.deep.back();
cold.deep.pop_back();
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
}
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
}
}
@@ -991,12 +1013,13 @@ indent_done:
return true;
}
/// a string (value or key) at p
/// a string at p: a value (Value) or a key
template<bool Value>
NLOHMANN_VIEW_ALWAYS_INLINE bool string()
{
++p; // opening quote
const unsigned char* const s = p;
p = scan_string_run(p, e);
p = scan_string_run<Value>(p, e);
if (NLOHMANN_VIEW_LIKELY(p != e && *p == '"'))
{
emit(value_t::string, 0, 0, static_cast<std::size_t>(s - b), static_cast<std::uint64_t>(p - s));
+317
View File
@@ -0,0 +1,317 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // sort, stable_sort
#include <cstddef> // size_t
#include <string> // string
#include <utility> // move, pair
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
// Equality of views, and of views with basic_json values, with the semantics
// of basic_json's operator== applied to the values parse() would produce:
// numbers compare by value across their types, an object is compared by its
// members with duplicate keys resolved as parse() resolves them (the last
// value, at the position of the first occurrence), and in document order if
// the object type keeps an order (ordered_json), by key otherwise.
/// one side of a comparison: a view
template<typename BasicJsonType, typename View>
class view_side
{
public:
using string_view_t = typename View::string_view_t;
explicit view_side(const View& v) noexcept
: m_view(v)
{}
value_t type() const noexcept
{
return m_view.type();
}
std::size_t size() const noexcept
{
return m_view.size();
}
string_view_t string() const
{
return m_view.get_string();
}
/// a number, boolean, or null as a basic_json value (no allocation)
BasicJsonType scalar() const
{
switch (m_view.type())
{
case value_t::number_integer:
return BasicJsonType(m_view.template get<typename BasicJsonType::number_integer_t>());
case value_t::number_unsigned:
return BasicJsonType(m_view.template get<typename BasicJsonType::number_unsigned_t>());
case value_t::number_float:
return BasicJsonType(m_view.template get<typename BasicJsonType::number_float_t>());
case value_t::boolean:
return BasicJsonType(m_view.template get<bool>());
case value_t::null:
case value_t::object:
case value_t::array:
case value_t::string:
case value_t::binary:
case value_t::discarded:
default:
return BasicJsonType(nullptr);
}
}
void elements(std::vector<view_side>& out) const
{
out.reserve(m_view.size());
for (const View e : m_view)
{
out.emplace_back(e);
}
}
/// the members as parse() keeps them: one per key, the last value at the
/// position of the first occurrence; in that order, or sorted by key
void members(std::vector<std::pair<string_view_t, view_side>>& out, bool ordered) const
{
struct member
{
string_view_t key;
View value;
std::size_t position;
};
std::vector<member> all;
all.reserve(m_view.size());
std::size_t position = 0;
for (auto it = m_view.begin(); it != m_view.end(); ++it)
{
all.push_back(member{it.key(), it.value(), position++});
}
std::stable_sort(all.begin(), all.end(), [](const member & a, const member & b)
{
return a.key < b.key;
});
std::vector<member> unique;
unique.reserve(all.size());
for (std::size_t i = 0; i < all.size();)
{
std::size_t last = i;
while (last + 1 < all.size() && all[last + 1].key == all[i].key)
{
++last;
}
unique.push_back(member{all[i].key, all[last].value, all[i].position});
i = last + 1;
}
if (ordered)
{
std::sort(unique.begin(), unique.end(), [](const member & a, const member & b)
{
return a.position < b.position;
});
}
out.reserve(unique.size());
for (const member& m : unique)
{
out.emplace_back(m.key, view_side(m.value));
}
}
private:
View m_view;
};
/// the other side of a comparison: a basic_json value
template<typename BasicJsonType, typename StringView>
class json_side
{
public:
using string_view_t = StringView;
explicit json_side(const BasicJsonType& j) noexcept
: m_json(&j)
{}
value_t type() const noexcept
{
return m_json->type();
}
std::size_t size() const noexcept
{
return m_json->size();
}
string_view_t string() const
{
const auto& s = m_json->template get_ref<const typename BasicJsonType::string_t&>();
return string_view_t(s.data(), s.size());
}
BasicJsonType scalar() const
{
return *m_json;
}
void elements(std::vector<json_side>& out) const
{
out.reserve(m_json->size());
for (const auto& e : *m_json)
{
out.emplace_back(e);
}
}
void members(std::vector<std::pair<string_view_t, json_side>>& out, bool ordered) const
{
out.reserve(m_json->size());
for (auto it = m_json->cbegin(); it != m_json->cend(); ++it)
{
out.emplace_back(string_view_t(it.key().data(), it.key().size()), json_side(it.value()));
}
if (!ordered)
{
std::sort(out.begin(), out.end(), [](const std::pair<string_view_t, json_side>& a, const std::pair<string_view_t, json_side>& b)
{
return a.first < b.first;
});
}
}
private:
const BasicJsonType* m_json;
};
/// whether two sides are equal; iterative, so that the nesting depth is
/// limited by memory only
template<typename BasicJsonType, typename A, typename B>
bool equal(const A& a0, const B& b0)
{
using string_view_t = typename A::string_view_t;
const bool ordered = is_ordered_map<typename BasicJsonType::object_t>::value;
struct frame
{
std::vector<A> elements_a{};
std::vector<B> elements_b{};
std::vector<std::pair<string_view_t, A>> members_a{};
std::vector<std::pair<string_view_t, B>> members_b{};
bool object = false;
std::size_t next = 0;
};
std::vector<frame> stack;
A a = a0;
B b = b0;
for (;;)
{
const value_t ta = a.type();
const value_t tb = b.type();
const bool numbers = (ta == value_t::number_integer || ta == value_t::number_unsigned || ta == value_t::number_float)
&& (tb == value_t::number_integer || tb == value_t::number_unsigned || tb == value_t::number_float);
if (ta == value_t::discarded || tb == value_t::discarded)
{
// basic_json decides (JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON)
if (ta != tb || !(BasicJsonType(value_t::discarded) == BasicJsonType(value_t::discarded)))
{
return false;
}
}
else
{
if (!numbers && ta != tb)
{
return false;
}
if (ta == value_t::string)
{
if (!(a.string() == b.string()))
{
return false;
}
}
else if (ta == value_t::array || ta == value_t::object)
{
if (a.size() != b.size() && ta == value_t::array)
{
return false;
}
frame f;
f.object = ta == value_t::object;
if (f.object)
{
a.members(f.members_a, ordered);
b.members(f.members_b, ordered);
if (f.members_a.size() != f.members_b.size())
{
return false;
}
}
else
{
a.elements(f.elements_a);
b.elements(f.elements_b);
}
stack.push_back(std::move(f));
}
else if (!(a.scalar() == b.scalar())) // numbers (also of different types), null, boolean
{
return false;
}
}
// the next pair of values
for (;;)
{
if (stack.empty())
{
return true;
}
frame& f = stack.back();
const std::size_t count = f.object ? f.members_a.size() : f.elements_a.size();
if (f.next == count)
{
stack.pop_back();
continue;
}
if (f.object)
{
if (!(f.members_a[f.next].first == f.members_b[f.next].first))
{
return false;
}
a = f.members_a[f.next].second;
b = f.members_b[f.next].second;
}
else
{
a = f.elements_a[f.next];
b = f.elements_b[f.next];
}
++f.next;
break;
}
}
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+128 -1
View File
@@ -10,10 +10,14 @@
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy
#include <limits> // numeric_limits
#include <map> // map
#include <memory> // unique_ptr
#include <new> // bad_alloc, operator new, placement new
#include <string> // string
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
@@ -37,10 +41,44 @@ struct document_data
node* inline_tape = nullptr; ///< node array allocated together with this header
std::size_t inline_cap = 0;
std::string arena; ///< decoded strings that contained escapes
std::size_t arena_size = 0; ///< bytes of decoded strings at base[1] (the arena, or those of a loaded image)
std::string owned; ///< owned copy of the input, if any
std::array<const char*, 4> base = {{nullptr, nullptr, nullptr, nullptr}}; ///< string bases: source, arena (indexed by flags & node_flags::storage)
std::vector<std::uint8_t> owned_image; ///< a loaded image the document owns (the text and the decoded strings point into it)
// hash indexes of large objects (see object_index.hpp)
static constexpr std::uint32_t index_min_members = 128;
struct object_index
{
std::size_t start; ///< first slot in index_slots
std::uint32_t mask; ///< slot count - 1 (a power of two minus one)
};
std::vector<object_index> indexes;
std::vector<std::uint32_t> index_slots;
std::vector<std::uint32_t> large_objects; ///< positions of the objects to index (noted while parsing)
std::array<const char*, 4> base = {{nullptr, nullptr, nullptr, nullptr}}; ///< string bases: source, arena, edit arena (indexed by flags & node_flags::storage)
bool discarded = true;
/// The storage of edits (editable documents only; see edit_storage.hpp).
/// Edits never move or resize the parsed index, so views stay valid: an
/// array/object whose elements change gets node_flags::moved, and its
/// elements then live in a separate sequence (a header node, then the
/// entries), whose entries link to the values.
struct edit_state
{
std::vector<node*> moved{}; ///< element sequences of moved arrays/objects (header node first) // NOLINT(readability-redundant-member-init)
std::vector<std::size_t> moved_cap{}; ///< capacity in nodes of a growable block; 0: a fixed sequence (a new value) // NOLINT(readability-redundant-member-init)
std::vector<std::unique_ptr<node[]>> chunks{}; ///< storage of new values and blocks; never moved // NOLINT(readability-redundant-member-init,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
std::map<const node*, node*> regions{}; ///< new arrays/objects: root -> container that uses it as its element sequence (nullptr: linked from a block) // NOLINT(readability-redundant-member-init)
node* chunk_cur = nullptr;
node* chunk_end = nullptr;
std::size_t chunk_next = 64;
std::vector<std::unique_ptr<char[]>> texts{}; ///< edit arena, the current buffer last; earlier ones stay alive for string views // NOLINT(readability-redundant-member-init,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
std::size_t text_used = 0;
std::size_t text_cap = 0;
std::size_t bytes = 0; ///< memory held by edits
};
std::unique_ptr<edit_state> edits; ///< created by the first edit
/// one allocation for the header and room for `nodes` nodes; large
/// documents get a separate node array instead (so it can be trimmed)
static document_data* create(std::size_t nodes)
@@ -76,6 +114,11 @@ struct document_data
document_data() noexcept
: arena() // NOLINT(readability-redundant-member-init)
, owned() // NOLINT(readability-redundant-member-init)
, owned_image() // NOLINT(readability-redundant-member-init)
, indexes() // NOLINT(readability-redundant-member-init)
, index_slots() // NOLINT(readability-redundant-member-init)
, large_objects() // NOLINT(readability-redundant-member-init)
, edits() // NOLINT(readability-redundant-member-init)
{}
document_data(const document_data&) = delete;
document_data(document_data&&) = delete;
@@ -152,6 +195,90 @@ struct document_data
{
return n + n->next;
}
/// (editable documents) first element or key, also of a moved container
NLOHMANN_VIEW_ALWAYS_INLINE const node* first_child_edited(const node* n) const noexcept
{
return NLOHMANN_VIEW_LIKELY((n->flags & node_flags::moved) == 0) ? n + 1 : edits->moved[n->off] + 1;
}
/// (editable documents) end of the elements, also of a moved container
NLOHMANN_VIEW_ALWAYS_INLINE const node* child_end_edited(const node* n) const noexcept
{
if (NLOHMANN_VIEW_LIKELY((n->flags & node_flags::moved) == 0))
{
return n + n->next;
}
const node* const h = edits->moved[n->off];
return h + h->next;
}
/// (editable documents) the value at an element position: entries of
/// moved sequences are links. The link case is out of line, so that this
/// compiles to a predicted branch rather than a select that delays the
/// following loads.
static NLOHMANN_VIEW_ALWAYS_INLINE const node* deref(const node* n) noexcept
{
return NLOHMANN_VIEW_LIKELY(n->kind != kind_link) ? n : follow_link(n);
}
static NLOHMANN_VIEW_NOINLINE const node* follow_link(const node* n) noexcept
{
return link_target(*n);
}
};
/// How the index is walked: views of read-only documents follow the node
/// array alone and compile without any of the edit handling; views of
/// editable documents also follow moved element sequences and links.
template<bool Editable>
struct navigation
{
/// whether the walk follows edits (the template argument as a runtime
/// condition: `Editable && ...` is a constant condition for MSVC, C4127)
static NLOHMANN_VIEW_ALWAYS_INLINE bool editable() noexcept
{
return false;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* first(const document_data& /*d*/, const node* n) noexcept
{
return n + 1;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* end(const document_data& /*d*/, const node* n) noexcept
{
return n + n->next;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* value(const node* n) noexcept
{
return n;
}
};
template<>
struct navigation<true>
{
static NLOHMANN_VIEW_ALWAYS_INLINE bool editable() noexcept
{
return true;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* first(const document_data& d, const node* n) noexcept
{
return d.first_child_edited(n);
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* end(const document_data& d, const node* n) noexcept
{
return d.child_end_edited(n);
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* value(const node* n) noexcept
{
return document_data::deref(n);
}
};
} // namespace view
+883
View File
@@ -0,0 +1,883 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cmath> // isinf, isnan
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t, uint32_t, uint64_t
#include <cstring> // memcmp, memmove
#include <limits> // numeric_limits
#include <string> // string, to_string
#include <type_traits> // decay, enable_if, integral_constant, is_arithmetic, is_convertible, is_floating_point, is_same, is_signed
#include <utility> // forward
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/edit_storage.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/lookup.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
template<typename BasicJsonType, bool Editable>
class basic_json_view;
namespace detail
{
namespace view
{
/// the index of the first true condition (the number of conditions if none is)
template<bool... Conditions>
struct first_true : std::integral_constant<int, 0> {};
template<bool... Conditions>
struct first_true<false, Conditions...> : std::integral_constant < int, 1 + first_true<Conditions...>::value > {};
/// Checks a string the way basic_json's serializer does when it writes it
/// (type_error.316 with the same message), so that an editable document
/// only holds valid UTF-8: the error is at the first byte that no
/// well-formed sequence can continue with (Unicode, Table 3-7).
inline void check_utf8(const char* s, std::size_t n)
{
const auto* const p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const auto hex = [](unsigned char c)
{
constexpr const char* digits = "0123456789ABCDEF";
return std::string{digits[c >> 4u], digits[c & 0xFu]};
};
for (std::size_t i = 0; i < n;)
{
const unsigned char c = p[i];
if (c < 0x80)
{
++i;
continue;
}
std::size_t len = 0;
unsigned char lo = 0x80;
unsigned char hi = 0xBF;
if (c >= 0xC2 && c <= 0xDF)
{
len = 2;
}
else if (c >= 0xE0 && c <= 0xEF)
{
len = 3;
lo = c == 0xE0 ? 0xA0 : 0x80;
hi = c == 0xED ? 0x9F : 0xBF;
}
else if (c >= 0xF0 && c <= 0xF4)
{
len = 4;
lo = c == 0xF0 ? 0x90 : 0x80;
hi = c == 0xF4 ? 0x8F : 0xBF;
}
else
{
throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", hex(c)));
}
for (std::size_t k = 1; k < len; ++k)
{
if (i + k == n)
{
throw_type_error(316, concat("incomplete UTF-8 string; last byte: 0x", hex(p[n - 1])));
}
const unsigned char b = p[i + k];
if (b < (k == 1 ? lo : 0x80) || b > (k == 1 ? hi : 0xBF))
{
throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i + k), ": 0x", hex(b)));
}
}
i += len;
}
}
/*!
@brief the edits of an editable basic_json_document
Values are accepted as views (of any document), BasicJsonType values, and
everything BasicJsonType can be constructed from. The source text is never
written: new values go to storage owned by the document (see
edit_storage.hpp).
*/
template<typename BasicJsonType, typename View>
class editor
{
using number_integer_t = typename BasicJsonType::number_integer_t;
using number_unsigned_t = typename BasicJsonType::number_unsigned_t;
using number_float_t = typename BasicJsonType::number_float_t;
using string_t = typename BasicJsonType::string_t;
using string_view_t = typename View::string_view_t;
using nav = navigation<true>;
public:
explicit editor(document_data& d) noexcept
: m_doc(d)
{}
/// replace a value; returns its view
template<typename V>
View set(const View& target, V&& value)
{
node* const slot = own(target);
const encoded e = encode(std::forward<V>(value));
assign(slot, e, nullptr, false);
return View(&m_doc, slot);
}
/// set a member (appended if missing; a null value becomes an object);
/// returns a view of the member value
template<typename V>
View set(const View& object, string_view_t key, V&& value)
{
node* const o = own(object);
if (o->kind != static_cast<std::uint8_t>(value_t::object) && o->kind != static_cast<std::uint8_t>(value_t::null))
{
throw_type_error(305, "cannot use operator[] with a string argument with ", object.type_name());
}
check_utf8(key.data(), key.size());
const encoded e = encode(std::forward<V>(value));
if (o->kind == static_cast<std::uint8_t>(value_t::null))
{
become_empty(o, value_t::object);
}
// an existing member: assign it (and drop later duplicates, so that
// lookups, iteration, and materialize() agree)
node* slot = nullptr;
bool duplicates = false;
for (const node* k = nav::first(m_doc, o), *end = nav::end(m_doc, o); k != end; k = document_data::after(k + 1))
{
if (key_equals(*k, key))
{
if (slot != nullptr)
{
duplicates = true;
break;
}
slot = const_cast<node*>(nav::value(k + 1)); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to this document
}
}
if (slot != nullptr)
{
if (duplicates)
{
erase_members(o, key, true);
}
assign(slot, e, o, true);
return View(&m_doc, slot);
}
const node k = string_node(key.data(), key.size());
slot = new_slot(e);
node* const h = block_of(m_doc, o, 2);
h[h->next] = k;
make_link(h[h->next + 1], slot);
h->next += 2;
++h->len;
++o->len;
return View(&m_doc, slot);
}
/// assign an existing array element; returns a view of it
template<typename V>
View set(const View& array, std::size_t idx, V&& value)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(305, "cannot use operator[] with a numeric argument with ", array.type_name());
}
check_index(idx, a->len);
const encoded e = encode(std::forward<V>(value));
node* const slot = const_cast<node*>(nav::value(element_at<true>(m_doc, a, idx))); // NOLINT(cppcoreguidelines-pro-type-const-cast)
assign(slot, e, a, true);
return View(&m_doc, slot);
}
/// append to an array (a null value becomes an array); returns a view of
/// the new element
template<typename V>
View push_back(const View& array, V&& value)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array) && a->kind != static_cast<std::uint8_t>(value_t::null))
{
throw_type_error(308, "cannot use push_back() with ", array.type_name());
}
const encoded e = encode(std::forward<V>(value));
if (a->kind == static_cast<std::uint8_t>(value_t::null))
{
become_empty(a, value_t::array);
}
node* const slot = new_slot(e);
node* const h = block_of(m_doc, a, 1);
make_link(h[h->next], slot);
++h->next;
++h->len;
++a->len;
return View(&m_doc, slot);
}
/// insert into an array before position idx (idx <= size()); returns a
/// view of the new element
template<typename V>
View insert(const View& array, std::size_t idx, V&& value)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(309, "cannot use insert() with ", array.type_name());
}
check_index(idx, a->len + 1);
const encoded e = encode(std::forward<V>(value));
node* const slot = new_slot(e);
node* const h = block_of(m_doc, a, 1);
std::memmove(h + 2 + idx, h + 1 + idx, (h->next - 1 - idx) * sizeof(node));
make_link(h[1 + idx], slot);
++h->next;
++h->len;
++a->len;
return View(&m_doc, slot);
}
/// remove all members with this key; returns their number
std::size_t erase(const View& object, string_view_t key)
{
node* const o = own(object);
if (o->kind != static_cast<std::uint8_t>(value_t::object))
{
throw_type_error(307, "cannot use erase() with ", object.type_name());
}
for (const node* k = nav::first(m_doc, o), *end = nav::end(m_doc, o); k != end; k = document_data::after(k + 1))
{
if (key_equals(*k, key))
{
return erase_members(o, key, false);
}
}
return 0;
}
/// remove an array element
void erase(const View& array, std::size_t idx)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(307, "cannot use erase() with ", array.type_name());
}
check_index(idx, a->len);
node* const h = block_of(m_doc, a, 0);
std::memmove(h + 1 + idx, h + 2 + idx, (h->next - 2 - idx) * sizeof(node));
--h->next;
--h->len;
--a->len;
}
private:
/// an encoded value: a scalar node, or the root of a new array/object
struct encoded
{
node scalar{};
node* region = nullptr;
};
/// the node of a view of this document
node* own(const View& v)
{
if (NLOHMANN_VIEW_UNLIKELY(v.m_doc != &m_doc || v.m_node == nullptr))
{
throw_invalid_iterator(202, "view does not belong to this document");
}
edit_state_of(m_doc);
return const_cast<node*>(v.m_node); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to this document
}
static void check_index(std::size_t idx, std::size_t limit)
{
if (idx >= limit)
{
throw_out_of_range(401, concat("array index ", std::to_string(idx), " is out of range"));
}
}
bool key_equals(const node& k, string_view_t key) const noexcept
{
return k.len == key.size() && (key.size() == 0 || std::memcmp(m_doc.str(k), key.data(), key.size()) == 0);
}
/// remove the members with this key (all, or all but the first) from an object
std::size_t erase_members(node* o, string_view_t key, bool keep_first)
{
node* const h = block_of(m_doc, o, 0);
node* w = h + 1;
std::size_t erased = 0;
bool kept = false;
for (const node* r = h + 1, *end = h + h->next; r != end; r += 2)
{
const bool match = key_equals(*r, key);
if (match && (kept || !keep_first))
{
++erased;
continue;
}
kept = kept || match;
if (w != r)
{
w[0] = r[0];
w[1] = r[1];
}
w += 2;
}
h->next = static_cast<std::uint32_t>(w - h);
h->len -= static_cast<std::uint32_t>(erased);
o->len -= static_cast<std::uint32_t>(erased);
return erased;
}
/// turn a null into an empty array/object in place
static void become_empty(node* n, value_t k) noexcept
{
const std::uint8_t linked = n->flags & node_flags::linked;
*n = node{};
n->kind = static_cast<std::uint8_t>(k);
n->flags = static_cast<std::uint8_t>(node_flags::is_new | linked);
n->next = 1;
}
/// Replace the value at slot; `parent` is the container whose elements
/// include slot (if known).
void assign(node* slot, const encoded& e, node* parent, bool parent_known)
{
// an entry of a moved sequence links to the slot: it can take any extent
const std::uint8_t linked = slot->flags & node_flags::linked;
if (e.region == nullptr)
{
if (is_container(*slot) && slot->next > 1 && slot != m_doc.tape && linked == 0)
{
// The slot spans its old elements in the enclosing sequence, but
// a scalar is one node: the enclosing container first switches to
// links (then the extent of the slot no longer matters). Looking
// for the container is linear in the size of the document, so
// links (which are marked in the slot) avoid it.
node* const p = parent_known ? parent : find_parent(m_doc, slot);
if (p != nullptr && ((p->flags & node_flags::moved) == 0 || moved_capacity(m_doc, p) == 0))
{
block_of(m_doc, p, 0);
}
}
*slot = e.scalar;
slot->flags = static_cast<std::uint8_t>(slot->flags | linked);
return;
}
// an array/object: the slot keeps its extent (so that the enclosing
// sequence still steps over it), and the elements come from the new
// sequence
const node* const r = e.region;
const std::uint32_t extent = is_container(*slot) ? slot->next : 1;
const bool was_moved = (slot->flags & node_flags::moved) != 0;
// Everything that can throw happens before the slot changes: a slot
// that is a container without the moved flag would show its old
// elements. reserve_moved() makes the set_moved() below, which sets
// the flag, safe; the entry of `regions` exists already (encode()
// added it), so that the assignment at the end does not allocate.
if (!was_moved)
{
reserve_moved(m_doc);
}
slot->kind = r->kind;
slot->extra = 0;
slot->len = r->len;
slot->next = extent;
// (set_moved() adds the moved flag to a slot that does not have it yet)
slot->flags = static_cast<std::uint8_t>((was_moved ? node_flags::moved | node_flags::is_new : 0) | linked);
set_moved(m_doc, slot, e.region, 0);
edit_state_of(m_doc).regions[e.region] = slot;
}
/// a node for a new element (links point to it; it never moves)
node* new_slot(const encoded& e)
{
if (e.region != nullptr)
{
return e.region;
}
node* const s = alloc_nodes(m_doc, 1);
*s = e.scalar;
return s;
}
//////////////
// encoding //
//////////////
template<int N>
using encode_tag = std::integral_constant<int, N>;
template<typename T>
struct is_view : std::false_type {};
template<typename J, bool E>
struct is_view<basic_json_view<J, E>> : std::true_type {};
template<typename V>
encoded encode(V&& v)
{
using D = typename std::decay<V>::type;
return encode_impl(std::forward<V>(v), encode_tag<first_true<is_view<D>::value,
std::is_same<D, BasicJsonType>::value,
std::is_same<D, std::nullptr_t>::value,
std::is_same<D, bool>::value,
std::is_arithmetic<D>::value,
std::is_convertible<const D&, string_view_t>::value>::value> {});
}
/// a view of any document (copied; nothing is shared with it)
template<typename J, bool E>
encoded encode_impl(const basic_json_view<J, E>& v, encode_tag<0> /*view*/)
{
if (NLOHMANN_VIEW_UNLIKELY(v.m_node == nullptr))
{
throw_type_error(302, "type must be a value, but is ", "discarded");
}
encoded r;
if (!is_container(*v.m_node))
{
r.scalar = copy_scalar(*v.m_doc, *v.m_node);
return r;
}
r.region = alloc_nodes(m_doc, count_nodes<E>(*v.m_doc, v.m_node));
fill_nodes<E>(*v.m_doc, v.m_node, r.region);
edit_state_of(m_doc).regions.emplace(r.region, nullptr);
return r;
}
encoded encode_impl(const BasicJsonType& j, encode_tag<1> /*json*/)
{
encoded r;
if (!j.is_structured())
{
r.scalar = json_scalar(j);
return r;
}
r.region = alloc_nodes(m_doc, count_nodes(j));
fill_nodes(j, r.region);
edit_state_of(m_doc).regions.emplace(r.region, nullptr);
return r;
}
encoded encode_impl(std::nullptr_t /*unused*/, encode_tag<2> /*null*/)
{
encoded r;
r.scalar = plain_node(value_t::null);
return r;
}
encoded encode_impl(bool b, encode_tag<3> /*boolean*/)
{
encoded r;
r.scalar = plain_node(value_t::boolean);
r.scalar.flags = static_cast<std::uint8_t>(r.scalar.flags | (b ? node_flags::is_true : 0));
return r;
}
template<typename T>
encoded encode_impl(T x, encode_tag<4> /*number*/)
{
encoded r;
r.scalar = number_node(x, std::integral_constant<int, first_true<std::is_floating_point<T>::value, std::is_signed<T>::value>::value> {});
return r;
}
template<typename T>
encoded encode_impl(const T& s, encode_tag<5> /*string*/)
{
const string_view_t sv(s);
check_utf8(sv.data(), sv.size());
encoded r;
r.scalar = string_node(sv.data(), sv.size());
return r;
}
template<typename T>
encoded encode_impl(T&& x, encode_tag<6> /*other*/)
{
return encode_impl(BasicJsonType(std::forward<T>(x)), encode_tag<1> {});
}
static node plain_node(value_t k) noexcept
{
node n{};
n.kind = static_cast<std::uint8_t>(k);
n.flags = node_flags::is_new;
return n;
}
template<typename T>
node number_node(T x, std::integral_constant<int, 0> /*floating-point*/)
{
return float_node(static_cast<number_float_t>(x));
}
template<typename T>
node number_node(T x, std::integral_constant<int, 1> /*signed*/)
{
return integer_node(static_cast<std::uint64_t>(static_cast<std::int64_t>(x)), value_t::number_integer);
}
template<typename T>
node number_node(T x, std::integral_constant<int, 2> /*unsigned*/)
{
return integer_node(static_cast<std::uint64_t>(x), value_t::number_unsigned);
}
/// an integer with its canonical token in the edit arena
node integer_node(std::uint64_t bits, value_t k)
{
const bool negative = k == value_t::number_integer && static_cast<std::int64_t>(bits) < 0;
std::uint64_t magnitude = negative ? 0 - bits : bits;
std::array<char, 24> buf{};
char* p = buf.data() + buf.size(); // NOLINT(misc-const-correctness): digits are written through p
do
{
*--p = static_cast<char>('0' + (magnitude % 10));
magnitude /= 10;
}
while (magnitude != 0);
if (negative)
{
*--p = '-';
}
const auto len = static_cast<std::size_t>(buf.data() + buf.size() - p);
node n = plain_node(k);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
n.off = append_text(m_doc, p, len);
// number_length() adds one for the sign of number_integer nodes
n.extra = static_cast<std::uint16_t>(k == value_t::number_integer ? len - 1 : len);
set_integer_bits(n, bits);
return n;
}
/// a float with its shortest round-trip token (as basic_json::dump()
/// writes it), or nan, inf, -inf, in the edit arena
node float_node(number_float_t x)
{
string_t text;
if (std::isnan(x))
{
text = "nan";
}
else if (std::isinf(x))
{
text = x > 0 ? "inf" : "-inf";
}
else
{
text = BasicJsonType(x).dump();
}
node n = plain_node(value_t::number_float);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
n.extra = 0xFFFFu; // (the digit layout is not recorded)
n.off = append_text(m_doc, text.data(), text.size());
n.len = static_cast<std::uint32_t>(text.size());
return n;
}
/// a string (or key) in the edit arena
node string_node(const char* s, std::size_t len)
{
if (NLOHMANN_VIEW_UNLIKELY(len >= 0xFFFFFFFFu))
{
throw_out_of_range(416, "strings of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE
}
node n = plain_node(value_t::string);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
n.off = append_text(m_doc, s, len);
n.len = static_cast<std::uint32_t>(len);
return n;
}
/// a scalar of a view (of any document) as a node of this document
node copy_scalar(const document_data& from, const node& n)
{
if (&from == &m_doc)
{
return n; // the same storage
}
switch (static_cast<value_t>(n.kind))
{
case value_t::string:
// (an editable document only holds valid UTF-8, whatever the check of the other document was)
check_utf8(from.str(n), n.len);
return string_node(from.str(n), n.len);
case value_t::number_integer:
case value_t::number_unsigned:
return integer_node(integer_bits(n), static_cast<value_t>(n.kind));
case value_t::number_float:
{
node r = plain_node(value_t::number_float);
r.flags = static_cast<std::uint8_t>(r.flags | node_flags::edited);
r.off = append_text(m_doc, from.str(n), n.len);
r.len = n.len;
r.extra = n.extra;
return r;
}
case value_t::boolean:
{
node r = plain_node(value_t::boolean);
r.flags = static_cast<std::uint8_t>(r.flags | (n.flags & node_flags::is_true));
return r;
}
case value_t::null:
case value_t::object:
case value_t::array:
case value_t::binary:
case value_t::discarded:
default:
return plain_node(value_t::null);
}
}
node json_scalar(const BasicJsonType& j)
{
switch (j.type())
{
case value_t::null:
return plain_node(value_t::null);
case value_t::boolean:
{
node r = plain_node(value_t::boolean);
r.flags = static_cast<std::uint8_t>(r.flags | (j.template get<bool>() ? node_flags::is_true : 0));
return r;
}
case value_t::number_integer:
return integer_node(static_cast<std::uint64_t>(static_cast<std::int64_t>(j.template get<number_integer_t>())), value_t::number_integer);
case value_t::number_unsigned:
return integer_node(static_cast<std::uint64_t>(j.template get<number_unsigned_t>()), value_t::number_unsigned);
case value_t::number_float:
return float_node(j.template get<number_float_t>());
case value_t::string:
{
const auto& s = j.template get_ref<const string_t&>();
check_utf8(s.data(), s.size());
return string_node(s.data(), s.size());
}
case value_t::binary:
throw_type_error(319, "cannot store a binary value in a json_document", "");
case value_t::discarded:
case value_t::object:
case value_t::array:
default:
throw_type_error(302, "type must be a value, but is ", "discarded");
}
}
// The subtrees are walked with an explicit stack (as materialize() does):
// the nesting depth is limited by memory only, not by the call stack.
/// number of nodes of a subtree (containers, keys, scalars)
template<bool E>
static std::size_t count_nodes(const document_data& d, const node* n)
{
using walk = navigation<E>;
struct frame
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
};
std::vector<frame> open;
std::size_t r = 0;
for (;;)
{
++r;
if (is_container(*n))
{
open.push_back(frame{walk::first(d, n), walk::end(d, n), n->kind == static_cast<std::uint8_t>(value_t::object)});
}
// the next value: close finished containers, then step over the key
for (;;)
{
if (open.empty())
{
return r;
}
frame& f = open.back();
if (f.pos == f.end)
{
open.pop_back();
continue;
}
const node* v = f.pos;
if (f.object)
{
++r; // the key
++v;
}
f.pos = document_data::after(v);
n = walk::value(v);
break;
}
}
}
/// copy a subtree (of any document) as a contiguous sequence of
/// count_nodes() nodes
template<bool E>
void fill_nodes(const document_data& d, const node* n, node* out)
{
using walk = navigation<E>;
struct frame
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
node* self; ///< the container in the copy
};
std::vector<frame> open;
for (;;)
{
if (is_container(*n))
{
node* const self = out++;
*self = plain_node(static_cast<value_t>(n->kind));
self->len = n->len;
open.push_back(frame{walk::first(d, n), walk::end(d, n), n->kind == static_cast<std::uint8_t>(value_t::object), self});
}
else
{
*out++ = copy_scalar(d, *n);
}
// the next value: close finished containers, then copy the key
for (;;)
{
if (open.empty())
{
return;
}
frame& f = open.back();
if (f.pos == f.end)
{
f.self->next = static_cast<std::uint32_t>(out - f.self);
open.pop_back();
continue;
}
const node* v = f.pos;
if (f.object)
{
*out++ = copy_scalar(d, *v);
++v;
}
f.pos = document_data::after(v);
n = walk::value(v);
break;
}
}
}
static std::size_t count_nodes(const BasicJsonType& j)
{
using iterator = typename BasicJsonType::const_iterator;
struct frame
{
iterator pos;
iterator end;
bool object;
};
std::vector<frame> open;
const BasicJsonType* n = &j;
std::size_t r = 0;
for (;;)
{
++r;
if (n->is_structured())
{
open.push_back(frame{n->cbegin(), n->cend(), n->is_object()});
}
for (;;)
{
if (open.empty())
{
return r;
}
frame& f = open.back();
if (f.pos == f.end)
{
open.pop_back();
continue;
}
r += f.object ? 1 : 0; // the key
n = &*f.pos;
++f.pos;
break;
}
}
}
void fill_nodes(const BasicJsonType& j, node* out)
{
using iterator = typename BasicJsonType::const_iterator;
struct frame
{
iterator pos;
iterator end;
bool object;
node* self; ///< the container in the copy
};
std::vector<frame> open;
const BasicJsonType* n = &j;
for (;;)
{
if (n->is_structured())
{
node* const self = out++;
*self = plain_node(n->type());
self->len = static_cast<std::uint32_t>(n->size());
open.push_back(frame{n->cbegin(), n->cend(), n->is_object(), self});
}
else
{
*out++ = json_scalar(*n);
}
for (;;)
{
if (open.empty())
{
return;
}
frame& f = open.back();
if (f.pos == f.end)
{
f.self->next = static_cast<std::uint32_t>(out - f.self);
open.pop_back();
continue;
}
if (f.object)
{
const auto& key = f.pos.key();
check_utf8(key.data(), key.size());
*out++ = string_node(key.data(), key.size());
}
n = &*f.pos;
++f.pos;
break;
}
}
}
document_data& m_doc;
};
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
@@ -0,0 +1,266 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // max, min
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy
#include <functional> // less
#include <memory> // unique_ptr
#include <utility> // move
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
// The storage of edits. Edits never move or resize the parsed index: every
// value keeps its node, so views stay valid. New values and element sequences
// live in chunks that never move; strings and number tokens written by edits
// live in the edit arena. An array/object whose elements change gets
// node_flags::moved: its elements then live in a separate sequence (a header
// node, then the entries), whose entries link to the values (kind_link).
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
inline document_data::edit_state& edit_state_of(document_data& d)
{
if (!d.edits)
{
d.edits.reset(new document_data::edit_state()); // NOLINT(cppcoreguidelines-owning-memory,modernize-make-unique): owned by the unique_ptr; std::make_unique is C++14
}
return *d.edits;
}
/// k consecutive nodes that never move (new values and blocks)
inline node* alloc_nodes(document_data& d, std::size_t k)
{
document_data::edit_state& e = edit_state_of(d);
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(e.chunk_end - e.chunk_cur) < k))
{
const std::size_t count = (std::max)(k, e.chunk_next);
std::unique_ptr<node[]> fresh(new node[count]()); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
e.chunks.push_back(std::move(fresh));
e.chunk_cur = e.chunks.back().get();
e.chunk_end = e.chunk_cur + count;
e.chunk_next = (std::min)(e.chunk_next * 2, std::size_t{65536});
e.bytes += count * sizeof(node);
}
node* const r = e.chunk_cur;
e.chunk_cur += k;
return r;
}
/// The capacity of the edit arena after it grows by n bytes (`used` of `cap`
/// are taken): doubled, or what is needed plus some room, but never more than
/// the 4 GiB - 1 bytes that the 32-bit offsets of nodes can address. An error
/// if n more bytes do not fit even then.
inline std::size_t text_capacity(std::size_t cap, std::size_t used, std::size_t n)
{
constexpr std::size_t limit = 0xFFFFFFFFu;
if (NLOHMANN_VIEW_UNLIKELY(used > limit || n > limit - used))
{
throw_out_of_range(416, "edits of 4 GiB or more are not supported by json_document");
}
const std::size_t needed = used + n;
const std::size_t wanted = needed + (std::min)(limit - needed, std::size_t{256});
const std::size_t doubled = cap > limit / 2 ? limit : cap * 2;
return (std::max)(doubled, wanted);
}
/// copy n bytes into the edit arena and return their offset; a new buffer
/// leaves the old one alive, so that string views into it remain valid
inline std::uint32_t append_text(document_data& d, const char* s, std::size_t n)
{
document_data::edit_state& e = edit_state_of(d);
if (NLOHMANN_VIEW_UNLIKELY(e.text_cap - e.text_used < n))
{
const std::size_t cap = text_capacity(e.text_cap, e.text_used, n);
std::unique_ptr<char[]> fresh(new char[cap]); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
if (e.text_used != 0)
{
std::memcpy(fresh.get(), e.texts.back().get(), e.text_used);
}
e.texts.push_back(std::move(fresh));
e.text_cap = cap;
e.bytes += cap;
d.base[2] = e.texts.back().get();
}
const auto off = static_cast<std::uint32_t>(e.text_used);
if (n != 0)
{
std::memcpy(e.texts.back().get() + e.text_used, s, n);
}
e.text_used += n;
return off;
}
/// the capacity in nodes of the block of a moved container (0: a fixed
/// sequence, the elements of a new value)
inline std::size_t moved_capacity(const document_data& d, const node* n) noexcept
{
return d.edits->moved_cap[n->off];
}
/// Make room for one more moved container. This is the part of set_moved()
/// that can throw: a caller that changes a node before it calls set_moved()
/// calls this first, so that a failure leaves the node as it was.
inline void reserve_moved(document_data& d)
{
document_data::edit_state& e = edit_state_of(d);
if (e.moved.size() >= 0xFFFFFFFFu)
{
throw_out_of_range(416, "more than 4294967295 edited arrays and objects are not supported by json_document"); // LCOV_EXCL_LINE
}
if (e.moved.size() == e.moved.capacity() || e.moved_cap.size() == e.moved_cap.capacity())
{
// both grow before either changes, so that the push_backs cannot throw
e.moved.reserve((2 * e.moved.size()) + 16);
e.moved_cap.reserve((2 * e.moved.size()) + 16);
}
}
/// let container n take its elements from `seq` (header node first); cannot
/// throw if n is moved already or reserve_moved() was called
inline void set_moved(document_data& d, node* n, node* seq, std::size_t cap)
{
document_data::edit_state& e = edit_state_of(d);
if ((n->flags & node_flags::moved) != 0)
{
e.moved[n->off] = seq;
e.moved_cap[n->off] = cap;
return;
}
reserve_moved(d);
e.moved.push_back(seq);
e.moved_cap.push_back(cap);
n->off = static_cast<std::uint32_t>(e.moved.size() - 1);
n->flags = static_cast<std::uint8_t>(n->flags | node_flags::moved | node_flags::is_new);
}
/// Make the elements of container n a growable block with room for `extra`
/// more nodes, and return its header. The entries link to the existing
/// values, which stay where they are. A block that grows is copied (its old
/// space is not reused).
inline node* block_of(document_data& d, node* n, std::size_t extra)
{
if ((n->flags & node_flags::moved) != 0 && moved_capacity(d, n) != 0)
{
node* const h = d.edits->moved[n->off];
if (h->next + extra <= moved_capacity(d, n))
{
return h;
}
const std::size_t cap = (std::max)(2 * moved_capacity(d, n), h->next + extra);
node* const nh = alloc_nodes(d, cap);
std::memcpy(nh, h, h->next * sizeof(node));
set_moved(d, n, nh, cap);
return nh;
}
reserve_moved(d); // (so that set_moved() below cannot throw: the links are marked before)
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
const std::size_t used = 1 + (static_cast<std::size_t>(n->len) * (object ? 2 : 1));
const std::size_t cap = used + extra;
node* const h = alloc_nodes(d, cap);
*h = node{};
h->kind = n->kind;
h->len = n->len;
h->next = static_cast<std::uint32_t>(used);
node* o = h + 1;
for (const node* c = d.first_child_edited(n), *e = d.child_end_edited(n); c != e;)
{
if (object)
{
*o++ = *c++; // the key
}
make_link(*o, const_cast<node*>(document_data::deref(c))); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to the document
++o;
c = document_data::after(c);
}
set_moved(d, n, h, cap);
return h;
}
/// The container whose elements include `target`; nullptr for the root, for
/// a value that is no longer part of the document, and for a value that is
/// only reached through a link. Values never move between allocations, so
/// the path to `target` stays inside the allocation that holds it (the parsed
/// index, or one new value), where the extent of each container (`next`)
/// still covers its original subtree.
inline node* find_parent(const document_data& d, const node* target)
{
const std::less<const node*> lt{};
const node* lo = d.tape;
const node* hi = d.tape + d.tape_size;
const node* c = d.tape;
if (lt(target, lo) || !lt(target, hi))
{
if (!d.edits)
{
return nullptr; // LCOV_EXCL_LINE (nodes outside the index exist only after edits)
}
auto it = d.edits->regions.upper_bound(target);
if (it == d.edits->regions.begin())
{
return nullptr; // LCOV_EXCL_LINE (an array/object with elements is in the index or a new value)
}
--it;
lo = it->first;
hi = lo + lo->next;
if (!lt(target, hi))
{
return nullptr; // LCOV_EXCL_LINE (a single-node value, reached through a link)
}
// the root of a new value is the element sequence of its owner, or a linked value
c = it->second != nullptr ? it->second : lo;
}
if (target == lo)
{
return nullptr;
}
for (;;)
{
if (!is_container(*c))
{
return nullptr; // LCOV_EXCL_LINE (the value is inside c)
}
const bool object = c->kind == static_cast<std::uint8_t>(value_t::object);
const node* down = nullptr;
for (const node* p = d.first_child_edited(c), *e = d.child_end_edited(c); p != e;)
{
const node* const at = object ? p + 1 : p;
const node* const v = document_data::deref(at);
if (v == target)
{
return const_cast<node*>(c); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to the document
}
if (is_container(*v) && !lt(v, lo) && lt(v, target) && lt(target, v + v->next))
{
down = v;
break;
}
p = document_data::after(at);
}
if (down == nullptr)
{
return nullptr;
}
c = down;
}
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+5
View File
@@ -30,6 +30,11 @@ namespace view
NLOHMANN_VIEW_THROW(type_error::create(id, concat(prefix, type), nullptr));
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(int id, const std::string& msg)
{
NLOHMANN_VIEW_THROW(type_error::create(id, msg, nullptr));
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_out_of_range(int id, const std::string& msg)
{
NLOHMANN_VIEW_THROW(out_of_range::create(id, msg, nullptr));
+755
View File
@@ -0,0 +1,755 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // sort
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t, uint16_t, uint32_t, uint64_t
#include <cstring> // memcmp, memcpy
#include <limits> // numeric_limits
#include <string> // string
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/number.hpp>
#include <nlohmann/detail/view/object_index.hpp>
#include <nlohmann/detail/view/scan.hpp>
// Images: a document stored so that loading it needs no parsing.
//
// Layout (little-endian): a 64-byte header, the nodes, the text (the source,
// followed by the number tokens written by edits), a NUL, the decoded strings
// (followed by the strings written by edits), a NUL. The idea is that of
// zero-copy formats such as FlatBuffers (https://github.com/google/flatbuffers)
// and YaFF (https://github.com/yandex/yaff); no code is taken from them.
// check_image follows the idea of FlatBuffers' Verifier (bounds and
// structure) and also checks what the parser guarantees about strings and
// numbers, so that reading and serializing a checked image is safe and yields
// valid JSON.
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// how load() checks an image
enum class image_check
{
/// everything the parser guarantees: structure and bounds, strings (valid
/// UTF-8; source strings without quotes, backslashes, and control
/// characters), and numbers (well-formed, matching the stored values)
full,
/// structure and bounds only: reading and serializing are safe, but a
/// crafted image can yield invalid UTF-8, strings that serialize to
/// invalid JSON, or numbers that differ from their text
bounds,
/// none: for images from a trusted source only (a damaged image is
/// undefined behavior)
none,
};
struct image_header
{
std::array<char, 4> magic; ///< "NJVI"
std::uint32_t version; ///< 1
std::uint64_t node_count;
std::uint64_t text_size;
std::uint64_t arena_size;
std::array<std::uint64_t, 4> reserved; ///< zero (for later versions)
};
static_assert(sizeof(image_header) == 64, "the image header must be 64 bytes");
constexpr std::uint32_t image_version = 1;
/// the largest node count and text or string size of an image (as for parsed
/// documents, offsets and counts must fit 32 bits)
constexpr std::uint64_t image_limit = 0xFFFFFFF0u;
/// Copy the current structure of an edited document into nodes in document
/// order, as the parser would have written them. Text written by edits is
/// appended to text_tail (number tokens) and arena_tail (strings); floats that
/// are not finite become null, as dump() writes them.
inline void compact_nodes(const document_data& d, std::size_t arena_size, std::vector<node>& out, std::string& text_tail, std::string& arena_tail)
{
struct frame
{
const node* cur;
const node* end;
std::size_t index; ///< the container's node in out
std::uint32_t count;
bool object;
};
std::vector<frame> stack;
const auto string_node = [&](const node & s)
{
node r = s;
r.extra = 0;
r.flags = static_cast<std::uint8_t>(s.flags & node_flags::storage);
if (r.flags == node_flags::edited)
{
r.off = static_cast<std::uint32_t>(arena_size + arena_tail.size());
arena_tail.append(d.str(s), s.len);
r.flags = node_flags::escaped;
}
return r;
};
const auto emit = [&](const node * v)
{
node r = *v;
switch (static_cast<value_t>(v->kind))
{
case value_t::object:
case value_t::array:
r.flags = 0;
r.extra = 0;
r.off = (v->flags & (node_flags::moved | node_flags::is_new)) != 0 ? 0 : v->off;
r.len = 0; // counted below
r.next = 0; // set when the container is complete
stack.push_back(frame{d.first_child_edited(v), d.child_end_edited(v), out.size(), 0, v->kind == static_cast<std::uint8_t>(value_t::object)});
break;
case value_t::string:
r = string_node(*v);
break;
case value_t::number_integer:
case value_t::number_unsigned:
if ((v->flags & node_flags::storage) == node_flags::edited)
{
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(d.str(*v), number_length(*v));
}
r.flags = 0;
break;
case value_t::number_float:
if ((v->flags & node_flags::storage) == node_flags::edited)
{
const char* const t = d.str(*v);
if (t[0] == 'n' || t[0] == 'i' || (v->len > 1 && t[1] == 'i'))
{
r = node{}; // nan and infinity: null, as dump() writes them
r.kind = static_cast<std::uint8_t>(value_t::null);
break;
}
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(t, v->len);
r.extra = 0xFFFFu; // the digit layout is not recorded
}
r.flags = 0;
break;
case value_t::boolean:
r.flags = static_cast<std::uint8_t>(v->flags & node_flags::is_true);
break;
case value_t::null:
case value_t::binary:
case value_t::discarded:
default:
r.flags = 0;
break;
}
out.push_back(r);
};
emit(d.tape);
while (!stack.empty())
{
frame& top = stack.back();
if (top.cur == top.end)
{
node& c = out[top.index];
c.len = top.count;
c.next = static_cast<std::uint32_t>(out.size() - top.index);
stack.pop_back();
continue;
}
++top.count;
const node* v = nullptr;
if (top.object)
{
out.push_back(string_node(*top.cur));
v = document_data::deref(top.cur + 1);
top.cur = document_data::after(top.cur + 1);
}
else
{
v = document_data::deref(top.cur);
top.cur = document_data::after(top.cur);
}
emit(v); // may grow the stack (top is not used afterwards)
}
}
/// the document as an image
inline std::vector<std::uint8_t> save_image(const document_data& d)
{
#if !NLOHMANN_VIEW_LITTLE_ENDIAN
throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
#endif
const std::size_t arena_size = d.arena_size;
const node* nodes = d.tape;
std::size_t count = d.tape_size;
std::vector<node> compacted;
std::string text_tail;
std::string arena_tail;
if (d.edits)
{
compact_nodes(d, arena_size, compacted, text_tail, arena_tail);
nodes = compacted.data();
count = compacted.size();
}
const std::size_t text_size = d.size + text_tail.size();
const std::size_t total_arena = arena_size + arena_tail.size();
if (NLOHMANN_VIEW_UNLIKELY(text_size >= image_limit || total_arena >= image_limit || count >= image_limit))
{
// LCOV_EXCL_START (4 GiB)
throw_out_of_range(416, "images of 4 GiB or more are not supported by json_document");
// LCOV_EXCL_STOP
}
image_header h{};
h.magic = {{'N', 'J', 'V', 'I'}};
h.version = image_version;
h.node_count = count;
h.text_size = text_size;
h.arena_size = total_arena;
std::vector<std::uint8_t> image(sizeof(h) + (count * sizeof(node)) + text_size + 1 + total_arena + 1);
std::uint8_t* o = image.data();
std::memcpy(o, &h, sizeof(h));
o += sizeof(h);
std::memcpy(o, nodes, count * sizeof(node));
// the hash indexes are rebuilt by load()
for (std::size_t i = 0; i < count; ++i)
{
if (nodes[i].kind == static_cast<std::uint8_t>(value_t::object) && nodes[i].extra != 0)
{
node n = nodes[i];
n.extra = 0;
std::memcpy(o + (i * sizeof(node)), &n, sizeof(node));
}
}
o += count * sizeof(node);
const auto append = [&o](const char* s, std::size_t n)
{
if (n != 0)
{
std::memcpy(o, s, n);
o += n;
}
};
append(d.src, d.size);
append(text_tail.data(), text_tail.size());
*o++ = 0;
append(d.base[1], arena_size);
append(arena_tail.data(), arena_tail.size());
*o = 0;
return image;
}
/// the parts of a number token that the checks need
struct number_token
{
const unsigned char* int_start;
std::size_t int_digits;
std::size_t frac_digits;
std::int64_t exponent;
bool negative;
bool is_float; ///< a fraction or an exponent
};
/// whether [s, s + len) is a JSON number (the grammar the parser accepts)
inline bool scan_number_token(const unsigned char* s, std::size_t len, number_token& t)
{
const unsigned char* const e = s + len;
const unsigned char* p = s;
t.negative = p != e && *p == '-';
p += t.negative ? 1 : 0;
t.int_start = p;
if (p == e)
{
return false;
}
if (*p == '0')
{
++p;
}
else if (*p >= '1' && *p <= '9')
{
while (p != e && is_digit(*p))
{
++p;
}
}
else
{
return false;
}
t.int_digits = static_cast<std::size_t>(p - t.int_start);
t.frac_digits = 0;
t.is_float = false;
if (p != e && *p == '.')
{
const unsigned char* const f0 = ++p;
while (p != e && is_digit(*p))
{
++p;
}
if (p == f0)
{
return false;
}
t.frac_digits = static_cast<std::size_t>(p - f0);
t.is_float = true;
}
t.exponent = 0;
if (p != e && (*p | 0x20u) == 'e')
{
++p;
const bool exp_negative = p != e && *p == '-';
p += (p != e && (*p == '+' || *p == '-')) ? 1 : 0;
if (p == e || !is_digit(*p))
{
return false;
}
while (p != e && is_digit(*p))
{
t.exponent = t.exponent < 100000 ? (t.exponent * 10) + (*p - '0') : t.exponent;
++p;
}
t.exponent = exp_negative ? -t.exponent : t.exponent;
t.is_float = true;
}
return p == e;
}
/// the digit layout the parser records for a float token (compaction
/// writes "many" instead; the caller accepts both)
inline std::uint16_t float_layout(const number_token& t)
{
return static_cast<std::uint16_t>((t.int_digits < 255 ? t.int_digits : 255) | ((t.frac_digits < 255 ? t.frac_digits : 255) << 8u));
}
/// whether a float token (well-formed, per scan_number_token) is finite as
/// double; parse() rejects floats that overflow, and as there, only a number
/// whose magnitude could reach 1e308 needs the conversion
inline bool float_token_finite(const unsigned char* s, std::size_t len, const number_token& t)
{
if (static_cast<std::int64_t>(t.int_digits) + t.exponent > 300)
{
node n{};
n.kind = static_cast<std::uint8_t>(value_t::number_float);
n.len = static_cast<std::uint32_t>(len);
const auto v = float_value<double>(reinterpret_cast<const char*>(s), n); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return v <= (std::numeric_limits<double>::max)() && v >= -(std::numeric_limits<double>::max)();
}
return true;
}
/// whether an integer node matches its token the way the parser records it
/// (after the bounds check; the token has at most 256 characters)
inline bool check_integer(const node& n, const unsigned char* text)
{
const std::size_t len = number_length(n);
const unsigned char* const s = text + n.off;
number_token t{};
if (!scan_number_token(s, len, t))
{
return false;
}
// the token's value is the stored one; number_integer nodes of edits can
// be non-negative (as basic_json keeps the type of a value)
const bool integer = n.kind == static_cast<std::uint8_t>(value_t::number_integer);
if (t.is_float || t.int_digits > 20 || (t.negative && !integer))
{
return false;
}
// (at most 19 digits cannot overflow; 20 digits are compared with 2^64 - 1)
if (t.int_digits == 20 && std::memcmp(t.int_start, "18446744073709551615", 20) > 0)
{
return false;
}
std::uint64_t m = 0;
for (const unsigned char* d = t.int_start; d != t.int_start + t.int_digits; ++d)
{
m = (m * 10) + static_cast<std::uint64_t>(*d - '0');
}
if (integer && m > (t.negative ? std::uint64_t{1} << 63u : (std::uint64_t{1} << 63u) - 1))
{
return false;
}
return integer_bits(n) == (t.negative ? 0 - m : m);
}
/// a byte range of the text or of the decoded strings
struct byte_range
{
std::uint32_t off;
std::uint32_t len;
};
/// a float token and the digit layout its node records
struct float_range
{
std::uint32_t off;
std::uint32_t len;
std::uint16_t extra;
};
/// Run check once for each distinct range of ranges (which are within bounds
/// and not empty). Ranges that are not identical must not overlap: save()
/// writes every string and token to a place of its own, and nodes that share
/// a value (copies within a document) share the whole range. This bounds the
/// work by the size of the text, however many nodes point to the same bytes.
template<typename Check>
bool check_distinct_ranges(std::vector<byte_range>& ranges, Check check)
{
std::sort(ranges.begin(), ranges.end(), [](const byte_range & a, const byte_range & b)
{
return a.off != b.off ? a.off < b.off : a.len < b.len;
});
std::size_t end = 0;
for (std::size_t i = 0; i < ranges.size(); ++i)
{
const byte_range r = ranges[i];
if (i != 0 && r.off == ranges[i - 1].off && r.len == ranges[i - 1].len)
{
continue;
}
if (r.off < end || !check(r))
{
return false;
}
end = static_cast<std::size_t>(r.off) + r.len;
}
return true;
}
/// Check the float nodes: each token once (nodes of the same token must
/// record layouts that match it), tokens must not overlap.
inline bool check_float_ranges(std::vector<float_range>& ranges, const unsigned char* text)
{
std::sort(ranges.begin(), ranges.end(), [](const float_range & a, const float_range & b)
{
if (a.off != b.off)
{
return a.off < b.off;
}
if (a.len != b.len)
{
return a.len < b.len;
}
return a.extra < b.extra;
});
std::size_t end = 0;
std::size_t i = 0;
while (i < ranges.size())
{
const float_range r = ranges[i];
if (r.off < end)
{
return false;
}
number_token t{};
const unsigned char* const s = text + r.off;
if (!scan_number_token(s, r.len, t) || !float_token_finite(s, r.len, t))
{
return false;
}
// the digit layout the parser records (or "many", as compaction writes it)
const std::uint16_t layout = float_layout(t);
for (; i < ranges.size() && ranges[i].off == r.off && ranges[i].len == r.len; ++i)
{
if (ranges[i].extra != layout && ranges[i].extra != 0xFFFFu)
{
return false;
}
}
end = static_cast<std::size_t>(r.off) + r.len;
}
return true;
}
/// the content checks of check_image: source strings as the parser leaves
/// them (no quotes, backslashes, or control characters), decoded strings
/// (valid UTF-8), and float tokens
inline bool check_contents(const unsigned char* text, const unsigned char* arena,
std::vector<byte_range>& source_strings, std::vector<byte_range>& decoded_strings,
std::vector<float_range>& floats)
{
return check_distinct_ranges(source_strings, [text](const byte_range & r)
{
const unsigned char* const b = text + r.off;
return scan_string_run(b, b + r.len) == b + r.len;
})
&& check_distinct_ranges(decoded_strings, [arena](const byte_range & r)
{
return valid_utf8_prefix(arena + r.off, r.len) == r.len;
})
&& check_float_ranges(floats, text);
}
/// Check the nodes of a loaded image against its text and decoded strings:
/// kinds, flags, and `extra`; extents and element counts of arrays and
/// objects; keys; bounds; string contents (source strings as the parser
/// leaves them: no quotes, backslashes, or control characters; all strings
/// valid UTF-8); and number tokens.
///
/// The structure and the bounds are checked node by node. The contents of
/// strings and of float tokens are checked afterwards, once for each distinct
/// range (see check_distinct_ranges), so that the full check is linear in the
/// size of the image plus the sorting of the ranges, and not in the number of
/// nodes times the size of the text.
inline bool check_image(const node* nodes, std::size_t count, const unsigned char* text, std::size_t text_size,
const unsigned char* arena, std::size_t arena_size, bool full)
{
struct frame
{
std::size_t end;
std::uint32_t len;
std::uint32_t seen;
bool object;
bool expect_key;
};
std::vector<frame> stack;
std::vector<byte_range> source_strings;
std::vector<byte_range> decoded_strings;
std::vector<float_range> floats;
const auto check_string = [&](const node & n) -> bool
{
if ((n.flags & ~node_flags::escaped) != 0 || n.extra != 0)
{
return false;
}
const bool decoded = (n.flags & node_flags::escaped) != 0;
const std::size_t limit = decoded ? arena_size : text_size;
if (n.off > limit || n.len > limit - n.off)
{
return false;
}
if (full && n.len != 0)
{
(decoded ? decoded_strings : source_strings).push_back(byte_range{n.off, n.len});
}
return true;
};
// bounds of a number token; the recorded digit layout must lie within it
const auto number_in_bounds = [&](const node & n) -> bool
{
const std::size_t len = number_length(n);
if (len == 0 || n.off > text_size || len > text_size - n.off)
{
return false;
}
if (n.kind != static_cast<std::uint8_t>(value_t::number_float))
{
return (n.extra >> 8u) == 0;
}
// float_value() reads the sign, the integer digits, and the point and
// fraction digits the layout records (a layout of more than 19 digits
// means the general conversion, which stays within the token)
const std::size_t int_digits = n.extra & 0xFFu;
const std::size_t frac_digits = n.extra >> 8u;
const std::size_t need = (text[n.off] == '-' ? 1u : 0u) + int_digits + (frac_digits != 0 ? frac_digits + 1 : 0);
return int_digits + frac_digits > 19 || need <= len;
};
std::size_t i = 0;
for (;;)
{
// close finished arrays and objects
while (!stack.empty() && i == stack.back().end)
{
const frame f = stack.back();
if (f.seen != f.len || (f.object && !f.expect_key))
{
return false;
}
stack.pop_back();
if (!stack.empty())
{
++stack.back().seen;
stack.back().expect_key = true;
}
}
if (i == count)
{
return stack.empty() && (!full || check_contents(text, arena, source_strings, decoded_strings, floats));
}
if (i != 0 && stack.empty())
{
return false; // nodes after the root
}
const node& n = nodes[i];
if (!stack.empty() && stack.back().object && stack.back().expect_key)
{
if (n.kind != static_cast<std::uint8_t>(value_t::string) || !check_string(n))
{
return false;
}
stack.back().expect_key = false;
++i;
continue;
}
bool complete = true;
switch (static_cast<value_t>(n.kind))
{
case value_t::null:
// (the offset of a literal is read to size the output of dump())
if (n.flags != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
break;
case value_t::boolean:
if ((n.flags & ~node_flags::is_true) != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
break;
case value_t::string:
if (!check_string(n))
{
return false;
}
break;
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
if (n.flags != 0 || !number_in_bounds(n))
{
return false;
}
if (full)
{
// (the contents of float tokens are checked later; the token of an
// integer has at most 256 characters)
if (n.kind == static_cast<std::uint8_t>(value_t::number_float))
{
floats.push_back(float_range{n.off, n.len, n.extra});
}
else if (!check_integer(n, text))
{
return false;
}
}
break;
case value_t::array:
case value_t::object:
{
const std::size_t limit = stack.empty() ? count : stack.back().end;
if (n.flags != 0 || n.extra != 0 || n.next == 0 || n.next > limit - i || n.off > text_size)
{
return false;
}
stack.push_back(frame{i + n.next, n.len, 0, n.kind == static_cast<std::uint8_t>(value_t::object), true});
complete = false;
break;
}
case value_t::binary:
case value_t::discarded:
default:
return false;
}
++i;
if (complete && !stack.empty())
{
++stack.back().seen;
stack.back().expect_key = true;
}
}
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_image(const char* what)
{
throw_parse_error(116, concat("invalid json_document image: ", what));
}
/// Read an image into d. The text and the decoded strings stay in the image;
/// the nodes are copied (so that they are aligned, and edits can change them).
inline void load_image(document_data& d, const std::uint8_t* image, std::size_t size, image_check check)
{
#if !NLOHMANN_VIEW_LITTLE_ENDIAN
throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
#endif
if (image == nullptr || size < sizeof(image_header))
{
throw_invalid_image("too short");
}
image_header h{};
std::memcpy(&h, image, sizeof(h));
// (the reserved fields are for later versions)
if (std::memcmp(h.magic.data(), "NJVI", 4) != 0 || h.version != image_version
|| (h.reserved[0] | h.reserved[1] | h.reserved[2] | h.reserved[3]) != 0)
{
throw_invalid_image("unknown format");
}
const std::size_t room = size - sizeof(h);
if (h.node_count == 0 || h.node_count > room / sizeof(node) || h.node_count >= image_limit || h.text_size >= image_limit || h.arena_size >= image_limit)
{
throw_invalid_image("sizes out of range");
}
const auto count = static_cast<std::size_t>(h.node_count);
const auto text_size = static_cast<std::size_t>(h.text_size);
const auto arena_size = static_cast<std::size_t>(h.arena_size);
const std::size_t text_at = sizeof(h) + (count * sizeof(node));
// the text, a NUL, the decoded strings, a NUL, and nothing after them
if (size - text_at < 2 || text_size > size - text_at - 2 || arena_size != size - text_at - text_size - 2
|| image[text_at + text_size] != 0 || image[size - 1] != 0)
{
throw_invalid_image("sizes out of range");
}
d.discarded = true;
d.edits.reset();
d.base[2] = nullptr;
d.owned.clear();
if (d.owned_image.empty() || image != d.owned_image.data())
{
d.owned_image.clear();
}
d.arena.clear();
d.indexes.clear();
d.index_slots.clear();
d.large_objects.clear();
d.tape_size = 0;
d.reserve(count);
std::memcpy(d.tape, image + sizeof(h), count * sizeof(node));
d.tape_size = count;
const std::uint8_t* const text = image + text_at;
const std::uint8_t* const arena = text + text_size + 1;
d.src = reinterpret_cast<const char*>(text); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
d.size = text_size;
d.base[0] = d.src;
d.base[1] = reinterpret_cast<const char*>(arena); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
d.arena_size = arena_size;
if (check != image_check::none && !check_image(d.tape, count, text, text_size, arena, arena_size, check == image_check::full))
{
throw_invalid_image("the check failed");
}
// the hash indexes of large objects, as after parsing
for (std::size_t i = 0; i < count; ++i)
{
node& n = d.tape[i];
if (n.kind == static_cast<std::uint8_t>(value_t::object))
{
n.extra = 0;
if (n.len >= document_data::index_min_members)
{
d.large_objects.push_back(static_cast<std::uint32_t>(i));
}
}
}
build_object_indexes(d);
std::vector<std::uint32_t>().swap(d.large_objects); // (only needed while building)
d.discarded = false;
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+1 -1
View File
@@ -73,7 +73,7 @@ class view_iterator
NLOHMANN_VIEW_ALWAYS_INLINE View operator*() const noexcept
{
return View(m_doc, m_pos + m_value_offset);
return View(m_doc, View::navigation::value(m_pos + m_value_offset));
}
pointer operator->() const noexcept
+27 -12
View File
@@ -18,6 +18,7 @@
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/object_index.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -87,14 +88,20 @@ class short_key
/// nullptr (the search stops at the first match; materialize() and parse()
/// keep the last value of a duplicate key instead); most keys are rejected by
/// their length, from the index alone
inline const node* find_member(const document_data& d, const node* object, const char* key, std::size_t n) noexcept
template<bool Editable>
const node* find_member(const document_data& d, const node* object, const char* key, std::size_t n) noexcept
{
const node* const end = document_data::child_end(object);
using nav = navigation<Editable>;
if (NLOHMANN_VIEW_UNLIKELY(object->extra != 0) && (!nav::editable() || (object->flags & node_flags::moved) == 0))
{
return find_indexed(d, object, key, n); // a large object (whose members have not been edited)
}
const node* const end = nav::end(d, object);
const auto* const k = reinterpret_cast<const unsigned char*>(key); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
if (NLOHMANN_VIEW_LIKELY(n <= 16))
{
const short_key probe(k, n);
for (const node* m = document_data::first_child(object); m != end; m = document_data::after(m + 1))
for (const node* m = nav::first(d, object); m != end; m = document_data::after(m + 1))
{
if (m->len == n && probe.matches(reinterpret_cast<const unsigned char*>(d.str(*m)))) // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
{
@@ -103,7 +110,7 @@ inline const node* find_member(const document_data& d, const node* object, const
}
return nullptr;
}
for (const node* m = document_data::first_child(object); m != end; m = document_data::after(m + 1))
for (const node* m = nav::first(d, object); m != end; m = document_data::after(m + 1))
{
if (m->len == n && std::memcmp(d.str(*m), key, n) == 0)
{
@@ -131,10 +138,17 @@ SizeType to_index(IntegerType idx) noexcept
return (idx < zero || static_cast<IntegerType>(result) != idx) ? (std::numeric_limits<SizeType>::max)() : result;
}
/// the element of an array at an index below its size
inline const node* element_at(const node* array, std::size_t idx) noexcept
/// the entry of the element of an array at an index below its size (a link
/// in the moved sequences of editable documents)
template<bool Editable>
const node* element_at(const document_data& d, const node* array, std::size_t idx) noexcept
{
const node* e = document_data::first_child(array);
using nav = navigation<Editable>;
const node* e = nav::first(d, array);
if (nav::editable() && (array->flags & node_flags::moved) != 0 && d.edits->moved_cap[array->off] != 0)
{
return e + idx; // a growable block: one link per element
}
for (std::size_t i = 0; i < idx; ++i)
{
e = document_data::after(e);
@@ -142,13 +156,14 @@ inline const node* element_at(const node* array, std::size_t idx) noexcept
return e;
}
/// the last element of a non-empty array, or the key of the last member of a
/// non-empty object
inline const node* last_child(const node* container) noexcept
/// the entry of the last element of a non-empty array, or the key of the
/// last member of a non-empty object
template<bool Editable>
const node* last_child(const document_data& d, const node* container) noexcept
{
const std::size_t value_offset = container->kind == static_cast<std::uint8_t>(value_t::object) ? 1 : 0;
const node* const end = document_data::child_end(container);
const node* last = document_data::first_child(container);
const node* const end = navigation<Editable>::end(d, container);
const node* last = navigation<Editable>::first(d, container);
for (const node* c = document_data::after(last + value_offset); c != end; c = document_data::after(c + value_offset))
{
last = c;
@@ -19,3 +19,10 @@
#undef NLOHMANN_VIEW_THROW
#undef NLOHMANN_VIEW_LITTLE_ENDIAN
#undef NLOHMANN_VIEW_REPEAT16
#undef NLOHMANN_VIEW_NEON
#undef NLOHMANN_VIEW_SSE2
#undef NLOHMANN_VIEW_SSSE3
#undef NLOHMANN_VIEW_SSSE3_DISPATCH
#undef NLOHMANN_VIEW_SSSE3_TARGET
#undef NLOHMANN_VIEW_VECTOR
#undef NLOHMANN_VIEW_VECTOR_UTF8
+36 -28
View File
@@ -34,19 +34,28 @@ iterative, so the nesting depth is limited by memory only, as for parse().
Without a lexer the handler records no source positions
(JSON_DIAGNOSTIC_POSITIONS).
*/
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable>
BasicJsonType materialize(const document_data& d, const node* n)
{
using string_t = typename BasicJsonType::string_t;
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>>;
using nav = navigation<Editable>;
struct frame
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
};
BasicJsonType result;
sax_t sax(result, true);
const string_t no_token{};
// the ends of the open containers, and whether they are objects
std::vector<std::pair<const node*, bool>> open;
std::vector<frame> open;
for (;;)
{
// false positive: n comes from nav::value(), which never returns null for a valid index
// @infer-ignore NULLPTR_DEREFERENCE
switch (static_cast<value_t>(n->kind))
{
case value_t::object:
@@ -61,67 +70,66 @@ BasicJsonType materialize(const document_data& d, const node* n)
{
sax.start_array(n->len);
}
open.emplace_back(document_data::child_end(n), object);
n = document_data::first_child(n);
open.push_back(frame{nav::first(d, n), nav::end(d, n), object});
break;
}
case value_t::string:
{
string_t s(d.str(*n), n->len);
sax.string(s);
++n;
break;
}
case value_t::number_integer:
sax.number_integer(static_cast<typename BasicJsonType::number_integer_t>(static_cast<std::int64_t>(integer_bits(*n))));
++n;
break;
case value_t::number_unsigned:
sax.number_unsigned(static_cast<typename BasicJsonType::number_unsigned_t>(integer_bits(*n)));
++n;
break;
case value_t::number_float:
sax.number_float(float_value<typename BasicJsonType::number_float_t>(d.str(*n), *n), no_token);
++n;
sax.number_float(float_value<typename BasicJsonType::number_float_t>(d, *n), no_token);
break;
case value_t::boolean:
sax.boolean((n->flags & node_flags::is_true) != 0);
++n;
break;
case value_t::null:
case value_t::binary:
case value_t::discarded:
default:
sax.null();
++n;
break;
}
// the next value: close finished containers, then read the key
for (;;)
{
if (open.empty())
{
return result;
}
if (n != open.back().first)
frame& f = open.back();
if (f.pos == f.end)
{
break;
if (f.object)
{
sax.end_object();
}
else
{
sax.end_array();
}
open.pop_back();
continue;
}
if (open.back().second)
const node* entry = f.pos;
if (f.object)
{
sax.end_object();
string_t key(d.str(*entry), entry->len);
sax.key(key);
++entry;
}
else
{
sax.end_array();
}
open.pop_back();
}
if (open.back().second)
{
// the key of the next member
string_t key(d.str(*n), n->len);
sax.key(key);
++n;
n = nav::value(entry);
f.pos = document_data::after(entry);
break;
}
}
}
+28 -3
View File
@@ -41,20 +41,28 @@ constexpr std::size_t max_input_size() noexcept
struct node_flags
{
static constexpr std::uint8_t escaped = 1; ///< string payload lives in the decode arena, not the source
static constexpr std::uint8_t edited = 2; ///< string or number token lives in the edit arena (editable documents)
static constexpr std::uint8_t storage = 3; ///< mask: where a string or number token lives (index into document_data::base)
static constexpr std::uint8_t is_true = 4; ///< boolean value
static constexpr std::uint8_t moved = 8; ///< array/object: the elements live in a separate sequence (editable documents)
static constexpr std::uint8_t is_new = 16; ///< written by an edit: no source position
static constexpr std::uint8_t linked = 32; ///< an entry of a moved sequence links to this value (editable documents): its extent in the parsed layout no longer matters
};
/// kind of an entry of an edited sequence that stands for a value stored
/// elsewhere (the address of the value's node is kept in the len/next bytes)
constexpr std::uint8_t kind_link = 10;
/// One entry of the flat index, in document order. An object's members are
/// stored as key node followed by the value's subtree. Integers keep their
/// converted 64-bit value in the len/next bytes (the node after a scalar is
/// always the next one, and the token length follows from `extra`).
struct node
{
std::uint8_t kind; ///< value_t
std::uint8_t kind; ///< value_t, or kind_link
std::uint8_t flags; ///< node_flags
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if node_flags::escaped
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; objects: number of the hash index (1-based, 0 = none); otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if escaped/edited; number of the element sequence if moved
std::uint32_t len; ///< string: decoded bytes; float: token bytes; array/object: element count
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence)
};
@@ -65,6 +73,23 @@ NLOHMANN_VIEW_ALWAYS_INLINE bool is_container(const node& n) noexcept
return static_cast<unsigned>(n.kind) - 1u <= 1u;
}
/// the value a link node stands for
NLOHMANN_VIEW_ALWAYS_INLINE const node* link_target(const node& n) noexcept
{
const node* t = nullptr;
std::memcpy(static_cast<void*>(&t), reinterpret_cast<const unsigned char*>(&n) + 8, sizeof(const node*)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return t;
}
/// let the entry n stand for the value at target (and mark the value)
inline void make_link(node& n, node* target) noexcept
{
target->flags = static_cast<std::uint8_t>(target->flags | node_flags::linked);
n = node{};
n.kind = kind_link;
std::memcpy(reinterpret_cast<unsigned char*>(&n) + 8, static_cast<const void*>(&target), sizeof(const node*)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
/// the converted value of an integer node: len is its low half, next its high
/// half (on little-endian targets the two words are the value in memory)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t integer_bits(const node& n) noexcept
+178 -26
View File
@@ -9,11 +9,16 @@
#pragma once
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint64_t
#include <limits> // numeric_limits
#include <string> // string
#include <type_traits> // integral_constant
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/scan.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -21,47 +26,194 @@ namespace detail
namespace view
{
/*!
@brief locate the decimal point and the end of the mantissa of a float token
Also checks that the token is a JSON number. Tokens of the parser and of edits
always are; an image loaded with image_check::bounds can hold any bytes, which
must not reach the conversion (it expects a well-formed token).
*/
inline bool float_token_layout(const char* first, const char* last, std::size_t& dot, std::size_t& mantissa_end) noexcept
{
const auto digit = [last](const char* q)
{
return q != last && is_digit(static_cast<unsigned char>(*q));
};
const char* p = first;
p += (p != last && *p == '-') ? 1 : 0;
if (!digit(p) || (*p == '0' && digit(p + 1)))
{
return false;
}
while (digit(p))
{
++p;
}
dot = std::string::npos;
if (p != last && *p == '.')
{
dot = static_cast<std::size_t>(p - first);
if (!digit(++p))
{
return false;
}
while (digit(p))
{
++p;
}
}
mantissa_end = static_cast<std::size_t>(p - first);
if (p != last && (*p == 'e' || *p == 'E'))
{
++p;
p += (p != last && (*p == '+' || *p == '-')) ? 1 : 0;
if (!digit(p))
{
return false;
}
while (digit(p))
{
++p;
}
}
return p == last;
}
/*!
@brief the value of the float token of a node, as parse() converts it
Uses the lexer's conversion (detail::convert_float), so that the values are
bit-identical to parse(): float and double are converted without allocation
and independent of the locale. The digit layout recorded while parsing locates
the decimal point and the exponent without scanning the token.
and independent of the locale. A token that is not a JSON number (only in a
damaged image loaded with image_check::bounds) yields 0.
*/
template<typename FloatType>
NLOHMANN_VIEW_NOINLINE FloatType float_value(const char* first, const node& n)
{
const char* const last = first + n.len;
const std::size_t neg = first[0] == '-' ? 1 : 0;
const std::size_t int_digits = n.extra & 0xFFu;
const std::size_t frac_digits = n.extra >> 8u;
std::size_t dot = std::string::npos;
std::size_t mantissa_end = n.len;
if (int_digits != 255 && frac_digits != 255)
std::size_t dot = 0;
std::size_t mantissa_end = 0;
if (NLOHMANN_VIEW_UNLIKELY(!float_token_layout(first, last, dot, mantissa_end)))
{
dot = frac_digits != 0 ? neg + int_digits : std::string::npos;
mantissa_end = neg + int_digits + (frac_digits != 0 ? 1 + frac_digits : 0);
}
else
{
// more digits than the layout records: locate them
for (std::size_t i = 0; i < n.len; ++i)
{
if (first[i] == '.')
{
dot = i;
}
else if (first[i] == 'e' || first[i] == 'E')
{
mantissa_end = i;
break;
}
}
return FloatType{};
}
return convert_float<FloatType>(first, last, dot, mantissa_end);
}
/*!
@brief the digits of a float token with at most 19 digits, from its layout
The digit layout recorded while parsing says where the integer digits, the
fraction digits, and the exponent are, so the digits are read eight at a
time without scanning.
@param[in] p first character of the token
@param[in] e end of the token
@param[in] limit end of the readable memory (the source text)
*/
NLOHMANN_VIEW_ALWAYS_INLINE float_significand layout_decimal(const unsigned char* p, const unsigned char* e, unsigned int_digits, unsigned frac_digits, const unsigned char* limit) noexcept
{
const bool negative = *p == '-';
p += negative ? 1 : 0;
std::uint64_t w = parse_upto19(p, int_digits, limit);
p += int_digits;
std::int64_t q = 0;
if (frac_digits != 0)
{
w = (w * int_pow10(frac_digits)) + parse_upto19(p + 1, frac_digits, limit);
p += 1 + frac_digits;
q = -static_cast<std::int64_t>(frac_digits);
}
if (p != e)
{
// [eE][+-]digits; huge exponents saturate (the parser rejected
// overflow). The token is not read beyond e, and the digits are taken
// as unsigned, so that a token that is not well-formed (a damaged
// image loaded with image_check::bounds) yields a wrong value, but no
// overflow.
++p;
const bool exp_negative = p != e && *p == '-';
p += (p != e && (*p == '-' || *p == '+')) ? 1 : 0;
std::int64_t exp_value = 0;
for (; p != e; ++p)
{
if (exp_value < 0x10000000)
{
exp_value = (exp_value * 10) + static_cast<unsigned char>(*p - '0');
}
}
q += exp_negative ? -exp_value : exp_value;
}
float_significand d;
d.w = w;
d.exponent = q;
d.negative = negative;
return d;
}
/*!
@brief the value of a float token with at most 19 digits, from its layout
The result is correctly rounded by the lexer's conversion
(detail::decimal_to_float(): Clinger's fast path where both operands are
exact, else the Eisel-Lemire algorithm, which needs no fallback for up to 19
digits), so it is the value parse() produces.
*/
template<typename FloatType>
NLOHMANN_VIEW_ALWAYS_INLINE FloatType layout_float(const unsigned char* p, const unsigned char* e, unsigned int_digits, unsigned frac_digits, const unsigned char* limit) noexcept
{
return decimal_to_float<FloatType>(layout_decimal(p, e, int_digits, frac_digits, limit));
}
/// the value of a float set by an edit: its token (the shortest round-trip
/// text, or "nan", "inf", "-inf") in the edit arena
template<typename FloatType>
NLOHMANN_VIEW_NOINLINE FloatType edited_float(const char* token, const node& n)
{
if (token[0] == 'n')
{
return std::numeric_limits<FloatType>::quiet_NaN();
}
if (token[0] == 'i' || (token[0] == '-' && token[1] == 'i'))
{
return token[0] == 'i' ? std::numeric_limits<FloatType>::infinity() : -std::numeric_limits<FloatType>::infinity();
}
return float_value<FloatType>(token, n);
}
/// the value of the float token of a node, as parse() converts it; floats and
/// doubles with at most 19 digits are converted from the digit layout
template<typename FloatType>
FloatType float_value(const document_data& d, const node& n)
{
if (NLOHMANN_VIEW_UNLIKELY((n.flags & node_flags::storage) == node_flags::edited))
{
return edited_float<FloatType>(d.str(n), n);
}
return float_value<FloatType>(d, n, std::integral_constant<bool, has_native_float_format<FloatType>::value> {});
}
template<typename FloatType>
FloatType float_value(const document_data& d, const node& n, std::true_type /*binary32 or binary64*/)
{
const unsigned int_digits = n.extra & 0xFFu;
const unsigned frac_digits = n.extra >> 8u;
if (NLOHMANN_VIEW_LIKELY(int_digits + frac_digits <= 19)) // (255 marks "many")
{
// (a float token not written by an edit is in the text)
const auto* const first = reinterpret_cast<const unsigned char*>(d.src + n.off); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return layout_float<FloatType>(first, first + n.len, int_digits, frac_digits, reinterpret_cast<const unsigned char*>(d.src + d.size)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
return float_value<FloatType>(d.str(n), n);
}
template<typename FloatType>
FloatType float_value(const document_data& d, const node& n, std::false_type /*other*/)
{
return float_value<FloatType>(d.str(n), n);
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
@@ -0,0 +1,159 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <cstddef> // size_t
#include <cstdint> // uint32_t, uint64_t
#include <cstring> // memcmp
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
// Hash indexes of large objects, so that a lookup does not compare thousands
// of keys (as Boost.JSON switches from a linear search to a hash table for
// large objects). An object with document_data::index_min_members members or
// more gets an open-addressing table after parsing; its node stores the
// number of the table (1-based) in `extra`. A slot holds the offset of a key
// node from its object node (0: empty). Of duplicate keys, the first is kept,
// as for the linear search.
//
// The hash is not seeded, so keys chosen to collide could make the build
// quadratic. A key therefore sits at most index_max_displacement slots away
// from its home slot; if a key would sit further away, the table is dropped
// and the object is searched linearly (like a small one). For the same
// reason, a lookup visits at most index_max_displacement + 1 slots.
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// the farthest a key may sit from its home slot (a table with at most half of
/// its slots in use gives random keys a distance of about 50 for millions of
/// members; and every member costs at most this many steps while building)
constexpr std::size_t index_max_displacement = 64;
/// hash of a key: its bytes, eight at a time, in a fixed byte order
inline std::uint64_t key_hash(const char* s, std::size_t n) noexcept
{
std::uint64_t h = 0x9E3779B97F4A7C15u * (n + 1);
const auto* p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
while (n >= 8)
{
h = (h ^ read_eight_bytes(p)) * 0xBF58476D1CE4E5B9u;
h ^= h >> 29u;
p += 8;
n -= 8;
}
std::uint64_t w = 0;
for (std::size_t i = 0; i < n; ++i)
{
w |= static_cast<std::uint64_t>(p[i]) << (8u * i);
}
h = (h ^ w) * 0x94D049BB133111EBu;
return h ^ (h >> 31u);
}
/// build the table of a large object
inline void build_object_index(document_data& d, node* obj)
{
if (d.indexes.size() >= 0xFFFFu)
{
return; // LCOV_EXCL_LINE (the number must fit `extra`; more large objects are searched linearly)
}
std::size_t cap = 16;
while (cap < 2 * static_cast<std::size_t>(obj->len))
{
cap *= 2;
}
const std::size_t start = d.index_slots.size();
d.index_slots.resize(start + cap, 0);
std::uint32_t* const slots = d.index_slots.data() + start;
const std::size_t mask = cap - 1;
bool degenerate = false;
for (const node* k = document_data::first_child(obj), *end = document_data::child_end(obj); k != end; k = document_data::after(k + 1))
{
const char* const key = d.str(*k);
const std::uint64_t hash = key_hash(key, k->len); // (a cast of the call would be useless where std::uint64_t is std::size_t)
std::size_t i = static_cast<std::size_t>(hash) & mask;
bool duplicate = false;
std::size_t distance = 0;
while (slots[i] != 0)
{
const node* const other = obj + slots[i];
if (other->len == k->len && (k->len == 0 || std::memcmp(d.str(*other), key, k->len) == 0))
{
duplicate = true; // keep the first
break;
}
if (++distance > index_max_displacement)
{
degenerate = true; // too many keys share a home region
break;
}
i = (i + 1) & mask;
}
if (degenerate)
{
break;
}
if (!duplicate)
{
slots[i] = static_cast<std::uint32_t>(k - obj);
}
}
if (degenerate)
{
d.index_slots.resize(start); // no table: the object is searched linearly
return;
}
d.indexes.push_back(document_data::object_index{start, static_cast<std::uint32_t>(mask)});
obj->extra = static_cast<std::uint16_t>(d.indexes.size());
}
/// build the tables of the large objects the parser noted
inline void build_object_indexes(document_data& d)
{
for (const std::uint32_t i : d.large_objects)
{
build_object_index(d, d.tape + i);
}
}
/// the key node of the first member with this key of an indexed object, or
/// nullptr
inline const node* find_indexed(const document_data& d, const node* obj, const char* key, std::size_t n) noexcept
{
const document_data::object_index& ix = d.indexes[obj->extra - 1u];
const std::uint32_t* const slots = d.index_slots.data() + ix.start;
const std::uint64_t hash = key_hash(key, n); // (a cast of the call would be useless where std::uint64_t is std::size_t)
std::size_t i = static_cast<std::size_t>(hash) & ix.mask;
for (std::size_t distance = 0; distance <= index_max_displacement; ++distance)
{
const std::uint32_t s = slots[i];
if (s == 0)
{
return nullptr;
}
const node* const k = obj + s;
if (k->len == n && (n == 0 || std::memcmp(d.str(*k), key, n) == 0))
{
return k;
}
i = (i + 1) & ix.mask;
}
return nullptr; // (no key sits further from its home slot)
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+3 -1
View File
@@ -51,7 +51,9 @@ index_status array_index(const StringType& s, std::size_t& idx) noexcept
{
return index_status::leading_zero;
}
if (s.size() > 1 && !(s[0] >= '1' && s[0] <= '9'))
// as basic_json: also a single character, but "-" and the empty token are
// left to the conversion below and reported as unresolved
if (!s.empty() && !(s.size() == 1 && s[0] == '-') && !(s[0] >= '0' && s[0] <= '9'))
{
return index_status::not_number;
}
+46 -5
View File
@@ -16,6 +16,7 @@
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/simd.hpp>
// Scanning primitives of the view's parser. The unrolled checks at fixed
// offsets follow yyjson (https://github.com/ibireme/yyjson, MIT license): the
@@ -62,20 +63,45 @@ NLOHMANN_VIEW_ALWAYS_INLINE std::uint16_t load16(const unsigned char* p) noexcep
/// Advance over plain string bytes and well-formed UTF-8. Stops at a quote,
/// a backslash, a control character, ill-formed UTF-8, or the end. The first
/// 16 bytes are checked one by one, so that the position advances by
/// constants in predicted branches (most strings are short); longer runs
/// continue eight bytes at a time.
/// bytes are checked one by one, so that the position advances by constants
/// in predicted branches: 16 for keys, whose lengths repeat from record to
/// record, and 8 for string values (Value) where a vector loop follows, as
/// their lengths vary more. Longer runs continue 16 bytes at a time with NEON
/// or SSE2, else eight bytes at a time. With SSE2, the run is checked 16 bytes
/// at a time from its first byte instead: on x86-64, one compare that finds
/// the end of most keys and short values is faster than a branch per byte (on
/// AArch64, where a NEON mask costs more and branches predict well, slower).
template<bool Value = false>
NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned char* p, const unsigned char* e) noexcept
{
const std::uint8_t* plain = string_plain();
for (;;)
{
#if NLOHMANN_VIEW_SSE2
p = vector_plain_run(p, e);
#else
if (e - p >= 16)
{
#define NLOHMANN_VIEW_STEP(i) if (NLOHMANN_VIEW_LIKELY(plain[p[i]] != 0)) {} else { p += (i); goto stop; }
NLOHMANN_VIEW_REPEAT16(NLOHMANN_VIEW_STEP)
NLOHMANN_VIEW_STEP(0) NLOHMANN_VIEW_STEP(1) NLOHMANN_VIEW_STEP(2) NLOHMANN_VIEW_STEP(3)
NLOHMANN_VIEW_STEP(4) NLOHMANN_VIEW_STEP(5) NLOHMANN_VIEW_STEP(6) NLOHMANN_VIEW_STEP(7)
#if NLOHMANN_VIEW_VECTOR
if (!Value) // (a plain `!Value || !NLOHMANN_VIEW_VECTOR` is a constant condition for MSVC, C4127)
#endif
{
NLOHMANN_VIEW_STEP(8) NLOHMANN_VIEW_STEP(9) NLOHMANN_VIEW_STEP(10) NLOHMANN_VIEW_STEP(11)
NLOHMANN_VIEW_STEP(12) NLOHMANN_VIEW_STEP(13) NLOHMANN_VIEW_STEP(14) NLOHMANN_VIEW_STEP(15)
p += 8;
}
#undef NLOHMANN_VIEW_STEP
p += 16;
p += 8;
#if NLOHMANN_VIEW_VECTOR
p = vector_plain_run(p, e);
if (p != e && plain[*p] == 0)
{
goto stop;
}
#else
while (e - p >= 8)
{
const std::uint64_t special = swar_string_special(read_eight_bytes(p));
@@ -86,8 +112,10 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned
}
p += 8;
}
#endif
continue;
}
#endif
while (p != e && plain[*p] != 0)
{
++p;
@@ -96,11 +124,23 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned
{
return p;
}
#if !NLOHMANN_VIEW_SSE2
stop:
#endif
if (*p < 0x80)
{
return p; // quote, backslash, or control character
}
#if NLOHMANN_VIEW_VECTOR_UTF8
#if NLOHMANN_VIEW_SSSE3_DISPATCH
if (NLOHMANN_VIEW_LIKELY(cpu_has_ssse3()))
#endif
{
// non-ASCII: the vector check, out of line
return scan_string_vector(p, e, plain);
}
#endif
#if !NLOHMANN_VIEW_VECTOR_UTF8 || NLOHMANN_VIEW_SSSE3_DISPATCH
// non-ASCII: a run of well-formed sequences (the library's check, so
// that exactly what json::parse accepts is accepted)
do
@@ -113,6 +153,7 @@ stop:
p += n;
}
while (p != e && *p >= 0x80);
#endif
}
}
+958
View File
@@ -0,0 +1,958 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // max
#include <array> // array
#include <cmath> // isfinite
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy, memset
#include <limits> // numeric_limits
#include <type_traits> // integral_constant
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/number.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// append-only output buffer: writes through a raw pointer into a string that
/// is resized ahead, and trimmed by finish(). The estimate is reserved, and the
/// string grows in steps of 64 KiB within it: resize() fills the new bytes
/// with zeros (before C++23, a string cannot grow without), and a small step
/// is filled while the writer is about to use it, in the cache, instead of
/// filling the whole estimate in memory first.
template<typename StringType>
class output_buffer
{
public:
output_buffer(StringType& out, std::size_t estimate)
: m_out(sized(out, estimate))
, m_pos(&m_out[0])
, m_end(m_pos + m_out.size())
{}
void finish()
{
const auto size = static_cast<std::size_t>(m_pos - m_out.data());
m_out.resize(size);
}
NLOHMANN_VIEW_ALWAYS_INLINE void reserve(std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(m_end - m_pos) < n))
{
grow(n);
}
}
NLOHMANN_VIEW_ALWAYS_INLINE void put(char c)
{
reserve(1);
*m_pos++ = c;
}
NLOHMANN_VIEW_ALWAYS_INLINE void put(const char* s, std::size_t n)
{
reserve(n);
std::memcpy(m_pos, s, n);
m_pos += n;
}
void put_repeated(char c, std::size_t n)
{
reserve(n);
std::memset(m_pos, c, n);
m_pos += n;
}
/// the write position and the end of the writable space, for a writer
/// that keeps the position in a local variable (set_cursor() hands it back)
char* cursor() const noexcept
{
return m_pos;
}
char* limit() const noexcept
{
return m_end;
}
void set_cursor(char* p) noexcept
{
m_pos = p;
}
private:
/// the size of a growth step (a function: std::min() takes a reference,
/// which a static constexpr member does not have before C++17)
static constexpr std::size_t step() noexcept
{
return 65536;
}
static StringType& sized(StringType& out, std::size_t estimate)
{
out.reserve(estimate);
out.resize((std::min)((std::max)(estimate, static_cast<std::size_t>(64)), step()));
return out;
}
NLOHMANN_VIEW_NOINLINE void grow(std::size_t n)
{
const auto used = static_cast<std::size_t>(m_pos - m_out.data());
// (a step does not go beyond the reserved estimate, so that a good
// estimate is never copied to a larger allocation)
const std::size_t size = (std::max)((std::min)(m_out.size() + step(), m_out.capacity()), used + n + 256);
if (size > m_out.capacity())
{
m_out.reserve((std::max)(m_out.capacity() * 2, size));
}
m_out.resize(size);
m_pos = &m_out[0] + used;
m_end = &m_out[0] + m_out.size();
}
StringType& m_out;
char* m_pos;
char* m_end;
};
/// The length of the run at s that dump() writes unchanged without
/// ensure_ascii: all bytes but quotes, backslashes, and control characters.
/// Unlike detail::string_bulk_run(), non-ASCII bytes are not validated: the
/// strings of a document are valid UTF-8 (a damaged image loaded with
/// image_check::bounds can have others, which are then written unchanged).
inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexcept
{
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
const std::uint64_t v = read_eight_bytes(s + i);
const std::uint64_t q = v ^ 0x2222222222222222ull; // '"'
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\'
const std::uint64_t stop = (((q - ones) & ~q) | ((b - ones) & ~b) | ((v - 0x2020202020202020ull) & ~v)) & high;
if (stop != 0)
{
// the lowest flagged byte is the first stop: borrows only flag bytes above a true one
return i + (static_cast<std::size_t>(count_trailing_zeros(stop)) / 8);
}
}
for (; i < n; ++i)
{
if (s[i] == '"' || s[i] == '\\' || s[i] < 0x20)
{
return i;
}
}
return n;
}
/// A stack that starts in a buffer of the caller (a local array) and moves to
/// the heap (a vector of the caller) only when that is full, so that dumps of
/// shallow documents need no allocation. The top is a pointer, as in
/// std::vector. The address of the stack never escapes (the growth gets the
/// vector and returns the new storage), so its pointers stay in registers.
template<typename T>
class small_stack
{
public:
small_stack(T* buffer, std::size_t capacity, std::vector<T>& heap) noexcept
: m_begin(buffer), m_top(buffer), m_end(buffer + capacity), m_heap(&heap)
{}
small_stack(const small_stack&) = delete;
small_stack(small_stack&&) = delete;
small_stack& operator=(const small_stack&) = delete;
small_stack& operator=(small_stack&&) = delete;
~small_stack() = default;
NLOHMANN_VIEW_ALWAYS_INLINE void push_back(const T& x)
{
if (NLOHMANN_VIEW_UNLIKELY(m_top == m_end))
{
const std::size_t used = size();
const std::size_t capacity = 2 * static_cast<std::size_t>(m_end - m_begin);
m_begin = grow(*m_heap, m_begin, used, capacity);
m_top = m_begin + used;
m_end = m_begin + capacity;
}
*m_top++ = x;
}
NLOHMANN_VIEW_ALWAYS_INLINE T& back() noexcept
{
return m_top[-1];
}
NLOHMANN_VIEW_ALWAYS_INLINE void pop_back() noexcept
{
--m_top;
}
NLOHMANN_VIEW_ALWAYS_INLINE bool empty() const noexcept
{
return m_top == m_begin;
}
NLOHMANN_VIEW_ALWAYS_INLINE std::size_t size() const noexcept
{
return static_cast<std::size_t>(m_top - m_begin);
}
private:
/// the used entries moved to heap storage of the given capacity
NLOHMANN_VIEW_NOINLINE static T* grow(std::vector<T>& heap, const T* begin, std::size_t used, std::size_t capacity)
{
std::vector<T> bigger(capacity);
std::copy(begin, begin + used, bigger.begin());
heap.swap(bigger);
return heap.data();
}
T* m_begin;
T* m_top;
T* m_end;
std::vector<T>* m_heap;
};
/// how the view's dump() writes a value
struct dump_style
{
bool pretty = false; ///< indent >= 0
std::size_t indent = 0; ///< characters per level
char indent_char = ' ';
bool ensure_ascii = false;
bool source_numbers = false; ///< copy number tokens from the source
};
/*!
@brief write a view's subtree as basic_json::dump() writes the value
The output of a subtree equals ordered_json::parse(text).dump() of it for
the same arguments (members in document order): strings are escaped by the
same rules, with the library's scanning kernels; floats are written with
the library's conversion; integers are copied from the source, where they
are canonical (except "-0", which parse() reads as 0). The walk is
iterative, so the nesting depth is limited by memory only.
*/
template<typename BasicJsonType, bool Editable>
class view_serializer
{
using nav = navigation<Editable>;
using string_t = typename BasicJsonType::string_t;
using number_float_t = typename BasicJsonType::number_float_t;
/// doubles that are IEEE 754 binary64 are written from their bits (Zmij)
using fast_double = std::integral_constant < bool, std::is_same<number_float_t, double>::value && ::nlohmann::detail::dtoa_impl::is_binary64<double>::value >;
public:
view_serializer(const document_data& d, string_t& out, std::size_t estimate, const dump_style& style)
: m_doc(d), m_out(out, estimate), m_style(style)
{}
void dump(const node* root)
{
if (!m_style.pretty && !m_style.ensure_ascii)
{
if (m_style.source_numbers)
{
dump_compact<true>(root);
}
else
{
dump_compact<false>(root);
}
return;
}
struct frame
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
bool first; ///< nothing written yet
};
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
const node* n = root;
for (;;)
{
// write the value at n
if (is_container(*n))
{
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
if (n->len == 0)
{
m_out.put(object ? "{}" : "[]", 2);
}
else
{
m_out.put(object ? '{' : '[');
stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
}
}
else
{
write_scalar(*n);
}
// go to the next value: close finished containers, then separate
for (;;)
{
if (stack.empty())
{
m_out.finish();
return;
}
frame& f = stack.back();
if (f.pos == f.end)
{
const bool object = f.object;
stack.pop_back();
newline(stack.size());
m_out.put(object ? '}' : ']');
continue;
}
if (!f.first)
{
m_out.put(',');
}
f.first = false;
newline(stack.size());
if (f.object)
{
write_string(*f.pos);
if (m_style.pretty)
{
m_out.put(": ", 2);
}
else
{
m_out.put(':');
}
n = nav::value(f.pos + 1);
f.pos = document_data::after(f.pos + 1);
}
else
{
n = nav::value(f.pos);
f.pos = document_data::after(f.pos);
}
break;
}
}
}
private:
/// a string that is not a plain string of the source (decoded, or written
/// by an edit), without ensure_ascii: runs without characters to escape
/// are copied
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n)
{
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
m_out.put('"');
for (std::size_t i = 0; i < n.len;)
{
const std::size_t run = plain_output_run(s + i, n.len - i);
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
write_codepoint<false>(s[i], s + i, 1); // a quote, a backslash, or a control character
++i;
}
m_out.put('"');
}
/// the copies of dump_compact() that are not fixed-size moves (long
/// strings, or near the end of the source); out of line, so that the
/// compiler does not merge the fixed-size moves into this call
NLOHMANN_VIEW_NOINLINE static void copy_long(char* to, const char* from, std::size_t n) noexcept
{
std::memcpy(to, from, n);
}
/// a compile-time option as a runtime condition: testing the template
/// argument directly makes a condition like `Editable && c` constant when
/// the option is off, which MSVC reports as C4127
static NLOHMANN_VIEW_ALWAYS_INLINE bool enabled(bool option) noexcept
{
return option;
}
/*!
@brief the compact output without ensure_ascii (the default dump())
The same walk as dump(), with the write position in a local variable
(stores through char pointers would otherwise force a reload of the
buffer's members after each one), and with strings and number tokens of
the source copied by fixed-size moves of 32 bytes where the source has
that many bytes left, instead of a library call per token. The buffer
keeps 64 bytes of slack for the overshoot.
*/
template<bool SourceNumbers>
void dump_compact(const node* root)
{
struct frame
{
const node* pos; ///< (editable documents) next element, or key of the next member
const node* end;
bool object;
};
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
const char* const src = m_doc.src;
const char* const src_end = src + m_doc.size;
char* w = m_out.cursor();
char* lim = m_out.limit();
// room for n bytes and the slack
const auto room = [&](std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(lim - w) < n + 64))
{
m_out.set_cursor(w);
m_out.reserve(n + 64);
w = m_out.cursor();
lim = m_out.limit();
}
};
// copy n bytes of the source (after room(n))
const auto copy = [&](const char* from, std::size_t n)
{
if (n <= 32 && static_cast<std::size_t>(src_end - from) >= 32)
{
std::memcpy(w, from, 32);
}
else if (n <= 256 && static_cast<std::size_t>(src_end - from) >= n + 32)
{
for (std::size_t i = 0; i < n; i += 32)
{
std::memcpy(w + i, from + i, 32);
}
}
else
{
copy_long(w, from, n);
}
w += n;
};
// a literal of n bytes (after room(n))
const auto literal = [&](const char* text, std::size_t n)
{
std::memcpy(w, text, n);
w += n;
};
// a string that is not a plain string of the source (out of line, so
// that the cursor stays in a register here)
const auto escaped = [&](const node & n)
{
m_out.set_cursor(w);
write_decoded(n);
w = m_out.cursor();
lim = m_out.limit();
};
// Read-only documents: the elements of a container follow it in the
// node array, so the walk goes through the array in order, and a
// frame only needs the end of its container. Editable documents: the
// elements of a moved container live elsewhere, so a frame keeps the
// position of the next element (see navigation).
// The innermost open container is kept in registers (cur; end ==
// nullptr: none), the stack holds the ones around it.
frame cur{nullptr, nullptr, false};
const node* n = root;
for (;;)
{
// write the value at n (read-only documents: and advance n)
bool opened = false;
// false positive: n comes from nav::value(), which never returns null for a valid index
// @infer-ignore NULLPTR_DEREFERENCE
switch (static_cast<value_t>(n->kind))
{
case value_t::string:
if ((n->flags & node_flags::storage) == 0)
{
room(n->len + 2);
*w++ = '"';
copy(src + n->off, n->len);
*w++ = '"';
}
else
{
escaped(*n);
}
break;
case value_t::number_integer:
case value_t::number_unsigned:
{
const std::uint32_t len = number_length(*n);
room(len);
if (enabled(Editable) && (n->flags & node_flags::storage) != 0)
{
copy_long(w, m_doc.str(*n), len); // a canonical token written by an edit
w += len;
break;
}
const char* const token = src + n->off;
if (!enabled(SourceNumbers) && NLOHMANN_VIEW_UNLIKELY(len == 2 && token[0] == '-' && token[1] == '0'))
{
*w++ = '0'; // parse() reads -0 as the integer 0
}
else
{
copy(token, len);
}
break;
}
case value_t::number_float:
if (enabled(SourceNumbers) && (n->flags & node_flags::storage) != node_flags::edited)
{
room(n->len);
copy(src + n->off, n->len);
}
else if (enabled(fast_double::value))
{
room(64);
w = write_double_at(w, *n);
}
else
{
m_out.set_cursor(w);
write_float_node(*n);
w = m_out.cursor();
lim = m_out.limit();
}
break;
case value_t::boolean:
room(8);
if ((n->flags & node_flags::is_true) != 0)
{
literal("true", 4);
}
else
{
literal("false", 5);
}
break;
case value_t::object:
case value_t::array:
{
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
room(8);
if (n->len == 0)
{
literal(object ? "{}" : "[]", 2);
}
else
{
*w++ = object ? '{' : '[';
stack.push_back(cur);
if (Editable)
{
cur = frame{nav::first(m_doc, n), nav::end(m_doc, n), object};
}
else
{
cur = frame{nullptr, n + n->next, object};
}
opened = true;
}
break;
}
case value_t::null:
room(8);
literal("null", 4);
break;
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
if (!Editable)
{
++n; // the next node: the first element of an opened container, or the node after a scalar
}
// go to the next value: close finished containers, then separate
// (a container just opened has an element)
if (!opened)
{
for (;;)
{
if (cur.end == nullptr)
{
m_out.set_cursor(w);
m_out.finish();
return;
}
if ((Editable ? cur.pos : n) != cur.end)
{
break;
}
room(1);
*w++ = cur.object ? '}' : ']';
cur = stack.back();
stack.pop_back();
}
room(1);
*w++ = ',';
}
const node* const at = Editable ? cur.pos : n;
if (cur.object)
{
const node& key = *at;
if ((key.flags & node_flags::storage) == 0)
{
room(key.len + 3);
*w++ = '"';
copy(src + key.off, key.len);
w[0] = '"';
w[1] = ':';
w += 2;
}
else
{
escaped(key);
room(1);
*w++ = ':';
}
if (Editable)
{
n = nav::value(at + 1);
cur.pos = document_data::after(at + 1);
}
else
{
++n;
}
}
else if (Editable)
{
n = nav::value(at);
cur.pos = document_data::after(at);
}
}
}
void newline(std::size_t level)
{
if (m_style.pretty)
{
m_out.put('\n');
m_out.put_repeated(m_style.indent_char, level * m_style.indent);
}
}
void write_scalar(const node& n)
{
switch (static_cast<value_t>(n.kind))
{
case value_t::null:
m_out.put("null", 4);
break;
case value_t::boolean:
if ((n.flags & node_flags::is_true) != 0)
{
m_out.put("true", 4);
}
else
{
m_out.put("false", 5);
}
break;
case value_t::string:
write_string(n);
break;
case value_t::number_integer:
case value_t::number_unsigned:
{
const char* const token = m_doc.str(n);
const std::uint32_t len = number_length(n);
if (!m_style.source_numbers && len == 2 && token[0] == '-' && token[1] == '0')
{
m_out.put('0'); // parse() reads -0 as the integer 0
}
else
{
m_out.put(token, len);
}
break;
}
case value_t::number_float:
if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
{
m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
}
else
{
write_float_node(n);
}
break;
case value_t::object: // LCOV_EXCL_LINE (containers are written by dump())
case value_t::array: // LCOV_EXCL_LINE
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
}
/// a float node as dump() writes it
void write_float_node(const node& n)
{
write_float_node(n, fast_double {});
}
void write_float_node(const node& n, std::false_type /*other*/)
{
write_float(float_value<number_float_t>(m_doc, n));
}
void write_float_node(const node& n, std::true_type /*double*/)
{
m_out.reserve(64);
m_out.set_cursor(write_double_at(m_out.cursor(), n));
}
/*!
@brief (doubles) the float at n as dump() writes it, at w (64 bytes of room)
A token of at most 15 significant digits is written from its digits,
without a conversion: two decimals of at most 15 digits are farther
apart than the rounding interval of a (normal) double (the argument
behind DBL_DIG), so the token's digits are the shortest ones of its
double, which the library's conversion writes (Zmij). Other tokens are
converted from the digits already read.
*/
char* write_double_at(char* w, const node& n)
{
const unsigned int_digits = n.extra & 0xFFu;
const unsigned frac_digits = n.extra >> 8u;
if ((n.flags & node_flags::storage) != node_flags::edited && int_digits + frac_digits <= 19)
{
const auto* const first = reinterpret_cast<const unsigned char*>(m_doc.src + n.off); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const float_significand d = layout_decimal(first, first + n.len, int_digits, frac_digits, reinterpret_cast<const unsigned char*>(m_doc.src + m_doc.size)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
// (the exponent keeps the value far from subnormals and overflow)
if (d.w != 0 && d.w < 1000000000000000u && d.exponent >= -290 && d.exponent <= 290)
{
*w = '-';
w += d.negative ? 1 : 0;
// (without leading zeros, all digits of the token count; the
// check also keeps an image that was only checked for bounds,
// whose token may not be made of digits, from the counted
// overload)
const auto& powers = ::nlohmann::detail::dtoa_impl::powers_of_ten_16();
const unsigned count = int_digits + frac_digits;
return count - 1u < 15u && d.w >= powers[count - 1u] && d.w < powers[count]
? ::nlohmann::detail::dtoa_impl::write_short_decimal(w, d.w, static_cast<int>(count), static_cast<int>(d.exponent))
: ::nlohmann::detail::dtoa_impl::write_short_decimal(w, d.w, static_cast<int>(d.exponent));
}
return write_double_value_at(w, decimal_to_float<double>(d)); // (without reading the token again)
}
return write_double_value_at(w, static_cast<double>(float_value<number_float_t>(m_doc, n)));
}
/// n bytes of text at w
static char* write_text_at(char* w, const char* text, std::size_t n) noexcept
{
std::memcpy(w, text, n);
return w + n;
}
/// a double as dump() writes it, at w (64 bytes of room)
static char* write_double_value_at(char* w, double x)
{
// (from the bits: without the checks of to_chars())
std::uint64_t bits = 0;
std::memcpy(&bits, &x, sizeof(bits));
if (NLOHMANN_VIEW_UNLIKELY((bits & 0x7FF0000000000000u) == 0x7FF0000000000000u))
{
return write_text_at(w, "null", 4);
}
*w = '-';
w += bits >> 63u;
bits &= ~(std::uint64_t{1} << 63u);
if (bits == 0)
{
return write_text_at(w, "0.0", 3);
}
return ::nlohmann::detail::dtoa_impl::write_shortest(w, ::nlohmann::detail::zmij::to_shortest(bits));
}
/// as serializer::dump_float()
void write_float(number_float_t x)
{
if (!std::isfinite(x))
{
m_out.put("null", 4);
return;
}
write_float(x, std::integral_constant < bool,
(std::numeric_limits<number_float_t>::is_iec559 && std::numeric_limits<number_float_t>::digits == 24 && std::numeric_limits<number_float_t>::max_exponent == 128)
|| (std::numeric_limits<number_float_t>::is_iec559 && std::numeric_limits<number_float_t>::digits == 53 && std::numeric_limits<number_float_t>::max_exponent == 1024) > {});
}
void write_float(number_float_t x, std::true_type /*is_ieee_single_or_double*/)
{
std::array<char, 64> buf{};
const char* const end = ::nlohmann::detail::to_chars(buf.data(), buf.data() + buf.size(), x);
m_out.put(buf.data(), static_cast<std::size_t>(end - buf.data()));
}
void write_float(number_float_t x, std::false_type /*is_ieee_single_or_double*/)
{
// other types (e.g. long double) are rare: the library writes them
const string_t s = BasicJsonType(x).dump();
m_out.put(s.data(), s.size());
}
void write_string(const node& n)
{
const char* const s = m_doc.str(n);
m_out.put('"');
if ((n.flags & node_flags::storage) == 0 && !m_style.ensure_ascii)
{
// a string of the source without escape sequences has nothing to escape
m_out.put(s, n.len);
}
else if (m_style.ensure_ascii)
{
write_escaped<true>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
else
{
write_escaped<false>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
m_out.put('"');
}
/// as serializer::dump_escaped(); strings of a document are valid UTF-8,
/// except in a damaged image loaded with image_check::bounds, for which
/// this throws what basic_json::dump() throws for the string
template<bool EnsureAscii>
void write_escaped(const unsigned char* s, std::size_t n)
{
std::size_t i = 0;
while (i < n)
{
std::size_t run = 0;
if (!EnsureAscii)
{
run = string_bulk_run(s + i, n - i);
}
else if (is_ascii_copyable(s[i]))
{
run = find_ascii_copyable_run(s + i, n - i);
}
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
std::uint32_t codepoint = s[i];
std::size_t len = 1;
if (codepoint >= 0x80)
{
len = validate_one_utf8(s + i, n - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
invalid_utf8(s, n);
return;
}
codepoint &= 0xFFu >> (len + 1);
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
}
write_codepoint<EnsureAscii>(codepoint, s + i, len);
i += len;
}
}
/// throw what basic_json::dump() throws for a string that is not valid UTF-8
NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
{
const string_t dumped = BasicJsonType(string_t(reinterpret_cast<const char*>(s), n)).dump(); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
static_cast<void>(dumped);
}
template<bool EnsureAscii>
void write_codepoint(std::uint32_t codepoint, const unsigned char* bytes, std::size_t len)
{
switch (codepoint)
{
case 0x08:
m_out.put("\\b", 2);
return;
case 0x09:
m_out.put("\\t", 2);
return;
case 0x0A:
m_out.put("\\n", 2);
return;
case 0x0C:
m_out.put("\\f", 2);
return;
case 0x0D:
m_out.put("\\r", 2);
return;
case 0x22:
m_out.put("\\\"", 2);
return;
case 0x5C:
m_out.put("\\\\", 2);
return;
default:
break;
}
if (codepoint <= 0x1F || (EnsureAscii && codepoint >= 0x7F))
{
if (codepoint <= 0xFFFF)
{
write_u_escape(codepoint);
}
else
{
write_u_escape(0xD7C0u + (codepoint >> 10u));
write_u_escape(0xDC00u + (codepoint & 0x3FFu));
}
return;
}
m_out.put(reinterpret_cast<const char*>(bytes), len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast) LCOV_EXCL_LINE (printable characters are copied in runs)
}
void write_u_escape(std::uint32_t u)
{
static constexpr const char* hex = "0123456789abcdef";
const std::array<char, 6> e = {{'\\', 'u', hex[(u >> 12u) & 0xFu], hex[(u >> 8u) & 0xFu], hex[(u >> 4u) & 0xFu], hex[u & 0xFu]}};
m_out.put(e.data(), e.size());
}
const document_data& m_doc;
output_buffer<string_t> m_out;
const dump_style m_style;
};
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+339
View File
@@ -0,0 +1,339 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-FileCopyrightText: 2018-2025 The simdjson authors <https://github.com/simdjson/simdjson>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <atomic> // atomic
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint64_t
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
// Vector code for long runs of string bytes. NEON (AArch64) and SSE2 (x86-64)
// belong to the baseline instruction sets and are used by default. The vector
// UTF-8 check needs NEON or SSSE3. SSSE3 is not part of x86-64, and the code
// must not depend on the flags of a translation unit (two translation units
// with different flags would have different definitions of the same inline
// functions): the check is compiled for SSSE3 with a function attribute and
// used where the CPU has SSSE3 (all x86-64 CPUs since about 2011), else the
// portable check. JSON_VIEW_USE_SSSE3 skips the CPU check (for code compiled
// for SSSE3 anyway); JSON_VIEW_NO_SIMD selects the portable code.
// NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE (set by the GCC module interface, where GCC
// ignores the target attribute and then rejects the SSSE3 intrinsics) keeps the
// check portable without disabling the SSE2 code.
#if !defined(JSON_VIEW_NO_SIMD) && defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__)) && NLOHMANN_VIEW_LITTLE_ENDIAN
#include <arm_neon.h>
#define NLOHMANN_VIEW_NEON 1
#else
#define NLOHMANN_VIEW_NEON 0
#endif
// (x86 only: other targets can define __SSE2__ as well, e.g., WebAssembly with -msse2, but have no <cpuid.h>)
#if !defined(JSON_VIEW_NO_SIMD) && !NLOHMANN_VIEW_NEON && (defined(__x86_64__) || defined(__i386__) || defined(_M_X64) || defined(_M_IX86)) && (defined(__SSE2__) || defined(_M_X64) || (defined(_M_IX86_FP) && _M_IX86_FP >= 2))
#include <emmintrin.h>
#define NLOHMANN_VIEW_SSE2 1
#else
#define NLOHMANN_VIEW_SSE2 0
#endif
#if NLOHMANN_VIEW_SSE2 && defined(JSON_VIEW_USE_SSSE3)
#include <tmmintrin.h>
#define NLOHMANN_VIEW_SSSE3 1 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#else
#define NLOHMANN_VIEW_SSSE3 0 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#endif
#if NLOHMANN_VIEW_SSE2 && !NLOHMANN_VIEW_SSSE3 && !defined(NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE) && ((defined(__clang__) && __clang_major__ >= 4) || (defined(__GNUC__) && !defined(__clang__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9))))
// (GCC before 4.9 has no SSSE3 intrinsics without -mssse3)
#include <cpuid.h>
#include <tmmintrin.h>
#define NLOHMANN_VIEW_SSSE3_DISPATCH 1 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#define NLOHMANN_VIEW_SSSE3_TARGET __attribute__((target("ssse3")))
#elif NLOHMANN_VIEW_SSE2 && !NLOHMANN_VIEW_SSSE3 && defined(_MSC_VER)
// (MSVC compiles intrinsics of any instruction set)
#include <intrin.h>
#include <tmmintrin.h>
#define NLOHMANN_VIEW_SSSE3_DISPATCH 1 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#define NLOHMANN_VIEW_SSSE3_TARGET
#else
#define NLOHMANN_VIEW_SSSE3_DISPATCH 0 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#define NLOHMANN_VIEW_SSSE3_TARGET
#endif
#define NLOHMANN_VIEW_VECTOR (NLOHMANN_VIEW_NEON || NLOHMANN_VIEW_SSE2)
#define NLOHMANN_VIEW_VECTOR_UTF8 (NLOHMANN_VIEW_NEON || NLOHMANN_VIEW_SSSE3 || NLOHMANN_VIEW_SSSE3_DISPATCH)
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
#if NLOHMANN_VIEW_VECTOR
/*!
@brief the first byte of a string run that is a quote, a backslash, a control
character, or not ASCII, 16 bytes per step
Stops at such a byte, or where fewer than 16 bytes are left (the caller tells
the two apart). A signed compare with 0x20 finds control characters and
non-ASCII bytes at once.
*/
NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned char* p, const unsigned char* e) noexcept
{
while (e - p >= 16)
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
// one nibble per byte (the usual NEON replacement of x86's movemask, see
// D. Kutenin, "Porting x86 vector bitmask optimizations to Arm NEON", 2022)
const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
if (bits != 0)
{
return p + (count_trailing_zeros(bits) >> 2u);
}
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
if (bits != 0)
{
return p + count_trailing_zeros(bits);
}
#endif
p += 16;
}
return p;
}
#endif
#if NLOHMANN_VIEW_SSSE3_DISPATCH
/// whether the CPU has SSSE3 (CPUID leaf 1, ECX bit 9)
inline bool cpu_ssse3() noexcept
{
#if defined(_MSC_VER) && !defined(__clang__)
std::array<int, 4> regs {{}};
__cpuid(regs.data(), 1);
return (static_cast<unsigned>(regs[2]) & (1u << 9u)) != 0;
#else
unsigned eax = 0;
unsigned ebx = 0;
unsigned ecx = 0;
unsigned edx = 0;
return __get_cpuid(1, &eax, &ebx, &ecx, &edx) != 0 && (ecx & (1u << 9u)) != 0;
#endif
}
/// whether the CPU has SSSE3, asked once: the answer is kept in an atomic
/// that is initialized at compile time, so that neither a guard of a local
/// static nor a global constructor is needed (threads that ask at the same
/// time all store the same answer)
NLOHMANN_VIEW_ALWAYS_INLINE bool cpu_has_ssse3() noexcept
{
static std::atomic<int> known{0}; // 0: not asked yet, 1: no, 2: yes
int state = known.load(std::memory_order_relaxed);
if (NLOHMANN_VIEW_UNLIKELY(state == 0))
{
state = cpu_ssse3() ? 2 : 1;
known.store(state, std::memory_order_relaxed);
}
return state == 2;
}
#endif
#if NLOHMANN_VIEW_VECTOR_UTF8
/// Tables of the UTF-8 check of J. Keiser and D. Lemire, "Validating UTF-8 In
/// Less Than One Instruction Per Byte" (2021), as in simdjson ("lookup4"): each
/// maps a nibble (high and low nibble of the previous byte, high nibble of the
/// current byte) to the errors it allows; a byte pair is ill-formed if all
/// three have an error bit in common.
template<typename Dummy = void>
struct utf8_lookup4
{
static constexpr std::uint8_t too_short = 1u << 0u, too_long = 1u << 1u, overlong_3 = 1u << 2u, too_large = 1u << 3u;
static constexpr std::uint8_t surrogate = 1u << 4u, overlong_2 = 1u << 5u, too_large_1000 = 1u << 6u, overlong_4 = 1u << 6u;
static constexpr std::uint8_t two_conts = 1u << 7u, carry = too_short | too_long | two_conts;
static const std::array<std::uint8_t, 16> byte_1_high;
static const std::array<std::uint8_t, 16> byte_1_low;
static const std::array<std::uint8_t, 16> byte_2_high;
};
template<typename Dummy>
const std::array<std::uint8_t, 16> utf8_lookup4<Dummy>::byte_1_high =
{
{
too_long, too_long, too_long, too_long, too_long, too_long, too_long, too_long,
two_conts, two_conts, two_conts, two_conts,
too_short | overlong_2, too_short, too_short | overlong_3 | surrogate, too_short | too_large | too_large_1000 | overlong_4
}
};
template<typename Dummy>
const std::array<std::uint8_t, 16> utf8_lookup4<Dummy>::byte_1_low =
{
{
carry | overlong_3 | overlong_2 | overlong_4, carry | overlong_2, carry, carry,
carry | too_large, carry | too_large | too_large_1000, carry | too_large | too_large_1000, carry | too_large | too_large_1000,
carry | too_large | too_large_1000, carry | too_large | too_large_1000, carry | too_large | too_large_1000, carry | too_large | too_large_1000,
carry | too_large | too_large_1000, carry | too_large | too_large_1000 | surrogate, carry | too_large | too_large_1000, carry | too_large | too_large_1000
}
};
template<typename Dummy>
const std::array<std::uint8_t, 16> utf8_lookup4<Dummy>::byte_2_high =
{
{
too_short, too_short, too_short, too_short, too_short, too_short, too_short, too_short,
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | overlong_3 | too_large_1000 | overlong_4),
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | overlong_3 | too_large),
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | surrogate | too_large),
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | surrogate | too_large),
too_short, too_short, too_short, too_short
}
};
/// the end of scan_string_vector from block, where the vector loop stopped
/// (ill-formed UTF-8, or fewer than 16 bytes left): one byte or sequence at a
/// time, from the start of a sequence that crosses into the block
inline const unsigned char* scan_string_finish(const unsigned char* p, const unsigned char* block, const unsigned char* e, const std::uint8_t* plain) noexcept
{
for (int i = 1; i <= 3 && block - i >= p; ++i)
{
const unsigned char c = block[-i];
if (c < 0x80)
{
break;
}
if (c >= 0xC0)
{
const int len = 2 + static_cast<int>(c >= 0xE0) + static_cast<int>(c >= 0xF0);
if (len > i)
{
block -= i;
}
break;
}
}
for (p = block; p != e;)
{
if (*p < 0x80)
{
if (plain[*p] == 0)
{
return p;
}
++p;
continue;
}
const std::size_t n = validate_one_utf8(p, static_cast<std::size_t>(e - p));
if (n == 0)
{
return p;
}
p += n;
}
return p;
}
/*!
@brief the rest of a string from p (a character boundary), 16 bytes per step
The first quote, backslash, or control character is found with vector
compares, and the UTF-8 check covers the bytes up to it. Returns where the
string scan stops, like scan_string_run: before ill-formed UTF-8 and for the
last bytes of the input, the bytes are checked one sequence at a time. Out of
line, so that no constants of the check occupy registers in the parse loop.
On x86-64, it is compiled for SSSE3 (see cpu_has_ssse3()).
*/
NLOHMANN_VIEW_SSSE3_TARGET NLOHMANN_VIEW_NOINLINE inline const unsigned char* scan_string_vector(const unsigned char* p, const unsigned char* e, const std::uint8_t* plain) noexcept
{
using lookup = utf8_lookup4<>;
const unsigned char* block = p;
#if NLOHMANN_VIEW_NEON
const uint8x16_t t1h = vld1q_u8(lookup::byte_1_high.data());
const uint8x16_t t1l = vld1q_u8(lookup::byte_1_low.data());
const uint8x16_t t2h = vld1q_u8(lookup::byte_2_high.data());
uint8x16_t prev = vdupq_n_u8(0);
while (e - block >= 16)
{
const uint8x16_t in = vld1q_u8(block);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))), vcltq_u8(in, vdupq_n_u8(0x20)));
const uint8x16_t prev1 = vextq_u8(prev, in, 15);
const uint8x16_t sc = vandq_u8(vandq_u8(vqtbl1q_u8(t1h, vshrq_n_u8(prev1, 4)), vqtbl1q_u8(t1l, vandq_u8(prev1, vdupq_n_u8(0x0F)))), vqtbl1q_u8(t2h, vshrq_n_u8(in, 4)));
const uint8x16_t must23 = vorrq_u8(vqsubq_u8(vextq_u8(prev, in, 14), vdupq_n_u8(0xE0 - 0x80)), vqsubq_u8(vextq_u8(prev, in, 13), vdupq_n_u8(0xF0 - 0x80)));
const uint8x16_t err = veorq_u8(vandq_u8(must23, vdupq_n_u8(0x80)), sc);
const std::uint64_t special_bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
const std::uint64_t err_bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(vtstq_u8(err, err)), 4)), 0);
if (special_bits != 0)
{
// errors up to the special byte count (an incomplete sequence
// before a quote shows at the quote); the bytes after it do not
const unsigned k = static_cast<unsigned>(count_trailing_zeros(special_bits)) >> 2u;
const std::uint64_t upto = k == 15 ? ~std::uint64_t{0} :
(std::uint64_t{1} << (4u * (k + 1u))) - 1u;
if ((err_bits & upto) == 0)
{
return block + k;
}
break;
}
if (err_bits != 0)
{
break;
}
prev = in;
block += 16;
}
#else
// the same with SSSE3 (pshufb for the table lookups; nibbles from 16-bit
// shifts, as there are no byte shifts)
const __m128i t1h = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(lookup::byte_1_high.data()))); // NOLINT(bugprone-casting-through-void)
const __m128i t1l = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(lookup::byte_1_low.data()))); // NOLINT(bugprone-casting-through-void)
const __m128i t2h = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(lookup::byte_2_high.data()))); // NOLINT(bugprone-casting-through-void)
const __m128i nibble = _mm_set1_epi8(0x0F);
const __m128i zero = _mm_setzero_si128();
__m128i prev = zero;
while (e - block >= 16)
{
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(block))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmpeq_epi8(_mm_subs_epu8(in, _mm_set1_epi8(0x1F)), zero)); // in < 0x20
const __m128i prev1 = _mm_alignr_epi8(in, prev, 15);
const __m128i sc = _mm_and_si128(_mm_and_si128(_mm_shuffle_epi8(t1h, _mm_and_si128(_mm_srli_epi16(prev1, 4), nibble)),
_mm_shuffle_epi8(t1l, _mm_and_si128(prev1, nibble))),
_mm_shuffle_epi8(t2h, _mm_and_si128(_mm_srli_epi16(in, 4), nibble)));
const __m128i must23 = _mm_or_si128(_mm_subs_epu8(_mm_alignr_epi8(in, prev, 14), _mm_set1_epi8(0xE0 - 0x80)),
_mm_subs_epu8(_mm_alignr_epi8(in, prev, 13), _mm_set1_epi8(0xF0 - 0x80)));
const __m128i err = _mm_xor_si128(_mm_and_si128(must23, _mm_set1_epi8(static_cast<char>(-128))), sc);
const auto special_bits = static_cast<unsigned>(_mm_movemask_epi8(special));
const auto err_bits = ~static_cast<unsigned>(_mm_movemask_epi8(_mm_cmpeq_epi8(err, zero))) & 0xFFFFu;
if (special_bits != 0)
{
const auto k = static_cast<unsigned>(count_trailing_zeros(static_cast<std::uint64_t>(special_bits)));
if ((err_bits & ((2u << k) - 1u)) == 0)
{
return block + k;
}
break;
}
if (err_bits != 0)
{
break;
}
prev = in;
block += 16;
}
#endif
return scan_string_finish(p, block, e, plain);
}
#endif
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+1 -1
View File
@@ -49,7 +49,7 @@ NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const nod
case value_t::number_integer:
return static_cast<T>(static_cast<typename BasicJsonType::number_integer_t>(static_cast<std::int64_t>(integer_bits(n))));
case value_t::number_float:
return static_cast<T>(float_value<typename BasicJsonType::number_float_t>(d.str(n), n));
return static_cast<T>(float_value<typename BasicJsonType::number_float_t>(d, n));
case value_t::boolean:
return static_cast<T>((n.flags & node_flags::is_true) != 0);
case value_t::null:
+461 -23
View File
@@ -7,14 +7,16 @@
// SPDX-License-Identifier: MIT
/****************************************************************************\
* Zero-copy, read-only view of a parsed JSON text. *
* Zero-copy view of a parsed JSON text. *
* *
* json_document::parse() builds a flat index of the values of a JSON text *
* (16 bytes per value) instead of a tree of basic_json values. Strings and *
* numbers stay in the source text; only strings with escapes are decoded, *
* into one buffer. json_view is a handle to one value of the document, with *
* the read-only part of the basic_json interface; materialize() turns a *
* subtree into the basic_json value that parse() would produce. *
* subtree into the basic_json value that parse() would produce. An editable *
* document (json_editable_document) also has set(), push_back(), insert(), *
* and erase(): edits never write to the source text, and views stay valid. *
* *
* The source text must outlive a document that borrows it (lvalue byte *
* containers, C strings); rvalue strings, streams, and other inputs are *
@@ -24,12 +26,17 @@
#ifndef INCLUDE_NLOHMANN_JSON_VIEW_HPP_
#define INCLUDE_NLOHMANN_JSON_VIEW_HPP_
#include <algorithm> // all_of, min
#include <array> // array
#include <cstddef> // nullptr_t, size_t // IWYU pragma: keep
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy, strlen
#include <iterator> // distance, input_iterator_tag, iterator_traits
#include <map> // map
#include <memory> // unique_ptr
#ifndef JSON_NO_IO
#include <ostream> // ostream
#endif
#include <string> // string
#include <tuple> // tuple_element, tuple_size // IWYU pragma: keep
#include <type_traits> // decay, enable_if, integral_constant, is_arithmetic, is_base_of, is_integral, is_same, remove_cv, remove_extent
@@ -59,15 +66,20 @@
#endif
#include <nlohmann/detail/view/builder.hpp>
#include <nlohmann/detail/view/compare.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/edit.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/image.hpp>
#include <nlohmann/detail/view/input.hpp>
#include <nlohmann/detail/view/iterator.hpp>
#include <nlohmann/detail/view/lookup.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/materialize.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/object_index.hpp>
#include <nlohmann/detail/view/pointer.hpp>
#include <nlohmann/detail/view/serializer.hpp>
#include <nlohmann/detail/view/string_ref.hpp>
#include <nlohmann/detail/view/value.hpp>
@@ -79,11 +91,13 @@ NLOHMANN_JSON_NAMESPACE_BEGIN
Trivially copyable (two pointers). Valid as long as the document is alive and
has not been re-parsed, and as long as a borrowed source text is alive.
*/
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable = false>
class basic_json_view
{
using node = detail::view::node;
using document_data = detail::view::document_data;
/// how the index is walked (with edits only for editable documents)
using navigation = detail::view::navigation<Editable>;
public:
using value_t = detail::value_t;
@@ -280,7 +294,7 @@ class basic_json_view
}
detail::view::throw_type_error(305, "cannot use operator[] with a numeric argument with ", type_name());
}
return idx < m_node->len ? basic_json_view(m_doc, detail::view::element_at(m_node, idx)) : basic_json_view();
return idx < m_node->len ? basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx))) : basic_json_view();
}
/// any other integer type (int, unsigned, long, std::int64_t, ...; a
@@ -343,7 +357,7 @@ class basic_json_view
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return basic_json_view(m_doc, detail::view::element_at(m_node, idx));
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
}
/// any other integer type, see operator[]; negative values are out of
@@ -418,7 +432,7 @@ class basic_json_view
{
if (is_structured() && m_node->len != 0)
{
return basic_json_view(m_doc, detail::view::last_child(m_node) + (is_object() ? 1 : 0));
return basic_json_view(m_doc, navigation::value(detail::view::last_child<Editable>(*m_doc, m_node) + (is_object() ? 1 : 0)));
}
return front();
}
@@ -435,7 +449,7 @@ class basic_json_view
{
return end();
}
const node* const k = detail::view::find_member(*m_doc, m_node, key.data(), key.size());
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? iterator(m_doc, k, true) : end();
}
@@ -452,7 +466,7 @@ class basic_json_view
/// whether this is an object with a member with this key
bool contains(string_view_t key) const
{
return is_object() && detail::view::find_member(*m_doc, m_node, key.data(), key.size()) != nullptr;
return is_object() && detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size()) != nullptr;
}
bool contains(const char* key) const
@@ -499,7 +513,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_LIKELY(is_structured()))
{
return iterator(m_doc, document_data::first_child(m_node), is_object());
return iterator(m_doc, navigation::first(*m_doc, m_node), is_object());
}
return iterator(m_doc, m_node, false);
}
@@ -508,7 +522,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_LIKELY(is_structured()))
{
return iterator(m_doc, document_data::child_end(m_node), is_object());
return iterator(m_doc, navigation::end(*m_doc, m_node), is_object());
}
return iterator(m_doc, (is_null() || is_discarded()) ? m_node : m_node + 1, false);
}
@@ -573,6 +587,111 @@ class basic_json_view
return {m_doc->str(*m_node), detail::view::number_length(*m_node)};
}
///////////////////
// serialization //
///////////////////
/// how dump() writes numbers
enum class number_format
{
/// as basic_json::dump(): integers canonically, floats with the
/// library's shortest round-trip digits ("1.5", "100.0", "1e+100")
shortest,
/// the number text of the source as it is ("1.50", "1E2", "-0", all
/// digits of a long integer)
source,
};
/// the text of this value; with number_format::shortest, the output of
/// ordered_json::parse(text).dump() with the same arguments (members in
/// document order, all of them should a key occur more than once)
string_t dump(const int indent = -1, const char indent_char = ' ', const bool ensure_ascii = false,
const number_format numbers = number_format::shortest) const
{
string_t out;
if (m_node == nullptr)
{
out = "<discarded>"; // as basic_json::dump() of a discarded value
return out;
}
detail::view::dump_style style;
style.pretty = indent >= 0;
style.indent = indent >= 0 ? static_cast<std::size_t>(indent) : 0;
style.indent_char = indent_char;
style.ensure_ascii = ensure_ascii;
style.source_numbers = numbers == number_format::source;
// the compact text is about as long as the source text of the value;
// the compact writer keeps 64 bytes of slack, so that it does not grow
// the buffer just before the end
const std::size_t estimate = source_extent() + (style.pretty ? source_extent() / 2 : 0) + 160;
detail::view::view_serializer<BasicJsonType, Editable>(*m_doc, out, estimate, style).dump(m_node);
return out;
}
#ifndef JSON_NO_IO
/// as operator<< of basic_json: a stream width > 0 is the indentation,
/// the fill character the indentation character
friend std::ostream& operator<<(std::ostream& o, const basic_json_view& v)
{
const bool pretty = o.width() > 0;
const auto indentation = pretty ? o.width() : 0;
o.width(0);
const string_t s = v.dump(pretty ? static_cast<int>(indentation) : -1, o.fill());
return o.write(s.data(), static_cast<std::streamsize>(s.size()));
}
#endif
////////////////
// comparison //
////////////////
/// whether the values parse() would produce for two views are equal, as
/// by BasicJsonType's operator== (numbers by value, objects by their
/// members with duplicate keys resolved as parse() resolves them)
friend bool operator==(const basic_json_view& a, const basic_json_view& b)
{
return detail::view::equal<BasicJsonType>(side(a), side(b));
}
friend bool operator!=(const basic_json_view& a, const basic_json_view& b)
{
return !(a == b);
}
/// a view of an editable document compares with one of a read-only document
template < bool E, typename std::enable_if < E != Editable, int >::type = 0 >
friend bool operator==(const basic_json_view& a, const basic_json_view<BasicJsonType, E>& b)
{
return detail::view::equal<BasicJsonType>(side(a), detail::view::view_side<BasicJsonType, basic_json_view<BasicJsonType, E>>(b));
}
template < bool E, typename std::enable_if < E != Editable, int >::type = 0 >
friend bool operator!=(const basic_json_view& a, const basic_json_view<BasicJsonType, E>& b)
{
return !(a == b);
}
/// whether the value parse() would produce for a view equals a value
friend bool operator==(const basic_json_view& a, const BasicJsonType& j)
{
return detail::view::equal<BasicJsonType>(side(a), json_side_t(j));
}
friend bool operator==(const BasicJsonType& j, const basic_json_view& a)
{
return a == j;
}
friend bool operator!=(const basic_json_view& a, const BasicJsonType& j)
{
return !(a == j);
}
friend bool operator!=(const BasicJsonType& j, const basic_json_view& a)
{
return !(a == j);
}
/////////////////
// materialize //
/////////////////
@@ -585,7 +704,7 @@ class basic_json_view
{
return BasicJsonType(value_t::discarded);
}
return detail::view::materialize<BasicJsonType>(*m_doc, m_node);
return detail::view::materialize<BasicJsonType, Editable>(*m_doc, m_node);
}
/// byte offset of this value in the source text (for strings: of the
@@ -593,24 +712,76 @@ class basic_json_view
/// a discarded view and for strings with escapes, which are decoded
std::size_t source_offset() const noexcept
{
return m_node != nullptr && (m_node->flags & detail::view::node_flags::storage) == 0
return m_node != nullptr && (m_node->flags & (detail::view::node_flags::storage | detail::view::node_flags::moved | detail::view::node_flags::is_new)) == 0
? m_node->off : static_cast<std::size_t>(-1);
}
private:
template<typename> friend class basic_json_document;
template<typename, bool> friend class basic_json_document;
template<typename, bool> friend class basic_json_view;
template<typename, typename> friend class detail::view::editor;
friend iterator;
basic_json_view(const document_data* d, const node* n) noexcept
: m_doc(d), m_node(n)
{}
using json_side_t = detail::view::json_side<BasicJsonType, string_view_t>;
static detail::view::view_side<BasicJsonType, basic_json_view> side(const basic_json_view& v) noexcept
{
return detail::view::view_side<BasicJsonType, basic_json_view>(v);
}
/// the template argument as a runtime condition: testing it directly in
/// `Editable && ...` makes the condition constant, which MSVC reports as
/// C4127
static bool editable() noexcept
{
return Editable;
}
/// the number of source bytes of this value (estimated for values with
/// decoded strings); the estimate sizes the output buffer of dump()
std::size_t source_extent() const noexcept
{
if (editable() && m_doc->edits != nullptr)
{
// positions of moved and new values are not source offsets
return m_node == m_doc->tape ? m_doc->size + m_doc->edits->text_used : 64;
}
const node* const end = m_doc->tape + m_doc->tape_size;
if ((m_node->flags & detail::view::node_flags::storage) != 0)
{
return m_node->len;
}
// the value ends where the next node in the source begins; nodes
// with decoded strings (their offset is in the arena) are skipped,
// but only a few of them, to keep the walk short
const node* next = document_data::after(m_node);
for (int skipped = 0; next != end && skipped < 16; ++skipped, ++next)
{
if ((next->flags & detail::view::node_flags::storage) == 0)
{
return next->off >= m_node->off ? next->off - m_node->off : 0;
}
}
if (next == end)
{
return m_doc->size - m_node->off;
}
// the end is unknown: assume a few bytes per node, the output buffer
// grows should the value be larger
const auto nodes = static_cast<std::size_t>(document_data::after(m_node) - m_node);
return (std::min)(m_doc->size - m_node->off, static_cast<std::size_t>(1024) + (nodes * 16));
}
/// the value of the first member with this key, or a discarded view
/// (object required)
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view lookup(string_view_t key) const noexcept
{
const node* const k = detail::view::find_member(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, k + 1) : basic_json_view();
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, navigation::value(k + 1)) : basic_json_view();
}
// --- get() dispatch ---
@@ -701,7 +872,7 @@ Borrowed parses keep a pointer to the caller's text, which must outlive the
document. Owned parses (parse_copy, rvalue std::string, streams, and inputs
that are not contiguous byte ranges) keep their own copy.
*/
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable = false>
class basic_json_document
{
using document_data = detail::view::document_data;
@@ -710,7 +881,7 @@ class basic_json_document
"json_view supports 64-bit integer types only");
public:
using view_type = basic_json_view<BasicJsonType>;
using view_type = basic_json_view<BasicJsonType, Editable>;
using value_t = detail::value_t;
/// an empty (discarded) document
@@ -838,7 +1009,7 @@ class basic_json_document
/// whether the document holds its own copy of the text
bool owns_source() const noexcept
{
return m_data && !m_data->owned.empty() && m_data->src == m_data->owned.data();
return m_data && ((!m_data->owned.empty() && m_data->src == m_data->owned.data()) || !m_data->owned_image.empty());
}
/// number of index nodes (values plus object keys)
@@ -847,7 +1018,7 @@ class basic_json_document
return m_data ? m_data->tape_size : 0;
}
/// bytes held by the document (index, decoded strings, owned text)
/// bytes held by the document (index, decoded strings, owned text or image)
std::size_t memory_usage() const noexcept
{
if (!m_data)
@@ -856,7 +1027,10 @@ class basic_json_document
}
return sizeof(document_data) + (m_data->inline_cap * sizeof(detail::view::node))
+ (m_data->tape != m_data->inline_tape ? m_data->tape_cap * sizeof(detail::view::node) : 0)
+ m_data->arena.capacity() + m_data->owned.capacity();
+ m_data->arena.capacity() + m_data->owned.capacity() + m_data->owned_image.capacity()
+ (m_data->indexes.capacity() * sizeof(document_data::object_index)) + (m_data->index_slots.capacity() * sizeof(std::uint32_t))
+ (m_data->large_objects.capacity() * sizeof(std::uint32_t))
+ (m_data->edits != nullptr ? m_data->edits->bytes : 0);
}
/// release unused capacity of the index and the decoded strings; like
@@ -873,10 +1047,15 @@ class basic_json_document
// allocate everything first, so that an exception leaves the document
// unchanged
// (the decoded strings of a loaded image stay in the image)
const bool arena_in_use = d.base[1] == d.arena.data();
const bool shrink_arena = d.arena.capacity() > d.arena.size();
std::string arena(shrink_arena ? d.arena : std::string());
const bool shrink_tape = d.tape != d.inline_tape && d.tape_size != d.tape_cap;
std::string arena(shrink_arena && arena_in_use ? d.arena : std::string());
// (edits link to the nodes of the index, which then stays in place)
const bool shrink_tape = d.tape != d.inline_tape && d.tape_size != d.tape_cap && d.edits == nullptr;
const bool into_header = d.tape_size <= d.inline_cap;
std::vector<document_data::object_index> indexes(d.indexes.capacity() > d.indexes.size() ? d.indexes : std::vector<document_data::object_index>());
std::vector<std::uint32_t> index_slots(d.index_slots.capacity() > d.index_slots.size() ? d.index_slots : std::vector<std::uint32_t>());
node* fresh = (shrink_tape && !into_header) ? static_cast<node*>(::operator new (d.tape_size * sizeof(node))) : d.inline_tape;
if (shrink_tape)
@@ -889,13 +1068,254 @@ class basic_json_document
if (shrink_arena)
{
d.arena.swap(arena);
d.base[1] = d.arena.data();
if (arena_in_use)
{
d.base[1] = d.arena.data();
}
}
if (d.indexes.capacity() > d.indexes.size())
{
d.indexes.swap(indexes);
}
if (d.index_slots.capacity() > d.index_slots.size())
{
d.index_slots.swap(index_slots);
}
}
////////////
// images //
////////////
/// how load() checks an image (full, bounds, or none)
using image_check = detail::view::image_check;
/// The document as an image that load() reads without parsing: the node
/// index, the text, and the decoded strings. An edited document is
/// written in its current state (floats that are not finite become null,
/// as in dump()).
std::vector<std::uint8_t> save() const
{
if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
{
detail::view::throw_type_error(320, "cannot save a discarded json_document");
}
return detail::view::save_image(*m_data);
}
/// Read an image written by save(). The image is borrowed: it must stay
/// alive and unchanged while the document is used.
NLOHMANN_VIEW_NODISCARD
static basic_json_document load(const std::uint8_t* image, std::size_t size, const image_check check = image_check::full)
{
basic_json_document d;
d.ensure_data(nullptr, 0);
detail::view::load_image(*d.m_data, image, size, check);
return d;
}
/// read an image (borrowed)
NLOHMANN_VIEW_NODISCARD
static basic_json_document load(const std::vector<std::uint8_t>& image, const image_check check = image_check::full)
{
return load(image.data(), image.size(), check);
}
/// read an image and keep it (no copy)
NLOHMANN_VIEW_NODISCARD
static basic_json_document load(std::vector<std::uint8_t>&& image, const image_check check = image_check::full)
{
basic_json_document d;
d.ensure_data(nullptr, 0);
d.m_data->owned_image = std::move(image);
detail::view::load_image(*d.m_data, d.m_data->owned_image.data(), d.m_data->owned_image.size(), check);
return d;
}
///////////
// edits //
///////////
// The source text is never written; new values go to storage owned by
// the document. A view keeps referring to the same value: after an
// assignment it sees the new value, and edits elsewhere do not affect it.
// A view of an erased value keeps its last value. An edit of an
// array/object invalidates the iterators over it. Values are accepted as
// views (of any document), BasicJsonType values, and everything
// BasicJsonType can be constructed from.
using string_view_t = typename view_type::string_view_t;
using json_pointer = typename BasicJsonType::json_pointer;
/// replace a value (a view of this document); returns a view of it
template<typename V>
view_type set(view_type target, V&& value)
{
return editor().set(target, std::forward<V>(value));
}
/// set a member (added if missing; a null value becomes an object);
/// returns a view of the member value
template<typename V>
view_type set(view_type object, string_view_t key, V&& value)
{
return editor().set(object, key, std::forward<V>(value));
}
/// assign an existing array element; returns a view of it
template < typename I, typename V, typename std::enable_if < std::is_integral<I>::value && !std::is_same<I, bool>::value, int >::type = 0 >
view_type set(view_type array, I idx, V && value)
{
return editor().set(array, index(idx), std::forward<V>(value));
}
/// set the value at a JSON pointer: its parent must exist; an object
/// member is set (added if missing), an array element assigned, and "-"
/// or the size of the array appends. A null parent becomes what
/// basic_json's operator[](json_pointer) makes of it: an array for "-"
/// and for digits (padded with nulls up to the index), an object otherwise.
template<typename V>
view_type set(const json_pointer& ptr, V&& value)
{
if (ptr.empty())
{
return set(root(), std::forward<V>(value));
}
const view_type parent = root().at(ptr.parent_pointer());
const auto& token = ptr.back();
if (parent.is_null())
{
const bool digits = std::all_of(token.begin(), token.end(), [](const char c)
{
return c >= '0' && c <= '9';
});
if (token == "-")
{
return push_back(parent, std::forward<V>(value));
}
if (digits)
{
// (an invalid index is an error before the parent changes)
const std::size_t idx = pointer_index(token);
if (idx >= 0xFFFFFFFFu)
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
for (std::size_t i = 0; i < idx; ++i)
{
push_back(parent, nullptr);
}
return push_back(parent, std::forward<V>(value));
}
}
if (parent.is_array())
{
const std::size_t idx = token == "-" ? parent.size() : pointer_index(token);
if (idx == parent.size())
{
return push_back(parent, std::forward<V>(value));
}
return set(parent, idx, std::forward<V>(value));
}
return set(parent, string_view_t(token.data(), token.size()), std::forward<V>(value));
}
/// append to an array (a null value becomes an array); returns a view of
/// the new element
template<typename V>
view_type push_back(view_type array, V&& value)
{
return editor().push_back(array, std::forward<V>(value));
}
/// insert into an array before position idx (idx <= size()); returns a
/// view of the new element
template < typename I, typename V, typename std::enable_if < std::is_integral<I>::value && !std::is_same<I, bool>::value, int >::type = 0 >
view_type insert(view_type array, I idx, V && value)
{
return editor().insert(array, index(idx), std::forward<V>(value));
}
/// remove all members with this key; returns their number
std::size_t erase(view_type object, string_view_t key)
{
return editor().erase(object, key);
}
/// remove an array element
template < typename I, typename std::enable_if < std::is_integral<I>::value && !std::is_same<I, bool>::value, int >::type = 0 >
void erase(view_type array, I idx)
{
editor().erase(array, index(idx));
}
/// remove the value at a JSON pointer; returns the number of removed
/// values
std::size_t erase(const json_pointer& ptr)
{
if (ptr.empty())
{
detail::view::throw_out_of_range(405, "JSON pointer has no parent");
}
const view_type parent = root().at(ptr.parent_pointer());
const auto& token = ptr.back();
if (parent.is_array())
{
erase(parent, pointer_index(token));
return 1;
}
return erase(parent, string_view_t(token.data(), token.size()));
}
private:
using input_kind = detail::view::input_kind;
detail::view::editor<BasicJsonType, view_type> editor()
{
static_assert(Editable, "only an editable document can be edited: use basic_json_document<BasicJsonType, true> (json_editable_document)");
if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
{
detail::view::throw_invalid_iterator(202, "view does not belong to this document");
}
return detail::view::editor<BasicJsonType, view_type>(*m_data);
}
/// an index (out_of_range.401 if negative)
template<typename I>
static std::size_t index(I idx)
{
return index(idx, std::is_signed<I> {});
}
template<typename I>
static std::size_t index(I idx, std::true_type /*signed*/)
{
if (idx < 0)
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return static_cast<std::size_t>(idx);
}
template<typename I>
static std::size_t index(I idx, std::false_type /*unsigned*/)
{
return static_cast<std::size_t>(idx);
}
/// the array index of a JSON pointer token (json_pointer's rules)
template<typename StringType>
static std::size_t pointer_index(const StringType& token)
{
std::size_t idx = 0;
const detail::view::index_status status = detail::view::array_index(token, idx);
if (status != detail::view::index_status::ok)
{
detail::view::throw_array_index_error(status, token);
}
return idx;
}
/// create the storage (sized for the input) on first use
void ensure_data(const char* src, std::size_t size)
{
@@ -922,10 +1342,16 @@ class basic_json_document
{
d.owned.clear();
}
d.owned_image.clear();
d.src = src;
d.size = size;
d.tape_size = 0;
d.edits.reset(); // (views of the previous text end here anyway)
d.base[2] = nullptr;
d.arena.clear();
d.indexes.clear();
d.index_slots.clear();
d.large_objects.clear();
d.discarded = true;
detail::view::parse_failure failure;
bool ok = false;
@@ -941,9 +1367,13 @@ class basic_json_document
{
d.base[0] = d.src;
d.base[1] = d.arena.data();
d.arena_size = d.arena.size();
detail::view::build_object_indexes(d);
std::vector<std::uint32_t>().swap(d.large_objects); // (only needed while parsing)
d.discarded = false;
return;
}
std::vector<std::uint32_t>().swap(d.large_objects);
if (allow_exceptions)
{
detail::view::throw_parse_failure<BasicJsonType>(failure, src, size, comments, trailing_commas);
@@ -1058,6 +1488,14 @@ using json_view = basic_json_view<json>;
using ordered_json_document = basic_json_document<ordered_json>;
/// a value of an ordered_json_document
using ordered_json_view = basic_json_view<ordered_json>;
/// an editable parsed JSON text for json
using json_editable_document = basic_json_document<json, true>;
/// a value of a json_editable_document
using json_editable_view = basic_json_view<json, true>;
/// an editable parsed JSON text for ordered_json
using ordered_json_editable_document = basic_json_document<ordered_json, true>;
/// a value of an ordered_json_editable_document
using ordered_json_editable_view = basic_json_view<ordered_json, true>;
NLOHMANN_JSON_NAMESPACE_END
+42
View File
@@ -27160,6 +27160,48 @@ inline char* write_shortest(char* first, const zmij::shortest_decimal d) noexcep
return end + (three ? 5 : 4);
}
/// the powers of ten up to 10^16
inline const std::array<std::uint64_t, 17>& powers_of_ten_16() noexcept
{
static const std::array<std::uint64_t, 17> powers =
{
{
1u, 10u, 100u, 1000u, 10000u, 100000u, 1000000u, 10000000u, 100000000u, 1000000000u, 10000000000u,
100000000000u, 1000000000000u, 10000000000000u, 100000000000000u, 1000000000000000u, 10000000000000000u
}
};
return powers;
}
/*!
@brief digits * 10^exp, as write_decimal() writes it, for the digits of a
double that need no conversion (count digits, at most 15, the first not 0;
trailing zeros allowed): extended to 16 digits and written by write_shortest()
@return a pointer past the text; up to 41 bytes at @a first are written
(some beyond the returned end)
*/
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_short_decimal(char* first, std::uint64_t digits, int count, int exp) noexcept
{
JSON_ASSERT(digits >= powers_of_ten_16()[static_cast<std::size_t>(count - 1)] && count <= 15);
const int scale = 16 - count;
return write_shortest(first, zmij::shortest_decimal{digits * powers_of_ten_16()[static_cast<std::size_t>(scale)], exp - scale - 1, 0, false});
}
/// as write_short_decimal(), counting the digits (not 0, less than 10^15)
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_short_decimal(char* first, std::uint64_t digits, int exp) noexcept
{
JSON_ASSERT(digits != 0 && digits < 1000000000000000u);
// floor(log10(2^bits)) + 1 digits, or one less
const int log2_bound = ((64 - count_leading_zeros(digits)) * 1233) >> 12;
const int count = log2_bound + (digits >= powers_of_ten_16()[static_cast<std::size_t>(log2_bound)] ? 1 : 0);
return write_short_decimal(first, digits, count, exp);
}
/*!
@brief whether FloatType is an IEEE 754 binary64 type (a double, or a long double
that has the same format, as with MSVC and on Apple's Arm CPUs)
+4867 -312
View File
File diff suppressed because it is too large. Load diff
+6
View File
@@ -17,6 +17,12 @@ module;
// 3. Or upgrade to a newer GCC version with better modules support.
// See: https://github.com/nlohmann/json/issues/5103
// GCC ignores the target attribute in modules ("'target' attribute currently
// unsupported in modules") and then rejects the SSSE3 intrinsics of json_view.
#if defined(__GNUC__) && !defined(__clang__)
#define NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE
#endif
#include <nlohmann/json.hpp>
#include <nlohmann/json_literals.hpp>
#include <nlohmann/json_view.hpp>
+19
View File
@@ -369,6 +369,25 @@ json_test_add_test_for(src/unit-diagnostic-positions.cpp
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
# the json_view parser again with the portable string scanning instead of
# NEON/SSE2, and on x86-64 with the SSSE3 UTF-8 check (JSON_VIEW_USE_SSSE3)
json_test_set_test_options(test-json_view_builder_portable
COMPILE_DEFINITIONS JSON_VIEW_NO_SIMD
)
json_test_add_test_for(src/unit-json_view_builder.cpp
NAME test-json_view_builder_portable
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
if(CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64|amd64)$" AND NOT MSVC)
json_test_set_test_options(test-json_view_builder_ssse3
COMPILE_DEFINITIONS JSON_VIEW_USE_SSSE3
COMPILE_OPTIONS -mssse3
)
json_test_add_test_for(src/unit-json_view_builder.cpp
NAME test-json_view_builder_ssse3
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
endif()
endif()
# *DO NOT* use json_test_set_test_options() below this line
+4 -1
View File
@@ -10,7 +10,7 @@ CXXFLAGS += -std=c++11
CPPFLAGS += -I ../single_include
FUZZER_ENGINE = src/fuzzer-driver_afl.cpp
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer parse_bon8_fuzzer parse_json_view_fuzzer
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer parse_bon8_fuzzer parse_json_view_fuzzer json_view_image_fuzzer
fuzzers: $(FUZZERS)
parse_afl_fuzzer:
@@ -19,6 +19,9 @@ parse_afl_fuzzer:
parse_json_view_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_json_view.cpp -o $@
json_view_image_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-json_view_image.cpp -o $@
parse_bson_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_bson.cpp -o $@
+1
View File
@@ -21,6 +21,7 @@ Micro-benchmarks for parsing, serialization and the binary formats, written with
| `ViewParseIndented` | as `ParseIndented`, with a reused `json_document` |
| `ViewAccept` | validate with `json_document::accept`; compare with `Accept` |
| `ViewMaterialize` | convert a parsed `json_document` into a `json` value |
| `ViewDump` | serialize a parsed `json_document`; compare with `Dump` |
The input files are those of [nativejson-benchmark](https://github.com/miloyip/nativejson-benchmark) (`canada`,
`citm_catalog`, `twitter`), a large `jeopardy` file, and number-heavy files (`floats`, `signed_ints`, ...).
+1
View File
@@ -0,0 +1 @@
build/
+90
View File
@@ -0,0 +1,90 @@
# json_view compared with other libraries
The in-tree benchmarks in [`tests/benchmarks`](../README.md) measure `json_document` against `json::parse` only. The
programs here compare it with [yyjson](https://github.com/ibireme/yyjson),
[simdjson](https://github.com/simdjson/simdjson), and [Boost.JSON](https://github.com/boostorg/json): the question
users ask when they pick a library. They are not built by CMake or run by CI.
## Reproducing the numbers
`compare.py` builds the programs against `include/` of this checkout, runs them, and writes the results together with
everything needed to reproduce them to `results/<date>-<host>.md` (and `.csv`): the date, the commit, the CPU, the
OS, the compiler, the flags, and the versions of all libraries.
```sh
python3 tests/benchmarks/json_view/compare.py --data <json_test_data directory> [--native] [--rounds 30]
```
- `--data` is the downloaded [test data](https://github.com/nlohmann/json_test_data), e.g. the `test_files` directory
of a CMake build directory. It needs `nativejson-benchmark/{twitter,citm_catalog,canada}.json` and
`jeopardy/jeopardy.json`.
- The other libraries come from the system: pkg-config, or Homebrew (`brew install yyjson simdjson boost`). With
`--download`, pinned releases are downloaded instead and checked against their SHA-256. Without Boost headers (or
with `--no-boost`), the Boost.JSON columns are skipped, and the results say so.
- `--corpus file...` adds files to the corpus benchmark, e.g. those of
[simdjson-data](https://github.com/simdjson/simdjson-data) or the
[yyjson benchmark](https://github.com/ibireme/yyjson_benchmark).
- Only the Python 3 standard library is used; a C++17 compiler is needed (`CXX` and `CC` are honored).
For numbers worth publishing, use a quiet machine (see [Getting stable numbers](../README.md#getting-stable-numbers)),
the default 30 rounds or more, and `--native` only if the other libraries were built for the same CPU.
### On GitHub-hosted runners
The workflow [json_view benchmarks](../../../.github/workflows/json_view_benchmarks.yml) runs `compare.py --download`
on demand: by hand (Actions → "json_view benchmarks" → "Run workflow"), on an x86-64 or AArch64 Ubuntu runner with GCC
or Clang, or when a pull request gets the label `benchmark`, on both architectures with GCC. The results appear as the
job summary and as an artifact. Shared runners are noisy, so these numbers show
where `json_view` stands on another architecture; they are not meant for publication.
## What is measured
`bench_view.cpp` runs four workloads on twitter, citm_catalog, canada, jeopardy, a single tweet (`status`), and a
JSON-RPC request (`rpc`):
| workload | what it does |
|---|---|
| parse | build and free a document |
| traverse | parse, then visit every value, convert every number, touch every string and key |
| select | parse, then read a few fields per record (e.g. id, user name, and retweet count of each tweet) |
| dump | serialize a parsed document (compact) |
`bench_corpus.cpp` runs parse, traverse, and dump on any list of files, so that no library is tuned to a handful of
documents. Its dump also writes the numbers as they are in the input: `json_view` with `number_format::source`, and
yyjson with numbers read as raw text (`YYJSON_READ_NUMBER_AS_RAW`), without converting them.
`bench_edit.cpp` measures read-modify-write: parse, apply the same logical edits with each library's own API, and
serialize (compact). Workloads: `patch` (a handful of edits at fixed places) and `update` (edits in every record).
An editable `json_document` edits in place; yyjson copies its immutable document into a mutable one first
(`yyjson_doc_mut_copy`); Boost.JSON and `json::parse` build mutable DOMs; simdjson cannot edit a document. All
outputs are checked to describe the same value.
Before anything is timed, all engines must accept each document and agree on the traversal: the number of values, the
bytes of all strings and keys, and the sum of all numbers. All engines run interleaved in every round, and the best
round is reported, as time and as a factor of the `json_view` time (below 1 means faster than `json_view`). Each timed
call follows an untimed call of the same engine: otherwise the engine after `json::parse` pays for the allocator
cleaning up the tens of thousands of nodes `json::parse` just freed (with glibc, this made `json_view` look 1.7 times
slower on citm_catalog traverse).
The engines do not all offer the same features, which the numbers should be read with:
| engine | document | random access | editable | notes |
|---|---|---|---|---|
| `json_view` | immutable index into the text | yes | no | a fresh document per parse; "reused" parses into the same document |
| yyjson | immutable (`yyjson_read`) | yes | via a mutable copy | |
| simdjson DOM | immutable, parser reused | yes | no | "fresh" uses a new parser per parse |
| simdjson On-Demand | none: forward-only, lazy | no | no | only traverse and select |
| Boost.JSON | owning, mutable DOM | yes | yes | monotonic resource |
| `json::parse` | owning, mutable DOM | yes | yes | |
Reusing memory matters as much as the parser. simdjson DOM reuses its parser, so it writes into memory it already
touched; a fresh `json_view` document or yyjson document gets new memory for every parse. On Linux, glibc returns large
blocks to the system when they are freed, so every fresh parse of a large document pays a page fault per 4 KiB page:
on x86-64 Linux, a fresh `json_view` parse of jeopardy took about twice as long as a reused one. On macOS on Apple
silicon, with 16 KiB pages, the difference is much smaller. Compare "json_view (reused)" with "simdjson DOM", and the
fresh `json_view` with "simdjson DOM (fresh)" and yyjson.
## Published results
Results are only published with the file `compare.py` wrote, which names the machine and the versions; see
`results/`. Numbers from one machine and compiler do not carry over to another: rerun the script.
+348
View File
@@ -0,0 +1,348 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// Corpus benchmark: the read-only workloads of bench_view.cpp on any list of
// JSON files (for example the benchmark sets of simdjson and yyjson).
//
// ./bench_corpus [--rounds N] file...
//
// For every file, all engines must accept it and agree on a traversal (value
// count, string bytes, sum of numbers) before anything is timed. Workloads:
// parse (build and free a document), traverse (visit every value, convert
// every number), dump (compact), and for json_view also dump with the source
// number text, compared with yyjson writing numbers read as raw text
// (YYJSON_READ_NUMBER_AS_RAW). Results go to bench_corpus.csv.
#include <nlohmann/json_view.hpp>
#if JSON_VIEW_BENCH_BOOST
#include <boost/json.hpp>
#include <boost/json/src.hpp>
#endif
#include <simdjson.h>
#include <yyjson.h>
#include <algorithm>
#include <chrono>
#include <cmath>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <fstream>
#include <functional>
#include <sstream>
#include <string>
#include <vector>
using nlohmann::json;
using nlohmann::json_document;
using nlohmann::json_view;
static volatile double g_sink;
struct stats
{
double num = 0;
std::size_t str = 0, nodes = 0;
};
static void walk(json_view v, stats& st)
{
++st.nodes;
switch (v.type())
{
case json::value_t::object:
for (auto it = v.begin(); it != v.end(); ++it)
{
st.str += it.key().size();
walk(*it, st);
}
break;
case json::value_t::array:
for (const json_view e : v)
{
walk(e, st);
}
break;
case json::value_t::string:
st.str += v.get_string().size();
break;
case json::value_t::number_integer:
st.num += static_cast<double>(v.get<std::int64_t>());
break;
case json::value_t::number_unsigned:
st.num += static_cast<double>(v.get<std::uint64_t>());
break;
case json::value_t::number_float:
st.num += v.get<double>();
break;
default:
break;
}
}
static void walk(yyjson_val* v, stats& st)
{
++st.nodes;
switch (yyjson_get_type(v))
{
case YYJSON_TYPE_OBJ:
{
std::size_t idx, max;
yyjson_val* k, * val;
yyjson_obj_foreach(v, idx, max, k, val)
{
st.str += yyjson_get_len(k);
walk(val, st);
}
break;
}
case YYJSON_TYPE_ARR:
{
std::size_t idx, max;
yyjson_val* val;
yyjson_arr_foreach(v, idx, max, val)
{
walk(val, st);
}
break;
}
case YYJSON_TYPE_STR:
st.str += yyjson_get_len(v);
break;
case YYJSON_TYPE_NUM:
st.num += yyjson_is_sint(v) ? static_cast<double>(yyjson_get_sint(v)) : yyjson_is_uint(v) ? static_cast<double>(yyjson_get_uint(v)) : yyjson_get_real(v);
break;
default:
break;
}
}
static void walk(simdjson::dom::element e, stats& st)
{
++st.nodes;
switch (e.type())
{
case simdjson::dom::element_type::OBJECT:
for (auto f : simdjson::dom::object(e))
{
st.str += f.key.size();
walk(f.value, st);
}
break;
case simdjson::dom::element_type::ARRAY:
for (auto c : simdjson::dom::array(e))
{
walk(c, st);
}
break;
case simdjson::dom::element_type::STRING:
st.str += std::string_view(e).size();
break;
case simdjson::dom::element_type::INT64:
st.num += static_cast<double>(int64_t(e));
break;
case simdjson::dom::element_type::UINT64:
st.num += static_cast<double>(uint64_t(e));
break;
case simdjson::dom::element_type::DOUBLE:
st.num += double(e);
break;
default:
break;
}
}
#if JSON_VIEW_BENCH_BOOST
static void walk(const boost::json::value& v, stats& st)
{
++st.nodes;
switch (v.kind())
{
case boost::json::kind::object:
for (const auto& kv : v.get_object())
{
st.str += kv.key().size();
walk(kv.value(), st);
}
break;
case boost::json::kind::array:
for (const auto& c : v.get_array())
{
walk(c, st);
}
break;
case boost::json::kind::string:
st.str += v.get_string().size();
break;
case boost::json::kind::int64:
st.num += static_cast<double>(v.get_int64());
break;
case boost::json::kind::uint64:
st.num += static_cast<double>(v.get_uint64());
break;
case boost::json::kind::double_:
st.num += v.get_double();
break;
default:
break;
}
}
#endif
static std::string slurp(const std::string& p)
{
std::ifstream f(p, std::ios::binary);
if (!f)
{
std::fprintf(stderr, "cannot open %s\n", p.c_str());
std::exit(1);
}
std::stringstream ss;
ss << f.rdbuf();
return ss.str();
}
static bool same(const stats& a, const stats& b)
{
return a.nodes == b.nodes && a.str == b.str && (a.num == b.num || std::fabs(a.num - b.num) <= 1e-9 * std::fabs(a.num));
}
int main(int argc, char** argv)
{
int rounds = 0; // 0: by size
std::vector<std::string> files;
for (int i = 1; i < argc; ++i)
{
if (std::strcmp(argv[i], "--rounds") == 0 && i + 1 < argc)
{
// (clamped: 0 rounds by size, at most a million)
rounds = static_cast<int>((std::min)((std::max)(std::strtol(argv[++i], nullptr, 10), 0L), 1000000L));
}
else
{
files.push_back(argv[i]);
}
}
std::FILE* csv = std::fopen("bench_corpus.csv", "w");
if (csv == nullptr)
{
std::perror("bench_corpus.csv");
return 1;
}
std::fprintf(csv, "file,bytes,workload,engine,ns\n");
json_document reused;
simdjson::dom::parser sj;
for (const auto& path : files)
{
const std::string s = slurp(path);
const std::string name = path.substr(path.rfind('/') + 1);
const simdjson::padded_string ps(s);
// all engines must agree before timing
stats a, b, c, d;
const json_document doc = json_document::parse(s);
walk(doc.root(), a);
yyjson_doc* y = yyjson_read(s.data(), s.size(), 0);
auto sjr = sj.parse(ps);
#if JSON_VIEW_BENCH_BOOST
boost::json::parse_options opt;
opt.numbers = boost::json::number_precision::precise;
boost::json::monotonic_resource mr0;
const boost::json::value bv = boost::json::parse(s, &mr0, opt);
#endif
if (y == nullptr || sjr.error())
{
std::printf("%-34s skipped (an engine rejects it)\n", name.c_str());
yyjson_doc_free(y);
continue;
}
walk(yyjson_doc_get_root(y), b);
walk(sjr.value_unsafe(), c);
#if JSON_VIEW_BENCH_BOOST
walk(bv, d);
#else
d = a;
#endif
yyjson_doc_free(y);
const bool ok = same(a, b) && same(a, c) && same(a, d);
const int r = rounds > 0 ? rounds : static_cast<int>(std::max<std::size_t>(3, std::min<std::size_t>(60, 400000000 / (s.size() + 1))));
struct engine
{
std::string name;
std::function<void()> fn;
};
json_document vd = json_document::parse(s);
yyjson_doc* yd = yyjson_read(s.data(), s.size(), 0);
yyjson_doc* yd_raw = yyjson_read(s.data(), s.size(), YYJSON_READ_NUMBER_AS_RAW);
simdjson::dom::parser sjd;
const simdjson::dom::element se = sjd.parse(ps).value_unsafe();
const std::vector<std::pair<std::string, std::vector<engine>>> workloads =
{
{
"parse", {
{"json_view", [&] { auto x = json_document::parse(s); g_sink = static_cast<double>(x.node_count()); }},
{"json_view (reused)", [&] { reused.read(s); g_sink = static_cast<double>(reused.node_count()); }},
{"yyjson", [&] { yyjson_doc* x = yyjson_read(s.data(), s.size(), 0); g_sink = static_cast<double>(yyjson_doc_get_val_count(x)); yyjson_doc_free(x); }},
{"simdjson DOM", [&] { auto e = sj.parse(ps).value_unsafe(); g_sink = e.is_object(); }},
{"simdjson DOM (fresh)", [&] { simdjson::dom::parser p; auto e = p.parse(ps).value_unsafe(); g_sink = e.is_object(); }},
#if JSON_VIEW_BENCH_BOOST
{"Boost.JSON", [&] { boost::json::monotonic_resource mr; auto v = boost::json::parse(s, &mr); g_sink = v.is_object(); }},
#endif
}
},
{
"traverse", {
{"json_view", [&] { auto x = json_document::parse(s); stats st; walk(x.root(), st); g_sink = st.num; }},
{"yyjson", [&] { yyjson_doc* x = yyjson_read(s.data(), s.size(), 0); stats st; walk(yyjson_doc_get_root(x), st); g_sink = st.num; yyjson_doc_free(x); }},
{"simdjson DOM", [&] { stats st; walk(sj.parse(ps).value_unsafe(), st); g_sink = st.num; }},
#if JSON_VIEW_BENCH_BOOST
{"Boost.JSON", [&] { boost::json::monotonic_resource mr; auto v = boost::json::parse(s, &mr); stats st; walk(v, st); g_sink = st.num; }},
#endif
}
},
{
"dump", {
{"json_view", [&] { std::string o = vd.root().dump(); g_sink = static_cast<double>(o.size()); }},
{"yyjson", [&] { std::size_t n = 0; char* o = yyjson_write(yd, 0, &n); g_sink = static_cast<double>(n); std::free(o); }},
{"simdjson DOM", [&] { std::string o = simdjson::to_string(se); g_sink = static_cast<double>(o.size()); }},
{"json_view (source numbers)", [&] { std::string o = vd.root().dump(-1, ' ', false, json_view::number_format::source); g_sink = static_cast<double>(o.size()); }},
{"yyjson (raw numbers)", [&] { std::size_t n = 0; char* o = yyjson_write(yd_raw, 0, &n); g_sink = static_cast<double>(n); std::free(o); }},
}
},
};
std::printf("%-34s %9zu B%s\n", name.c_str(), s.size(), ok ? "" : " [ENGINES DISAGREE]");
for (const auto& wl : workloads)
{
std::vector<double> best(wl.second.size(), 1e300);
for (int i = 0; i < r; ++i)
{
for (std::size_t k = 0; k < wl.second.size(); ++k)
{
// an untimed call first: whatever the previous engine left to the allocator
// (e.g. thousands of freed json nodes) is cleaned up here, not in the timing
wl.second[k].fn();
const auto t0 = std::chrono::steady_clock::now();
wl.second[k].fn();
best[k] = std::min(best[k], std::chrono::duration<double, std::nano>(std::chrono::steady_clock::now() - t0).count());
}
}
std::printf(" %-9s", wl.first.c_str());
for (std::size_t k = 0; k < wl.second.size(); ++k)
{
std::printf(" %s %.2f GB/s (%.2fx)", wl.second[k].name.c_str(), static_cast<double>(s.size()) / best[k], best[k] / best[0]);
std::fprintf(csv, "%s,%zu,%s,%s,%.1f\n", name.c_str(), s.size(), wl.first.c_str(), wl.second[k].name.c_str(), best[k]);
}
std::printf("\n");
std::fflush(stdout);
}
yyjson_doc_free(yd);
yyjson_doc_free(yd_raw);
}
std::fclose(csv);
}
+589
View File
@@ -0,0 +1,589 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// Read-modify-write benchmark: parse a document, apply the same logical edits
// with each library's own API, and serialize it (compact).
//
// json_view json_editable_document: edits in place, unchanged values stay in the index
// yyjson yyjson_read + yyjson_doc_mut_copy (the way to edit a parsed document)
// Boost.JSON parse into a mutable DOM (monotonic resource, precise numbers), serialize
// json::parse nlohmann::json today
// simdjson has no mutable document and is not part of this comparison.
//
// Workloads:
// patch a handful of edits at fixed places (scalars, a new member, a new array element)
// update edits in every record (twitter: 100 statuses, citm: 243 performances,
// canada: 480 rings, jeopardy: 216,930 questions): set scalars, erase a
// member, add a member (canada: replace the first point of every ring)
//
// Build: see README.md (same flags as bench_view.cpp).
#include <nlohmann/json_view.hpp>
#if JSON_VIEW_BENCH_BOOST
#include <boost/json.hpp>
#include <boost/json/src.hpp>
#endif
#include <yyjson.h>
#include <algorithm>
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <fstream>
#include <functional>
#include <sstream>
using nlohmann::json;
using nlohmann::json_editable_document;
using nlohmann::json_editable_view;
#if JSON_VIEW_BENCH_BOOST
namespace bj = boost::json;
#endif
static volatile std::size_t g_sink;
// ---------------- json_view ----------------
static std::string edit_view(const std::string& name, const std::string& s, bool update)
{
json_editable_document d = json_editable_document::parse(s);
const json_editable_view r = d.root();
if (name == "twitter")
{
if (update)
{
std::int64_t i = 0;
for (const json_editable_view st : r["statuses"])
{
d.set(st, "retweet_count", i++);
d.set(st, "favorited", true);
d.set(st, "text", "redacted");
d.erase(st, "entities");
d.set(st, "edited", true);
}
}
else
{
d.set(r["search_metadata"], "count", 200);
d.set(r["statuses"][0], "text", "patched");
d.set(r["statuses"][0]["user"], "followers_count", 1);
d.set(r["statuses"][99], "favorited", true);
d.set(r, "patched", true);
}
}
else if (name == "citm_catalog")
{
if (update)
{
for (const json_editable_view p : r["performances"])
{
d.set(p, "name", "performance");
d.set(p, "start", p["start"].get<std::int64_t>() + 1);
d.erase(p, "seatMapImage");
d.set(p, "edited", true);
}
}
else
{
d.set(r["events"]["138586341"], "name", "patched");
d.set(r["performances"][0], "start", 0);
d.set(r["venueNames"], "PLEYEL_PLEYEL", "Salle");
d.set(r, "patched", true);
}
}
else if (name == "canada")
{
const json_editable_view coords = r["features"][0]["geometry"]["coordinates"];
if (update)
{
for (const json_editable_view ring : coords)
{
d.set(ring, 0, json::array({0.5, 0.5}));
}
}
else
{
d.set(r["features"][0]["properties"], "name", "patched");
d.set(r, "type", "FeatureCollection2");
d.set(coords[0], 0, json::array({0.0, 0.0}));
}
}
else if (name == "jeopardy")
{
if (update)
{
for (const json_editable_view q : r)
{
d.set(q, "value", "$1");
d.erase(q, "air_date");
}
}
else
{
d.set(r[0], "value", "$0");
d.set(r[100000], "answer", "patched");
d.set(r[216929], "round", "x");
d.push_back(r, json::object({{"category", "NEW"}, {"value", "$5"}}));
}
}
else if (name == "status")
{
d.set(r, "retweet_count", 1);
d.set(r["user"], "name", "x");
}
else if (name == "rpc")
{
d.set(r, "id", 4);
d.set(r["params"], "subtrahend", 24);
}
return r.dump();
}
// ---------------- nlohmann::json ----------------
static std::string edit_json(const std::string& name, const std::string& s, bool update)
{
json r = json::parse(s);
if (name == "twitter")
{
if (update)
{
std::int64_t i = 0;
for (auto& st : r["statuses"])
{
st["retweet_count"] = i++;
st["favorited"] = true;
st["text"] = "redacted";
st.erase("entities");
st["edited"] = true;
}
}
else
{
r["search_metadata"]["count"] = 200;
r["statuses"][0]["text"] = "patched";
r["statuses"][0]["user"]["followers_count"] = 1;
r["statuses"][99]["favorited"] = true;
r["patched"] = true;
}
}
else if (name == "citm_catalog")
{
if (update)
{
for (auto& p : r["performances"])
{
p["name"] = "performance";
p["start"] = p["start"].get<std::int64_t>() + 1;
p.erase("seatMapImage");
p["edited"] = true;
}
}
else
{
r["events"]["138586341"]["name"] = "patched";
r["performances"][0]["start"] = 0;
r["venueNames"]["PLEYEL_PLEYEL"] = "Salle";
r["patched"] = true;
}
}
else if (name == "canada")
{
json& coords = r["features"][0]["geometry"]["coordinates"];
if (update)
{
for (auto& ring : coords)
{
ring[0] = json::array({0.5, 0.5});
}
}
else
{
r["features"][0]["properties"]["name"] = "patched";
r["type"] = "FeatureCollection2";
coords[0][0] = json::array({0.0, 0.0});
}
}
else if (name == "jeopardy")
{
if (update)
{
for (auto& q : r)
{
q["value"] = "$1";
q.erase("air_date");
}
}
else
{
r[0]["value"] = "$0";
r[100000]["answer"] = "patched";
r[216929]["round"] = "x";
r.push_back(json::object({{"category", "NEW"}, {"value", "$5"}}));
}
}
else if (name == "status")
{
r["retweet_count"] = 1;
r["user"]["name"] = "x";
}
else if (name == "rpc")
{
r["id"] = 4;
r["params"]["subtrahend"] = 24;
}
return r.dump();
}
// ---------------- yyjson ----------------
static std::string edit_yyjson(const std::string& name, const std::string& s, bool update)
{
yyjson_doc* idoc = yyjson_read(s.data(), s.size(), 0);
yyjson_mut_doc* d = yyjson_doc_mut_copy(idoc, nullptr);
yyjson_doc_free(idoc);
yyjson_mut_val* r = yyjson_mut_doc_get_root(d);
auto get = [](yyjson_mut_val * o, const char* k)
{
return yyjson_mut_obj_get(o, k);
};
if (name == "twitter")
{
yyjson_mut_val* sts = get(r, "statuses");
if (update)
{
std::size_t idx, max;
yyjson_mut_val* st;
std::int64_t i = 0;
yyjson_mut_arr_foreach(sts, idx, max, st)
{
yyjson_mut_set_sint(get(st, "retweet_count"), i++);
yyjson_mut_set_bool(get(st, "favorited"), true);
yyjson_mut_set_str(get(st, "text"), "redacted");
yyjson_mut_obj_remove_key(st, "entities");
yyjson_mut_obj_add_bool(d, st, "edited", true);
}
}
else
{
yyjson_mut_set_sint(get(get(r, "search_metadata"), "count"), 200);
yyjson_mut_val* s0 = yyjson_mut_arr_get(sts, 0);
yyjson_mut_set_str(get(s0, "text"), "patched");
yyjson_mut_set_sint(get(get(s0, "user"), "followers_count"), 1);
yyjson_mut_set_bool(get(yyjson_mut_arr_get(sts, 99), "favorited"), true);
yyjson_mut_obj_add_bool(d, r, "patched", true);
}
}
else if (name == "citm_catalog")
{
if (update)
{
std::size_t idx, max;
yyjson_mut_val* p;
yyjson_mut_arr_foreach(get(r, "performances"), idx, max, p)
{
yyjson_mut_set_str(get(p, "name"), "performance");
yyjson_mut_val* start = get(p, "start");
yyjson_mut_set_sint(start, yyjson_mut_get_sint(start) + 1);
yyjson_mut_obj_remove_key(p, "seatMapImage");
yyjson_mut_obj_add_bool(d, p, "edited", true);
}
}
else
{
yyjson_mut_set_str(get(get(get(r, "events"), "138586341"), "name"), "patched");
yyjson_mut_set_sint(get(yyjson_mut_arr_get(get(r, "performances"), 0), "start"), 0);
yyjson_mut_set_str(get(get(r, "venueNames"), "PLEYEL_PLEYEL"), "Salle");
yyjson_mut_obj_add_bool(d, r, "patched", true);
}
}
else if (name == "canada")
{
yyjson_mut_val* f0 = yyjson_mut_arr_get(get(r, "features"), 0);
yyjson_mut_val* coords = get(get(f0, "geometry"), "coordinates");
if (update)
{
static const double half[2] = {0.5, 0.5};
std::size_t idx, max;
yyjson_mut_val* ring;
yyjson_mut_arr_foreach(coords, idx, max, ring)
{
yyjson_mut_arr_replace(ring, 0, yyjson_mut_arr_with_real(d, half, 2));
}
}
else
{
static const double zero[2] = {0.0, 0.0};
yyjson_mut_set_str(get(get(f0, "properties"), "name"), "patched");
yyjson_mut_set_str(get(r, "type"), "FeatureCollection2");
yyjson_mut_arr_replace(yyjson_mut_arr_get(coords, 0), 0, yyjson_mut_arr_with_real(d, zero, 2));
}
}
else if (name == "jeopardy")
{
if (update)
{
std::size_t idx, max;
yyjson_mut_val* q;
yyjson_mut_arr_foreach(r, idx, max, q)
{
yyjson_mut_set_str(get(q, "value"), "$1");
yyjson_mut_obj_remove_key(q, "air_date");
}
}
else
{
yyjson_mut_set_str(get(yyjson_mut_arr_get(r, 0), "value"), "$0");
yyjson_mut_set_str(get(yyjson_mut_arr_get(r, 100000), "answer"), "patched");
yyjson_mut_set_str(get(yyjson_mut_arr_get(r, 216929), "round"), "x");
yyjson_mut_val* o = yyjson_mut_obj(d);
yyjson_mut_obj_add_str(d, o, "category", "NEW");
yyjson_mut_obj_add_str(d, o, "value", "$5");
yyjson_mut_arr_append(r, o);
}
}
else if (name == "status")
{
yyjson_mut_set_sint(get(r, "retweet_count"), 1);
yyjson_mut_set_str(get(get(r, "user"), "name"), "x");
}
else if (name == "rpc")
{
yyjson_mut_set_sint(get(r, "id"), 4);
yyjson_mut_set_sint(get(get(r, "params"), "subtrahend"), 24);
}
std::size_t n = 0;
char* out = yyjson_mut_write(d, 0, &n);
std::string result(out, n);
std::free(out);
yyjson_mut_doc_free(d);
return result;
}
#if JSON_VIEW_BENCH_BOOST
// ---------------- Boost.JSON ----------------
static std::string edit_boost(const std::string& name, const std::string& s, bool update)
{
bj::monotonic_resource mr;
bj::parse_options opt;
opt.numbers = bj::number_precision::precise; // correctly rounded, like the others
bj::value v = bj::parse(s, &mr, opt);
bj::object* const obj = v.if_object(); // nullptr for jeopardy (an array)
if (name == "twitter")
{
bj::array& sts = (*obj)["statuses"].as_array();
if (update)
{
std::int64_t i = 0;
for (auto& e : sts)
{
bj::object& st = e.as_object();
st["retweet_count"] = i++;
st["favorited"] = true;
st["text"] = "redacted";
st.erase("entities");
st["edited"] = true;
}
}
else
{
(*obj)["search_metadata"].as_object()["count"] = 200;
bj::object& s0 = sts[0].as_object();
s0["text"] = "patched";
s0["user"].as_object()["followers_count"] = 1;
sts[99].as_object()["favorited"] = true;
(*obj)["patched"] = true;
}
}
else if (name == "citm_catalog")
{
if (update)
{
for (auto& e : (*obj)["performances"].as_array())
{
bj::object& p = e.as_object();
p["name"] = "performance";
p["start"] = p["start"].as_int64() + 1;
p.erase("seatMapImage");
p["edited"] = true;
}
}
else
{
(*obj)["events"].as_object()["138586341"].as_object()["name"] = "patched";
(*obj)["performances"].as_array()[0].as_object()["start"] = 0;
(*obj)["venueNames"].as_object()["PLEYEL_PLEYEL"] = "Salle";
(*obj)["patched"] = true;
}
}
else if (name == "canada")
{
bj::object& f0 = (*obj)["features"].as_array()[0].as_object();
bj::array& coords = f0["geometry"].as_object()["coordinates"].as_array();
if (update)
{
for (auto& ring : coords)
{
ring.as_array()[0] = bj::array({0.5, 0.5});
}
}
else
{
f0["properties"].as_object()["name"] = "patched";
(*obj)["type"] = "FeatureCollection2";
coords[0].as_array()[0] = bj::array({0.0, 0.0});
}
}
else if (name == "jeopardy")
{
bj::array& a = v.as_array();
if (update)
{
for (auto& e : a)
{
bj::object& q = e.as_object();
q["value"] = "$1";
q.erase("air_date");
}
}
else
{
a[0].as_object()["value"] = "$0";
a[100000].as_object()["answer"] = "patched";
a[216929].as_object()["round"] = "x";
a.push_back(bj::object({{"category", "NEW"}, {"value", "$5"}}));
}
}
else if (name == "status")
{
(*obj)["retweet_count"] = 1;
(*obj)["user"].as_object()["name"] = "x";
}
else if (name == "rpc")
{
(*obj)["id"] = 4;
(*obj)["params"].as_object()["subtrahend"] = 24;
}
return bj::serialize(v);
}
#endif
// ---------------- harness ----------------
static std::string slurp(const std::string& p)
{
std::ifstream f(p, std::ios::binary);
if (!f)
{
std::fprintf(stderr, "cannot open %s\n", p.c_str());
std::exit(1);
}
std::stringstream ss;
ss << f.rdbuf();
return ss.str();
}
int main(int argc, char** argv)
{
if (argc < 2)
{
std::fprintf(stderr, "usage: %s <json_test_data directory> [rounds] [document]\n", argv[0]);
return 1;
}
const std::string T = std::string(argv[1]) + "/";
// (clamped to 1 to a million)
const int rounds = argc > 2 ? static_cast<int>((std::min)((std::max)(std::strtol(argv[2], nullptr, 10), 1L), 1000000L)) : 20;
const std::string only = argc > 3 ? argv[3] : "";
struct doc
{
std::string name, text;
int batch;
};
std::vector<doc> docs;
for (const char* f :
{"nativejson-benchmark/twitter.json", "nativejson-benchmark/citm_catalog.json", "nativejson-benchmark/canada.json", "jeopardy/jeopardy.json"
})
{
std::string n = std::string(f).substr(std::string(f).find('/') + 1);
docs.push_back({n.substr(0, n.size() - 5), slurp(T + f), 1});
}
docs.push_back({"status", json::parse(docs[0].text)["statuses"][0].dump(), 200});
docs.push_back({"rpc", R"({"jsonrpc": "2.0", "method": "subtract", "params": {"minuend": 42, "subtrahend": 23}, "id": 3})", 5000});
using fn = std::string (*)(const std::string&, const std::string&, bool);
const std::vector<std::pair<std::string, fn>> engines =
{
{"json_view", edit_view}, {"yyjson", edit_yyjson},
#if JSON_VIEW_BENCH_BOOST
{"Boost.JSON", edit_boost},
#endif
{"json::parse", edit_json}
};
std::FILE* csv = std::fopen("bench_edit.csv", "w");
if (csv == nullptr)
{
std::perror("bench_edit.csv");
return 1;
}
std::fprintf(csv, "doc,bytes,workload,engine,ns\n");
for (const auto& dc : docs)
{
if (!only.empty() && dc.name != only)
{
continue;
}
for (const bool update :
{
false, true
})
{
if (update && (dc.name == "status" || dc.name == "rpc"))
{
continue;
}
// all engines must produce the same value
const json expected = json::parse(edit_json(dc.name, dc.text, update));
bool ok = true;
for (const auto& e : engines)
{
ok = ok && json::parse(e.second(dc.name, dc.text, update)) == expected;
}
std::vector<double> best(engines.size(), 1e300);
const int r = dc.text.size() > 10000000 ? std::max(3, rounds / 4) : rounds;
for (int i = 0; i < r; ++i)
{
for (std::size_t k = 0; k < engines.size(); ++k)
{
// an untimed call first: whatever the previous engine left to the allocator
// (e.g. thousands of freed json nodes) is cleaned up here, not in the timing
g_sink = engines[k].second(dc.name, dc.text, update).size();
const auto t0 = std::chrono::steady_clock::now();
for (int b = 0; b < dc.batch; ++b)
{
g_sink = engines[k].second(dc.name, dc.text, update).size();
}
const double ns = std::chrono::duration<double, std::nano>(std::chrono::steady_clock::now() - t0).count() / dc.batch;
best[k] = std::min(best[k], ns);
}
}
const char* wl = update ? "update" : "patch";
std::printf("%-13s %-7s %s", dc.name.c_str(), wl, ok ? "" : "[OUTPUT MISMATCH] ");
for (std::size_t k = 0; k < engines.size(); ++k)
{
const double us = best[k] / 1e3;
std::printf(" %s %.*fus (%.2fx)", engines[k].first.c_str(), us < 10 ? 3 : (us < 1000 ? 1 : 0), us, best[k] / best[0]);
std::fprintf(csv, "%s,%zu,%s,%s,%.1f\n", dc.name.c_str(), dc.text.size(), wl, engines[k].first.c_str(), best[k]);
}
std::printf("\n");
std::fflush(stdout);
}
}
std::fclose(csv);
}
Loaded 100 of 113 files, more files were not shown because too many files have changed in this diff. Show more