Compare commits

..
86 Commits
Author SHA1 Message Date
Niels Lohmann fa01cf20dc Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:33 +02:00
Niels Lohmann 4de0133cfc Move the dump tests of this branch to unit-json_view_dump.cpp
The dump and comparison tests moved to their own file in the dump pull
request; the tests of the fast dump() join them there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:32 +02:00
Niels Lohmann d547e3bfec Split the value and pointer edit tests off unit-json_view_edit.cpp
The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view_edit.cpp has 35409 sections since the fast
dump instantiates more code for editable documents, so its test cases "views
and values", "deeply nested values", "pointers below a null value", and
"strings of other documents are checked" move into
unit-json_view_edit_values.cpp. The helper exception_of_call() that both
files use moves into json_view_test_helpers.hpp.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:31 +02:00
Niels Lohmann 8af1dc4ef4 Write doubles from their bits only where double is IEEE 754 binary64
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:30 +02:00
Niels Lohmann 5331e1ad6f Fix clang-tidy 22 findings in unit-to_chars.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:30 +02:00
Niels Lohmann e3423b2a0a Test write_short_decimal and powers_of_ten_16 against to_chars
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:29 +02:00
Niels Lohmann fea6e8dd4b Use a named document in the dump test: root() of a temporary does not compile
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:28 +02:00
Niels Lohmann d61e61baa1 Tidy the view serializer: doxygen block placement, enabled() for a constant
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:27 +02:00
Niels Lohmann 26b37690e9 Fix out-of-bounds read when dumping a float token of an unchecked image
A float node of an image loaded with image_check::bounds can hold a
token whose bytes are not digits, so its value need not have the digit
count recorded in the node. write_double_at() passed that count to
write_short_decimal(), which indexes its table of powers of ten by it:
an assertion failure in debug builds, an out-of-bounds read in release
builds. Use the counted overload only if the value has exactly that many
digits, otherwise count them.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:26 +02:00
Niels Lohmann dd4773e8c0 Fix MSVC C4127 and Infer finding in the view serializer
Test the template options through enabled() so that conditions combined
with them are not constant, and mark the nav::value() null dereference
as a false positive like materialize.hpp does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:25 +02:00
Niels Lohmann bad66b8707 Write json_view's dump() without a call or conversion per token
The default dump() (no indentation, no ensure_ascii) gets its own
writer that makes the same walk and produces the same output:

- the write position stays in a local variable instead of a
  member, so the compiler keeps it in a register across stores
  through aliasing char pointers;
- strings and number tokens are copied with fixed-size 32-byte
  moves wherever enough source bytes remain, instead of one
  memcpy call per token;
- the innermost open container lives in local variables; a stack
  that starts as a local array of 32 entries holds the rest;
- unedited documents are walked through the node array in order,
  and integer tokens are read from the source directly.

On top of that, float tokens of at most 15 significant digits are
written straight from their digits via zmij::to_shortest() and
write_shortest(), without converting to a double and back: such
decimals are farther apart than a double's rounding interval, so
the token's digits are the double's shortest digits. Tokens of
16+ digits, or edited values, still go through decimal_to_float().
The view's own NEON write_decimal() is removed in favor of the
shared writer, and the dump output now grows in 64 KiB steps
instead of being resized to its estimate at once.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:25 +02:00
Niels Lohmann 0c9aab60c9 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:24 +02:00
Niels Lohmann 84367f79d1 Build the image link test without exceptions
load_result() exists only with exceptions; with JSON_NOEXCEPTION the test
loads the image with the check directly (a failed check aborts).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:23 +02:00
Niels Lohmann b12ddeee94 Silence a clang-tidy C++17 finding in the image test
bugprone-suspicious-stringview-data-usage flags data() of the source view
under C++17, although the length is passed and checked by the REQUIRE
before it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:22 +02:00
Niels Lohmann 540db5c385 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- drop edit_storage.hpp, which edit.hpp includes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:21 +02:00
Niels Lohmann ab7f9a2027 Fix the indentation of a loop in the image test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:21 +02:00
Niels Lohmann ebf991636d Fix MSVC C4244 and C2017 in the image test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:20 +02:00
Niels Lohmann ed79a4b1aa Fix clang-tidy 22 findings in unit-json_view_image.cpp and image.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:19 +02:00
Niels Lohmann 6dd4b03202 Avoid useless casts of header fields in the image tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:18 +02:00
Niels Lohmann df22c17c9b Expect first-wins lookups in the image tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:17 +02:00
Niels Lohmann 4a4501f3ea Document that editable documents refuse invalid UTF-8 copied from a loaded image
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:16 +02:00
Niels Lohmann 93a69864c6 Test that invalid UTF-8 of a loaded image is not copied into an editable document
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:15 +02:00
Niels Lohmann efe02e0741 Release the list of large objects once load() has built the indexes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:15 +02:00
Niels Lohmann a6efb9335a Test lookups in loaded images with duplicate and colliding keys
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:14 +02:00
Niels Lohmann f274a1b70d Test that the targets of links never reach an image
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:13 +02:00
Niels Lohmann 12610c5664 Adapt the image tests to named documents and last-wins lookups
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:12 +02:00
Niels Lohmann 7e9b99635a Check each distinct string and float range once in the full image check
The full check scanned the contents of every string node and every float token over its own range, so many nodes pointing to one large range made load() quadratic. The structural walk now only collects the ranges; after it, each kind is sorted and checked once per distinct range. Ranges of one kind that overlap without being identical are rejected, as save() never writes them (nodes that share a value share the whole range). The cost is linear in the size of the image plus sorting the ranges.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:11 +02:00
Niels Lohmann 71e4c84e91 Fix MSVC errors in image test and add image_check doc anchor
Move the raw string literal out of the CHECK macro (MSVC preprocessor),
cast 64-bit header fields to std::size_t (C4244 on 32-bit), and give the
image_check section in load.md a real anchor for the mkdocs strict build.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:11 +02:00
Niels Lohmann 949df24b1a Add images of json_documents: save() and load()
An image is a document stored so that loading it needs no
parsing: save() writes the node index, the text and the decoded
strings; the static load() reads an image written by save().

load() takes a pointer and size, a borrowed vector, or an owned
rvalue vector; the nodes are copied so they are aligned and can
be edited, while the text and decoded strings stay in the image.

image_check controls how much load() trusts the input: full
checks structure, bounds, strings and numbers, the parser's own
guarantees; bounds checks structure and bounds only; none skips
all checks, for images from a trusted source.

Layout is little-endian only ("NJVI" header, nodes, text, decoded
strings), following the idea of zero-copy formats such as
FlatBuffers and YaFF; the check follows FlatBuffers' Verifier.

New errors: parse_error.116 for a malformed image or a failed
check, type_error.320 for a discarded document or a big-endian
target.

A dedicated fuzzer and 6,000 seeded corruptions, checked under
ASan/UBSan, found and fixed two gaps: unchecked reserved header
fields, and unbounded null/boolean offsets that could make
dump() throw std::length_error.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:10 +02:00
Niels Lohmann 2a3293079d Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:09 +02:00
Niels Lohmann fb39dc807e Keep the view's edit headers clean for clang-tidy in C++17 mode
clang-tidy 22 checks the headers with -std=gnu++17 as well, where it asks
for a transparent comparator and std::make_unique. The regions map uses its
default comparator, and the allocation of the edit state keeps new (the
library is C++11) with a NOLINT.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:08 +02:00
Niels Lohmann e86cd18fca Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), and the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), plus <version> for
std::nullptr_t, and removing <cstddef> and edit_storage.hpp.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- drop edit_storage.hpp, which edit.hpp includes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:07 +02:00
Niels Lohmann f33fdebb27 Keep a raw string with a backslash out of a CHECK (MSVC C2017)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:06 +02:00
Niels Lohmann 2135653299 Fix clang-tidy 22 findings in unit-json_view_edit.cpp and edit.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:06 +02:00
Niels Lohmann bd54e27937 Fix clang-tidy 22 findings in the json_view headers
Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast
would trip -Wcast-align=strict), use auto for a cast initialiser, add
parentheses to a mixed expression, and name bugprone-std-namespace-modification
in the NOLINTs of the tuple_size/tuple_element specialisations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
(cherry picked from commit 0effbaa7a2)
2026-10-14 17:54:05 +02:00
Niels Lohmann 736639bb39 Fix MSVC C4127 in the lookups of editable json documents
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:04 +02:00
Niels Lohmann 5c7e407f51 Assign the first member when set() meets duplicate keys again
Lookups find the first member of a repeated key again, so set(object, key,
value) assigns that member (keeping the key where it is) and drops the
others, as documented before, and a view taken from object[key] shows the
new value. This undoes the code and documentation changes of 5cbb8e6d6.
Lookups in edited objects (navigation of editable documents) stop at the
first match, like those of parsed objects.

The tests that commit added expect the first member from lookups now. In the
seeded differential test, the documents with repeated keys repeat them after
the real members; the reference holds the first members, which the edits
address and the lookups are compared with, while materialize() is compared
with what parse() makes of the document's dump.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:03 +02:00
Niels Lohmann b861264127 Assign the member that lookups find when set() meets duplicate keys
An editable document reads the last member of a repeated key (as the
read-only document does), but set(object, key, value) assigned the first
one: a view taken from object[key] before the call did not show the new
value. It now assigns the last member's value, keeps the key at the
position of its first occurrence (where materialize() puts it), and drops
the other members.

The seeded differential test now also edits documents whose objects
repeat keys and compares every lookup (operator[], at, find, value,
contains, count, and JSON pointers) with the parsed basic_json value.
A targeted test covers duplicates before and after edits that move the
object, in large objects with an index, in moved arrays, and in values
copied from other documents.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:02 +02:00
Niels Lohmann 5e25168741 Update the json_view banner and type_error.319 for editable documents
The banner still described a read-only view, and the documentation of
type_error.319 listed set and push_back only, although insert rejects
binary values as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:02 +02:00
Niels Lohmann 6da03bd6a3 Replace a container by a scalar in constant time once it is linked
Replacing an array or object that spans several nodes by a scalar looks
up its parent from the root (find_parent), which is linear in the size
of the document: setting every element of a 20,000 element array took
more than half a second. The parent only has to switch to links once;
afterward the extent of the replaced value no longer matters. Nodes that
an entry of a moved sequence links to are now marked (node_flags::linked),
and assign() skips the lookup for them.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:01 +02:00
Niels Lohmann c9691f2d61 Set a null value below a pointer as basic_json does
set(json_pointer, value) turned a null parent into an object for every
last reference token, so "/a/0" and "/a/-" below {"a":null} made
{"a":{"0":1}}, where basic_json's operator[](json_pointer) makes an
array. A null parent now becomes an array for "-" and for digit tokens
(filled with nulls up to the index) and an object otherwise. An invalid
index is reported before the parent changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:54:00 +02:00
Niels Lohmann 70fd3d891a Allow the edit arena to grow up to 4 GiB - 1 bytes
append_text() doubled the capacity of the arena and rejected the result
if it exceeded 4 GiB, so that an arena of more than 2 GiB could not grow
even though the 32-bit offsets of nodes address 4 GiB - 1 bytes. The
capacity is now clamped to that limit (text_capacity()), and an append
is rejected only if the bytes themselves do not fit.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:59 +02:00
Niels Lohmann dddc9be9b8 Check strings copied from another document for valid UTF-8
An editable document only holds valid UTF-8, but copy_scalar() copied the
strings and keys of a view of another document unchecked. A document of
a weaker check (or a borrowed text that changed after parsing) could
therefore bring ill-formed UTF-8 into it. The copy is checked now, with
the error that dump() reports for the string; copies within the same
document stay unchecked.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:58 +02:00
Niels Lohmann 87345f71da Do not leave a half-assigned container when set_moved() throws
assign() rewrote the kind, length and extent of a slot before set_moved()
registered its new element sequence. set_moved() can throw
(std::bad_alloc from reserving the bookkeeping vectors) for a slot that
is not moved yet, which left a container without the moved flag that
showed its old children. The bookkeeping is now reserved first
(reserve_moved()), so that nothing after the first write to the slot can
throw. block_of() reserves before it marks anything for the same reason.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:57 +02:00
Niels Lohmann 68196aa445 Copy nested values into a document without recursion
Counting and copying the nodes of a view or a basic_json value into an
editable document recursed once per nesting level, so that a deeply
nested value overflowed the stack. The four functions now walk the value
with an explicit stack, as materialize() does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:57 +02:00
Niels Lohmann f2bf9b7bf7 Fix old clang and MSVC C4127 in editable json documents
Value-initialize the const std::less in find_parent (clang 3.4/3.6 do not
implement DR 253), and test the Editable template argument through a
function to avoid MSVC C4127.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:56 +02:00
Niels Lohmann 1d29b01d84 Add editable json_documents: set, push_back, insert, and erase
basic_json_document gets a second template parameter, Editable
(false by default), plus the aliases json_editable_document,
json_editable_view, ordered_json_editable_document and
ordered_json_editable_view.

Editable documents can change values and structure without
rewriting the source text: set()/push_back() on values, keys,
array indices and JSON pointers; insert() before an array
element; erase() of an object key, array index or JSON pointer.

New values and element sequences go into edit storage that the
document owns and never moves, so views keep referring to their
value across edits and a parsed node never moves. Read-only
documents walk the plain node array and are unaffected.

Strings are checked for UTF-8 on entry, so dump() of an editable
document never throws type_error.316. Binary values cannot be
stored (type_error.319).

A seeded differential test applies random edits to an editable
document and to the equivalent ordered_json and compares both
after every step.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:55 +02:00
Niels Lohmann 6d72dad8fd Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:54 +02:00
Niels Lohmann 489a1a120a Move the dump, comparison, and large-object tests to unit-json_view_dump.cpp
The dump and comparison tests moved to their own file in the dump pull
request; the hash index tests of this branch join them there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:53 +02:00
Niels Lohmann 934655fc88 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:52 +02:00
Niels Lohmann 327832c968 Index strings with size_t in the colliding-keys test (MSVC C4244 on Win32)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:52 +02:00
Niels Lohmann d1dcc11adc Avoid GCC useless-cast and strict-overflow warnings in unit-json_view.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:51 +02:00
Niels Lohmann 5b256af6f6 Fix clang-tidy 22 findings in unit-json_view.cpp and unit-json_view_builder.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:50 +02:00
Niels Lohmann c562bbb5c8 Fix clang-tidy 22 findings in the json_view headers
Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast
would trip -Wcast-align=strict), use auto for a cast initialiser, add
parentheses to a mixed expression, and name bugprone-std-namespace-modification
in the NOLINTs of the tuple_size/tuple_element specialisations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:49 +02:00
Niels Lohmann ccca948c23 Keep the first of duplicate keys in the object hash index again
Lookups in objects with 128 members or more return the first member of a
repeated key again, like the linear search of smaller objects. This undoes
the code change of a69542046; its test now expects the first member from
lookups (with and without a table) and the last value from materialize()
and basic_json::parse().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:48 +02:00
Niels Lohmann f7a9f9bd69 Keep the last of duplicate keys in the object hash index
Lookups in objects with 128 members or more now return the last member of
a repeated key, like the linear search of smaller objects and like
materialize() and basic_json::parse(). build_object_index let the first
occurrence win, so the same text gave different results depending on the
size of the object.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:47 +02:00
Niels Lohmann 22e7a3c27a Document the hash index number in node::extra
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:47 +02:00
Niels Lohmann d5fd35ec95 Use the portable string scan on non-x86 targets that define __SSE2__
WebAssembly with -msse2 defines __SSE2__, but has no <cpuid.h> or x86 intrinsics headers.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:46 +02:00
Niels Lohmann c50d10be76 Release the spare capacity of the hash index in shrink_to_fit
shrink_to_fit() now trims the tables of large objects like the node array and the decoded strings, and the list of large objects is released as soon as the tables are built.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:45 +02:00
Niels Lohmann 40d518f2d7 Bound the probe length of the large-object hash index
The key hash is not seeded, so keys chosen to collide made building the table quadratic (20,000 colliding keys took 470 ms to parse). A key may now sit at most 64 slots from its home slot; if a key would sit further away, the table is dropped and the object is searched linearly. Lookups stop after the same distance.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:44 +02:00
Niels Lohmann 4a70524e9b Remove an accidentally committed GCC module cache
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:43 +02:00
Niels Lohmann d09643e3e3 Fix GCC module build and MSVC C4127 in the json_view SIMD code
GCC ignores the target attribute in modules, so the SSSE3 dispatch is
disabled for the module interface (the check stays portable, SSE2 is kept).
Make the 8-vs-16 byte unrolling condition in scan_string_run a
preprocessor/template split to avoid a constant condition (C4127).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:42 +02:00
Niels Lohmann 890918a310 Scan json_view strings with SIMD and index large objects
Speed up json_view's parser with SIMD scanning and a hash table
for large objects.

Long runs of string bytes are scanned 16 bytes at a time with NEON
(AArch64, GCC and Clang) and SSE2 (x86-64), both baseline
instruction sets. Keys keep 16 table checks before the vector
loop, because their lengths repeat from record to record; string
values get 8, because their lengths vary more. Non-ASCII text is
validated 16 bytes at a time with simdjson's "lookup4" check
(Keiser and Lemire, 2021), with NEON on AArch64 and, on x86-64,
with SSSE3. SSSE3 is not part of baseline x86-64, so the check is
compiled for SSSE3 with a function attribute and used only where
CPUID reports it, which all x86-64 CPUs since about 2011 do; the
answer is cached in a statically initialized atomic, so there is
no guard of a local static and no global constructor. The same
input is accepted either way. JSON_VIEW_NO_SIMD selects the
portable code.

On x86-64, string runs are now checked vector-first: one SSE2
compare from the first byte finds the end of most keys and short
values, instead of a branch per byte for the first 8-16 bytes.
AArch64 keeps the byte-wise steps, where a NEON mask costs more and
the branches predict well. Entering an object or array no longer
stalls: open() stores the parent's frame field by field instead of
building it on the stack and reading it back with wider loads,
which waited for the narrower stores to retire.

Objects with 128 members or more get an open-addressing hash table
built when the object closes, so operator[], at(), find(),
contains(), count(), value(), and JSON pointers take constant time
on average in such objects; of duplicate keys, the first is kept,
as for the linear search. The idea comes from Boost.JSON.

simdjson is credited in simd.hpp's SPDX block, the README, and
license.md.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:41 +02:00
Niels Lohmann ad89113e82 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:41 +02:00
Niels Lohmann 30fc666df7 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:40 +02:00
Niels Lohmann 913bdc26ec Split the dump and comparison tests off unit-json_view.cpp
The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view.cpp reaches that limit as the stack
grows, so its "json_view dump" and "json_view comparison" test cases move
into unit-json_view_dump.cpp. The test generator and has_duplicate_keys()
that both files use move into json_view_test_helpers.hpp.

The new file mentions JSON_HAS_CPP_17, so it is built for C++17 like the file
it was split from.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:39 +02:00
Niels Lohmann 9178d1c40c Fix the clang-tidy 22 findings in json_view and its tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:38 +02:00
Niels Lohmann 9e05cb7942 Document and test first-wins lookups next to dump() and ==
dump() writes every member and == resolves duplicate keys as parse()
does, whereas lookups find the first member of a duplicate key.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:37 +02:00
Niels Lohmann 47f24b64f3 Do not shrink the output of dump()
finish() copied the whole output when the buffer was more than twice as
large as the result (citm dump +11%). The tighter source_extent()
estimate already keeps the buffer of a small value small.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:36 +02:00
Niels Lohmann 5bc15dd9ec Hold the documents of the dump and comparison tests in names
root() of a temporary document is deleted: its views would dangle.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:36 +02:00
Niels Lohmann 0a4fa95d37 Size the dump buffer of a small value by its own extent
The source extent of a value was read from the next node, falling back to the rest of the document when that node held a decoded string. dump() of a small value could thus allocate a buffer as large as the document. Skip a few such nodes, cap the fallback estimate, and shrink a buffer that is much larger than its output.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:35 +02:00
Niels Lohmann 58510535b8 Avoid raw string with escapes inside CHECK macro (MSVC C2017)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:34 +02:00
Niels Lohmann 6a4adb908d Add dump() and comparisons to json_view
Add basic_json_view::dump() and the comparison operators, and read
floats from the parser's digit layout instead of rescanning the
token.

dump(indent, indent_char, ensure_ascii, number_format) writes a
value the way ordered_json::parse(text).dump() writes it for the
same arguments: members in document order, all of them should a
key occur more than once; strings escaped by the same rules, using
the library's scanning kernels; floats written with the library's
to_chars conversion, so the output equals basic_json's byte for
byte; integers copied from the source, where they are already
canonical, except -0, which parse() reads as 0. There is no
error_handler argument, because the view only holds valid UTF-8.
number_format::source copies numbers exactly as they appear in the
source (e.g. "1.50", "1E2", "-0"), which basic_json cannot provide.
operator<< takes the indentation from the stream width, as for
basic_json. The writer walks iteratively, so nesting depth is
limited by memory only.

operator== and operator!= compare two views, or a view and a
basic_json value in either order, by the rules basic_json's
operator== uses: numbers compare by value across their types,
objects compare by their members with duplicate keys resolved as
parse() resolves them, member order matters only where the object
type keeps one, and discarded views compare as discarded basic_json
values do, including under JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON.
Nothing is materialized except single scalars.

While parsing, the view now records where the integer digits, the
fraction digits, and the exponent of a float token are, so floats
and doubles with at most 19 digits are read from that layout with
the library's decimal_to_float() instead of rescanning the token.
Both round correctly, so the values are those of parse(). get<double>(),
materialize(), dump(), and the comparisons all use it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:33 +02:00
Niels Lohmann 29b3c0c133 Report one-character non-numeric array indices like basic_json in the view
Develop's #5800 made basic_json report a JSON pointer array index such as
"/a" as parse_error.109 instead of out_of_range.404, as longer ones were;
the view's array_index() follows, with "-" and the empty token still
unresolved.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:32 +02:00
Niels Lohmann 4fe127d2f9 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:31 +02:00
Niels Lohmann 9105130cd6 Fix the clang-tidy 22 findings in json_view and its tests
The same changes as on the dump branch, where they were first made, so
that this branch passes clang-tidy on its own.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:31 +02:00
Niels Lohmann 26820493c7 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:30 +02:00
Niels Lohmann 5ed7ac57bc Avoid GCC useless-cast warnings in the integer index tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:29 +02:00
Niels Lohmann 45fb3ee7e2 Make json_view.hpp pass include-what-you-use
Mark the includes that are needed by design (json.hpp, the macro
unscope, <tuple> for the tuple_size specialization), take size_t from
<cstring>, and drop the forward declaration of basic_json_document that
the friend declaration makes redundant.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:28 +02:00
Niels Lohmann 0460fc997b Do not force-inline the view's functions on MSVC
__forceinline makes MSVC report C4714 for function templates it cannot
inline (get<std::string>), an error under /WX. The forced inlining is
only a performance hint, so use plain inline there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:27 +02:00
Niels Lohmann ff3cb67362 Return the first member of duplicate keys from lookups again
Looking up the last member of a duplicate key cannot stop at a match, so
every lookup scanned the whole object (1.6 to 3.4 times slower for small
objects). operator[](key), at, find, value, contains, count, and JSON
pointer resolution return the first member again, as yyjson and simdjson
do; materialize() and get<map>() keep the last value, as parse().

The documentation says so in the feature page and on each lookup page,
and explains how to get the value parse() would give. The integer index
templates and the discarded chaining of operator[] stay.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:27 +02:00
Niels Lohmann 246d8527db Test discarded views with is_discarded() in the operator[] docs and examples
The explicit conversion to bool of basic_json_view is gone.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:26 +02:00
Niels Lohmann b7bd190efa Use named documents in the json_view tests that called root() on a temporary
root() of an rvalue document is deleted, because the views would dangle.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:25 +02:00
Niels Lohmann ca9db1910b Replace the removed operator bool of basic_json_view in the header and its tests
A view is tested with is_discarded(); the lookups in at(), value() and
contains(json_pointer) and the unit tests no longer rely on the explicit
conversion to bool.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:24 +02:00
Niels Lohmann c71fe8da35 Make view lookups last-wins and chained access safe
* Lookups (operator[], at, find, contains, count, value, JSON pointers)
  return the last member of a duplicate key, as materialize() and
  parse() keep it.
* operator[] on a discarded view returns a discarded view instead of
  throwing, so v["a"]["b"] is safe for a missing "a".
* operator[] and at() take any integer type (not only int and size_t),
  fixing ambiguous calls with unsigned, long, std::int64_t, ...
* Fix the operator[] documentation, which claimed a discarded view for
  a type mismatch where type_error.305 is thrown.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:23 +02:00
Niels Lohmann e9e97f1d3e Add element access, iteration, values, and JSON pointers to json_view
Give basic_json_view the read-only access functions of basic_json:
operator[] and at() with keys and indices, front()/back(), find(),
contains(), count(), begin()/end() and cbegin()/cend(), items()
with structured bindings from C++17 on, and type_name().

Exceptions have the ids and messages of the const functions of
basic_json. Where basic_json has undefined behavior the view
answers safely: operator[] with a missing key or an out-of-range
index returns a discarded view, and front()/back() of an empty
container throw invalid_iterator.214. Objects are iterated in
document order, and all members are visited; duplicate-key lookups
find the first member (as yyjson and simdjson do), while parse(),
materialize(), and the map conversions keep the last value, as
parse() does. Keys of up to 16 bytes are compared with two
overlapping loads.

Add value conversions: get<T>()/get_to() for arithmetic types,
bool, nullptr_t, strings (std::basic_string copied,
string_view_t without a copy), BasicJsonType, views, std::vector,
and maps with string keys; get_string() for the string without a
copy; number_token() for the number exactly as written in the
source; value() with keys and JSON pointers; and operator[]/at()/
contains() with JSON pointers. Everything else, including types
with from_json(), goes through materialize() of that subtree.
get<T>() of arithmetic types is inlined down to the conversion, so
reading an integer needs no call.

detail::json_pointer_access exposes a pointer's reference tokens
to code outside basic_json.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:53:22 +02:00
30 changed files with 565 additions and 1257 deletions

No files matched your search

@@ -12,11 +12,6 @@ Checks whether the input is valid JSON, accepting and rejecting exactly what
function never throws an exception for invalid input, and the returned `#!cpp bool` is the only result -- no document
is returned.
The result is the same as that of `BasicJsonType::accept()` by design; the difference is how the check is made. This
function runs the document's index builder (and discards the index), while `BasicJsonType::accept()` runs the SAX
parser with a handler that ignores every event. Use this one if you go on to parse the same input with
[`parse()`](parse.md) anyway, so that both use the same code; either is fine for a plain yes/no answer.
## Template parameters
`InputType`
@@ -16,8 +16,8 @@ basic_json_document(const basic_json_document&) = delete;
2. Move constructor. Takes over `other`'s index and, if owned, its text; `other` is left as an empty document. Views
taken from `other` before the move remain valid, because the index is heap-allocated independently of the
`basic_json_document` object.
3. `basic_json_document` is move-only. Copying is disabled because of its cost: a copy would duplicate the whole
index, the decoded-string buffer, and the owned text, while a move is enough in every use we know of.
3. `basic_json_document` is move-only. Copying is disabled because it would either duplicate a potentially large index
and text, or leave two documents claiming to borrow the same buffer.
## Parameters
@@ -30,7 +30,7 @@ No-throw guarantee: the default and move constructors never throw exceptions.
## Complexity
Constant.
Constant, for the default and move constructors.
## Examples
@@ -11,14 +11,13 @@ A parsed JSON text, held as a flat index of its values
([16 bytes per value](../../home/architecture.md#node-index-of-json-views)) instead of a tree of `BasicJsonType` values.
Strings and numbers stay in the source text; only strings that contain escapes are decoded, into one buffer owned by
the document. [`basic_json_view`](../basic_json_view/index.md) is a read-only handle to one value of a
`basic_json_document`; [`materialize()`](../basic_json_view/materialize.md) turns a subtree into the
`basic_json_document`; [`materialize()`](../basic_json_view/materialize.md) turns a subtree back into the
`BasicJsonType` value that [`BasicJsonType::parse()`](../basic_json/parse.md) would have produced for it.
A document may **borrow** the text it was parsed from (the caller's buffer must then outlive the document) or **own**
it (a copy, or an rvalue `#!cpp std::string` that was moved in); see [`owns_source`](owns_source.md). `basic_json_document`
is move-only: a copy would duplicate the whole index, the decoded-string buffer, and the owned text, and a move is
enough in every use we know of. The index, the decoded strings, and the owned text live in one heap block that the
document holds through a `#!cpp std::unique_ptr`; this is also why views survive a move of the document.
is move-only: copying a document would either duplicate a potentially large index and text, or leave two documents
claiming to borrow the same buffer, so it is disabled.
With `#!cpp Editable == true`, the document also offers [`set`](set.md), [`push_back`](push_back.md),
[`insert`](insert.md), and [`erase`](erase.md) to change values in place, see [Edits](#edits) below. The source text
@@ -4,7 +4,7 @@
bool owns_source() const noexcept;
```
Returns whether the document holds its own copy of the parsed text or is borrowing the caller's buffer.
Returns whether the document holds its own copy of the parsed text, as opposed to borrowing the caller's buffer.
## Return value
@@ -21,15 +21,13 @@ static basic_json_document parse(IteratorType first, IteratorType last,
Both overloads accept the same JSON text as [`BasicJsonType::parse()`](../basic_json/parse.md), with the same
`ignore_comments`/`ignore_trailing_commas` options, but build a [`basic_json_document`](index.md) (a flat index into
the input) instead of a tree of `BasicJsonType` values. They take the same inputs as `BasicJsonType::parse()`. Byte
inputs that stay alive can be borrowed; all others (streams, `#!cpp FILE*`, wide characters, non-contiguous
iterators) are read into a copy that the document owns -- see the Notes. Use [`parse_copy()`](parse_copy.md) to get an
owned copy of a borrowable input as well.
the input) instead of a tree of `BasicJsonType` values. The input must be byte-oriented (see the template parameters
below): not every input type of `BasicJsonType::parse()` is supported.
## Template parameters
`InputType`
: A compatible input, for instance:
: A byte-oriented input, one of:
- a `#!cpp std::string`, `#!cpp std::string_view`, or a C-style array of single-byte characters
- a pointer to a null-terminated string of single-byte characters (`#!cpp char`, `#!cpp signed char`,
@@ -37,10 +35,12 @@ owned copy of a borrowable input as well.
- a container for which `#!cpp obj.data()` and `#!cpp obj.size()` give contiguous single-byte access, e.g.
`#!cpp std::vector<char>` or `#!cpp std::vector<std::uint8_t>`
- an `#!cpp std::istream` object
- a `#!cpp FILE*` pointer
- wide characters (`#!cpp wchar_t`, `#!cpp char16_t`, `#!cpp char32_t`), which are converted to UTF-8: a wide
string object (`#!cpp std::wstring`, `#!cpp std::u16string`, `#!cpp std::u32string`), an array (e.g. a wide
string literal), or a pointer to a null-terminated wide string
- a wide string object (`#!cpp std::wstring`, `#!cpp std::u16string`, `#!cpp std::u32string`), which is converted
to UTF-8
Other inputs are not supported: a `#!cpp FILE*`, and pointers to or arrays of wide characters (`#!cpp wchar_t`,
`#!cpp char16_t`, `#!cpp char32_t`) are rejected at compile time by a `#!cpp static_assert`. (Use
[`BasicJsonType::parse()`](../basic_json/parse.md) for these.)
`IteratorType`
: an input iterator type, for instance a pair of pointers such as `ptr` and `ptr + len`, or a pair of
@@ -78,9 +78,7 @@ discarded; see [`is_discarded`](is_discarded.md).
Throws the same exception [`BasicJsonType::parse()`](../basic_json/parse.md) throws for the same input and options --
the same exception id, message, and position -- because on a failing input the library's own parser is run on the
same bytes to produce the diagnostic. This replay runs only for a failing input, and costs about a lexing pass up to
the error (its SAX handler builds no values); with `allow_exceptions` set to `#!cpp false`, it is skipped and a
discarded document is returned instead. Additionally throws
same bytes to produce the diagnostic. Additionally throws
[`out_of_range.416`](../../home/exceptions.md#jsonexceptionout_of_range416) if the input is 4294967280 bytes (4 GiB
minus 16 bytes) or larger, a size [`BasicJsonType::parse()`](../basic_json/parse.md) does not reject.
@@ -97,7 +95,7 @@ Linear in the length of the input.
| lvalue byte container (`std::string`, `std::vector<char>`, ...), `std::string_view`, C string, character array | **borrowed** -- `input` must outlive the document |
| non-const rvalue `#!cpp std::string` | **owned**, moved in without a copy |
| other rvalue byte container (including a `#!cpp const` rvalue `#!cpp std::string`) | **owned**, copied |
| stream, `#!cpp FILE*`, wide characters, or anything else read through the general input adapter | **owned**, read into a buffer (a stream is read to its end) |
| stream, wide string, or anything else read through the general input adapter | **owned**, read into a buffer (a stream is read to its end) |
For overload (2), a pair of pointers to single-byte integers (e.g. `#!cpp const char*`, `#!cpp std::uint8_t*`) is
borrowed. From C++20 on, so is any other contiguous iterator over single bytes, such as
@@ -47,9 +47,7 @@ Every view taken from `#!cpp *this` before the call -- including the previous [`
whether or not the new parse succeeds; take fresh views from [`root()`](root.md) afterward.
`input` is borrowed or owned by the same rules as [`parse()`](parse.md#notes); a document can borrow on one call and
own on the next, since ownership is decided freshly each time. An input that is copied (see the table in
[`parse()`](parse.md#notes)) is written into the document's previous owned copy, reusing its memory; a moved-in
`#!cpp std::string` replaces it.
own on the next, since ownership is decided freshly each time.
Reusing a document matters most for large inputs: the operating system provides the memory of a fresh node index one
page at a time, and every page costs a page fault the first time it is written. On x86-64 Linux (4 KiB pages), parsing
@@ -39,8 +39,7 @@ auto v = doc.root();
`root()` is a cheap handle into the document's index, not a copy of anything; call it as often as needed. The
returned view is valid under the same conditions as any other view of the document -- see
[Object inspection](../basic_json_view/index.md) -- in particular, it is invalidated by the next
[`read()`](read.md) or [`shrink_to_fit()`](shrink_to_fit.md) on this document. A view can dangle, like a
`#!cpp std::string_view`; see [Ownership and lifetime](../../features/json_view.md#ownership-and-lifetime).
[`read()`](read.md) or [`shrink_to_fit()`](shrink_to_fit.md) on this document.
## Examples
@@ -38,8 +38,7 @@ Constant.
## See also
- [is_structured](is_structured.md) - return whether the type is structured
- [size](size.md) - the number of elements
- [empty](empty.md) - whether there are no elements
- [size](size.md), [empty](empty.md) - the number of elements, and whether there are none
- [`BasicJsonType::is_array`](../basic_json/is_array.md) - the corresponding function of `basic_json`
## Version history
@@ -4,9 +4,8 @@
bool is_binary() const noexcept;
```
This function always returns `#!cpp false`: a document is only ever parsed from JSON text, never from CBOR or the other
binary formats, and a JSON text has no binary values, so a view can never refer to one. The function exists for
interface parity with [`BasicJsonType::is_binary`](../basic_json/is_binary.md).
This function always returns `#!cpp false`: a JSON text has no binary values, so a view can never refer to one. The
function exists for interface parity with [`BasicJsonType::is_binary`](../basic_json/is_binary.md).
## Return value
@@ -15,7 +15,7 @@ JSON pointer that cannot be resolved, and for any access on a view that is itsel
## Return value
`#!cpp true` if the view or its document is discarded, `#!cpp false` otherwise.
`#!cpp true` if the view is discarded, `#!cpp false` otherwise.
## Exception safety
@@ -38,8 +38,7 @@ Constant.
## See also
- [is_structured](is_structured.md) - return whether the type is structured
- [size](size.md) - the number of elements
- [empty](empty.md) - whether there are no elements
- [size](size.md), [empty](empty.md) - the number of elements, and whether there are none
- [`BasicJsonType::is_object`](../basic_json/is_object.md) - the corresponding function of `basic_json`
## Version history
@@ -37,8 +37,7 @@ Constant.
## See also
- [source_offset](source_offset.md) - byte offset of this value in the source text, which differs for strings: an
escaped string has none
- [source_offset](source_offset.md) - byte offset of this value in the document's source text
- [`BasicJsonType::is_string`](../basic_json/is_string.md) - the corresponding function of `basic_json`
## Version history
@@ -14,8 +14,8 @@ if the view is [discarded](is_discarded.md).
## Exception safety
Strong guarantee: there are no changes to the view or the document it refers to (nothing about either is mutated by
this function).
Strong guarantee: if an exception is thrown, there are no changes to the view or the document it refers to (nothing
about either is mutated by this function).
## Exceptions
@@ -28,13 +28,12 @@ Linear in the size of the subtree.
## Notes
`materialize()` replays the subtree through the same SAX builder [`BasicJsonType::parse()`](../basic_json/parse.md)
uses, so the result matches it exactly -- including, for an object, that a repeated key keeps only its
uses internally, so the result matches it exactly -- including, for an object, that a repeated key keeps only its
last value. The replay is iterative, so it is not limited by the call stack the way a naive recursive conversion
would be; the JSON nesting depth is limited only by available memory, as for `BasicJsonType::parse()` itself.
Unlike parsing with [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) enabled, the values
produced by `materialize()` do not carry source positions. The index has the start offset of every value, but not the
end of arrays, objects, and escaped strings, which the positions also need.
produced by `materialize()` do not carry source positions: there is no lexer run during the replay to record them.
Calling `materialize()` on the same view repeatedly builds a new, independent `BasicJsonType` value each time; it
never caches the result.
+9 -9
View File
@@ -10,15 +10,15 @@ Returns the number of elements, as [`BasicJsonType::size()`](../basic_json/size.
The return value depends on the type and is defined as follows:
| Value type | return value |
|------------|---------------------------|
| null | `0` |
| discarded | `0` |
| boolean | `1` |
| string | `1` |
| number | `1` |
| object | number of key/value pairs |
| array | number of elements |
| Value type | return value |
|----------------------|-----------------------------|
| null | `0` |
| discarded | `0` |
| boolean | `1` |
| string | `1` |
| number | `1` |
| object | number of key/value pairs |
| array | number of elements |
## Exception safety
+16 -29
View File
@@ -16,8 +16,8 @@ most of the resulting tree is thrown away almost immediately.
[`basic_json_document::parse()`](../api/basic_json_document/parse.md) parses the same JSON grammar, with the same
options, but instead of a tree it builds a flat index of the values it found: one
[16-byte entry](../home/architecture.md#node-index-of-json-views) per value (and one per object key), in document
order. Strings and numbers are referenced in place in the source text; only strings that contain escape sequences
are decoded, into one shared buffer owned by the document.
order. Strings and numbers are not copied out of the input; they stay in the source text, and are only decoded when
actually needed (for a string, only if it contains escape sequences, into one shared buffer owned by the document).
[`basic_json_view`](../api/basic_json_view/index.md) is a small, trivially copyable handle (two pointers) into that
index. It gives you the read-only, type-inspection part of the `basic_json` interface --
@@ -36,8 +36,7 @@ one [`parse()`](../api/basic_json/parse.md) would have produced for it.
Include `<nlohmann/json_view.hpp>` in addition to (or instead of) `<nlohmann/json.hpp>`. Parse into a
[`json_document`](../api/json_document.md), inspect its [`root()`](../api/basic_json_document/root.md), and
`materialize()` when you need a real value. `materialize()` works on any view, not just the root, so you can turn just
the subtree you need into a value:
`materialize()` when you need a real value:
??? example "Example: parse a document, inspect its root, and materialize it"
@@ -74,19 +73,15 @@ A view is valid only while all of the following hold:
- the document is alive,
- the document has not been re-parsed since the view was taken (with [`read()`](../api/basic_json_document/read.md)
or [`parse()`](../api/basic_json_document/parse.md) into it),
- the document has not had [`shrink_to_fit()`](../api/basic_json_document/shrink_to_fit.md) called on it since, and
or [`parse()`](../api/basic_json_document/parse.md) into it), and has not had
[`shrink_to_fit()`](../api/basic_json_document/shrink_to_fit.md) called on it since, and
- if the document borrows its source text, that text is still alive.
Moving the `basic_json_document<>` itself is fine and does **not** invalidate its views: the index is a separate heap
allocation that keeps its address across the move. Take a fresh view from [`root()`](../api/basic_json_document/root.md)
Moving the document itself is fine and does **not** invalidate its views: the index is a separate heap allocation
that keeps its address across the move. Take a fresh view from [`root()`](../api/basic_json_document/root.md)
whenever any of the other conditions above was not met.
A view can dangle, exactly like a `#!cpp std::string_view`: nothing stops you from using it after its document is
gone. This is deliberate. A view is two pointers and trivially copyable; making it safe (a `#!cpp std::shared_ptr` in
the document and a `#!cpp std::weak_ptr` in each view) would make it larger, add reference counting to every copy,
and a `lock()` to every access -- exactly the costs the view exists to avoid. What is caught is the obvious case:
because a view dies with its document, [`root()`](../api/basic_json_document/root.md) is not callable on a temporary
Because a view dies with its document, [`root()`](../api/basic_json_document/root.md) is not callable on a temporary
document: `#!cpp auto v = json_document::parse(text).root();` does not compile. Give the document a name first.
??? example "Example: borrowed and owned documents, and when views become invalid"
@@ -108,19 +103,14 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
[`basic_json::accept()`](../api/basic_json/accept.md)/[`basic_json::parse()`](../api/basic_json/parse.md), with the
same `ignore_comments` and `ignore_trailing_commas` options.
- **Errors.** A failing parse throws the same exception -- the same id, message, and position -- because on failure
the library's own parser is run on the same bytes to produce the diagnostic. Valid input never pays for this, and
the replay builds no values: it costs about a lexing pass up to the error.
`#!cpp allow_exceptions == false` skips the replay and gives a
[discarded](../api/basic_json_document/is_discarded.md) document instead of throwing, just as it gives a discarded
value for `#!cpp basic_json::parse()`.
the library's own parser is run on the same bytes to produce the diagnostic.
`#!cpp allow_exceptions == false` gives a [discarded](../api/basic_json_document/is_discarded.md) document instead
of throwing, just as it gives a discarded value for `#!cpp basic_json::parse()`.
- **Number classification.** An integer literal that does not fit into the 64-bit integer type becomes a
floating-point value, exactly as it does for `#!cpp basic_json::parse()`.
- **Macros.** [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) and
[`JSON_NOEXCEPTION`](../api/macros/json_noexception.md)/[`JSON_THROW_USER`](../api/macros/json_throw_user.md)
behave the same way they do for `<nlohmann/json.hpp>`. These are the macros that change what the parser accepts or
how it reports errors. The others either do not apply ([`JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md)
needs parent pointers in `basic_json` values) or only affect the values `materialize()` produces, through
`BasicJsonType`.
behave the same way they do for `<nlohmann/json.hpp>`.
## What is different
@@ -128,15 +118,12 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
`number_unsigned_t` to both be 64 bits wide; this is a compile-time `#!cpp static_assert`.
- **A 4 GiB input limit.** An input of 4294967280 bytes (4 GiB minus 16 bytes) or more throws
[`out_of_range.416`](../home/exceptions.md#jsonexceptionout_of_range416), a limit
`#!cpp basic_json::parse()` does not have. It keeps an index entry at 16 bytes: offsets and counts are 32 bits
wide. The limit is 16 bytes below 2^32, because the parser reads in 16-byte steps, and a position one step past the
end must still fit.
- **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`: the stream
is read to its end into a buffer the document owns, and parsing starts after that.
`#!cpp basic_json::parse()` does not have.
- **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`.
- **No source positions on `materialize()`.** Even with
[`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) enabled,
[`materialize()`](../api/basic_json_view/materialize.md) does not set them. The index has the start offset of every
value, but not the end of arrays, objects, and escaped strings, which the positions also need.
[`materialize()`](../api/basic_json_view/materialize.md) does not set them: there is no lexer run during the
replay to record them.
- **Objects iterate in document order.** [`begin()`](../api/basic_json_view/begin.md)/
[`end()`](../api/basic_json_view/end.md) and [`items()`](../api/basic_json_view/items.md) visit an object's
members in the order they appear in the source text. `basic_json`'s default `object_t` is a `std::map`, which
+140 -175
View File
@@ -74,10 +74,9 @@ class builder
{
public:
builder(document_data& d, const char* src, std::size_t size) noexcept
: m_doc(d)
, m_begin(reinterpret_cast<const unsigned char*>(src))
, m_end(m_begin + size)
, m_shallow()
: doc(d)
, b(reinterpret_cast<const unsigned char*>(src))
, e(b + size)
{}
/// returns false and fills `failure` on error
@@ -107,37 +106,28 @@ class builder
bool is_object;
};
/// 64 frames, left uninitialized: they are filled as containers open (the
/// user-provided constructor keeps the member initialization, which
/// -Weffc++ asks for, from zeroing them; that costs up to 30 % on small
/// documents)
struct frame_stack : std::array<frame, 64>
{
frame_stack() noexcept {} // NOLINT(modernize-use-equals-default,hicpp-use-equals-default,cppcoreguidelines-pro-type-member-init,hicpp-member-init): see above
};
document_data& m_doc;
const unsigned char* const m_begin;
const unsigned char* const m_end;
document_data& doc;
const unsigned char* const b;
const unsigned char* const e;
parse_failure m_failure{};
// the open array/object is in the cursor; enclosing ones on a stack that is
// inline for the first 64 levels
frame_stack m_shallow;
std::vector<frame> m_deep{};
frame shallow[64]; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays): not initialized on purpose; filled as containers open
std::vector<frame> deep{};
/// remember an object to index after parsing (out of line, so that the
/// parse loop only has a call for it)
NLOHMANN_VIEW_NOINLINE void note_large_object(std::uint32_t idx)
{
m_doc.large_objects.push_back(idx);
doc.large_objects.push_back(idx);
}
NLOHMANN_VIEW_NOINLINE bool fail(error_code c, const unsigned char* at) noexcept
{
m_failure.code = c;
m_failure.offset = static_cast<std::size_t>(at - m_begin);
m_doc.node_count = 0;
m_failure.offset = static_cast<std::size_t>(at - b);
doc.tape_size = 0;
return false;
}
@@ -166,7 +156,7 @@ class builder
/// the comment at p (*p == '/'): the position after it, or nullptr on error
NLOHMANN_VIEW_NOINLINE const unsigned char* comment(const unsigned char* p)
{
if (m_end - p < 2)
if (e - p < 2)
{
++p;
return fail_at(error_code::comment_start, p);
@@ -176,7 +166,7 @@ class builder
p += 2;
// (as in parse(), a null byte is the end of the input, so it is
// left for the caller to see)
while (p != m_end && *p != '\n' && *p != '\r' && !(NulIsEnd && *p == 0))
while (p != e && *p != '\n' && *p != '\r' && !(NulIsEnd && *p == 0))
{
++p;
}
@@ -187,11 +177,11 @@ class builder
p += 2;
for (;;)
{
if (p == m_end || (NulIsEnd && *p == 0))
if (p == e || (NulIsEnd && *p == 0))
{
return fail_at(error_code::comment_unterminated, p);
}
if (*p == '*' && p + 1 != m_end && p[1] == '/')
if (*p == '*' && p + 1 != e && p[1] == '/')
{
p += 2;
return p;
@@ -209,32 +199,31 @@ class builder
/// doubling repeatedly; returns the new node array
NLOHMANN_VIEW_NOINLINE node* grow(std::size_t n, const unsigned char* at)
{
// (n nodes are written; at the limit there is no room for another one)
const std::uint64_t limit = document_data::max_nodes();
// LCOV_EXCL_START (a node array that fills the address space)
if (NLOHMANN_VIEW_UNLIKELY(n >= limit))
{
document_data::throw_bad_alloc();
}
// LCOV_EXCL_STOP
const std::uint64_t done = static_cast<std::uint64_t>(at - m_begin) + 1;
const std::uint64_t guess = static_cast<std::uint64_t>(n) * static_cast<std::uint64_t>(m_end - m_begin + 1) / done;
const std::uint64_t done = static_cast<std::uint64_t>(at - b) + 1;
const std::uint64_t guess = static_cast<std::uint64_t>(n) * static_cast<std::uint64_t>(e - b + 1) / done;
const std::uint64_t grown = guess + (guess / 4) + 64; // a variable: GCC calls a cast of the sum useless where std::uint64_t is std::size_t
// (n is below 2^32: the input is smaller than 4 GiB; the sum cannot wrap)
const std::uint64_t wanted = (std::max)(grown, static_cast<std::uint64_t>(n) + (n / 2) + 64);
m_doc.node_count = n; // the number of nodes reserve() copies into the new array
const std::uint64_t limit = document_data::max_nodes();
doc.tape_size = n;
// LCOV_EXCL_START (a node array that fills the address space)
if (NLOHMANN_VIEW_UNLIKELY(n >= limit))
{
document_data::throw_bad_alloc(); // no room for another node
}
// LCOV_EXCL_STOP
// (a count beyond the limit is cut: the index does not grow beyond what can be addressed)
m_doc.reserve(static_cast<std::size_t>((std::min)(wanted, limit)));
return m_doc.nodes;
doc.reserve(static_cast<std::size_t>((std::min)(wanted, limit)));
return doc.tape;
}
/// the four hex digits of a unicode escape at p as a code unit, and p
/// moved past them; or -1, and p at the first bad digit (one table lookup
/// per digit and a single check: the library's table, after yyjson's
/// read_hex_u16)
/// four hex digits at p as a code unit (p moves past them), or -1 (p at
/// the first bad digit); one table lookup per digit and a single check,
/// the four hex digits of a unicode escape (the library's table, after
/// yyjson's read_hex_u16), or -1
NLOHMANN_VIEW_ALWAYS_INLINE int hex4(const unsigned char*& p) noexcept
{
if (NLOHMANN_VIEW_LIKELY(m_end - p >= 4))
if (NLOHMANN_VIEW_LIKELY(e - p >= 4))
{
const int cp = hex_codepoint(p);
if (NLOHMANN_VIEW_LIKELY(cp >= 0))
@@ -250,7 +239,7 @@ class builder
/// hex4() failed: the first bad digit (none: p)
NLOHMANN_VIEW_NOINLINE const unsigned char* hex4_error(const unsigned char* p) noexcept
{
if (m_end - p >= 4)
if (e - p >= 4)
{
while (is_hex(*p))
{
@@ -260,9 +249,8 @@ class builder
return p;
}
/// s points to the start of the string, p to a backslash, a control
/// character, an ill-formed UTF-8 byte, or the end: decode the string into
/// the arena
/// escapes present (or an error) in the string at s, scanned up to p:
/// decode into the arena
NLOHMANN_VIEW_NOINLINE decoded slow_string(const unsigned char* s, const unsigned char* p)
{
// single-character escapes; 0: invalid (and 'u', handled separately)
@@ -279,7 +267,7 @@ class builder
arena_run(s, static_cast<std::size_t>(p - s));
for (;;)
{
if (p == m_end)
if (p == e)
{
return failed(error_code::string_missing_quote, p);
}
@@ -291,20 +279,20 @@ class builder
}
if (c != '\\')
{
// (without JSON_STRICT_NUL_HANDLING, a NUL between values ends
// the input, as in json::parse; inside a string, a NUL is still
// a control character and an error)
// (a NUL before the end of the input is a control character, as
// for json::parse, also where a NUL ends the input between values)
return failed(c < 0x20 ? error_code::string_control_character : error_code::string_utf8, p);
}
++p;
if (p == m_end)
if (p == e)
{
return failed(error_code::string_missing_quote, p);
}
const unsigned char d = *p++;
arena_ensure(4);
if (d == 'u')
{
int cp = hex4(p); // (moves p past the four digits)
int cp = hex4(p);
if (NLOHMANN_VIEW_UNLIKELY(cp < 0))
{
return failed(error_code::string_unicode_hex, p);
@@ -315,12 +303,12 @@ class builder
{
return failed(error_code::string_surrogate_low, p);
}
if (m_end - p < 2 || p[0] != '\\' || p[1] != 'u')
if (e - p < 2 || p[0] != '\\' || p[1] != 'u')
{
return failed(error_code::string_surrogate_high, p);
}
p += 2;
const int lo = hex4(p); // (moves p past the four digits)
const int lo = hex4(p);
if (lo < 0)
{
return failed(error_code::string_unicode_hex, p);
@@ -331,25 +319,23 @@ class builder
}
cp = 0x10000 + ((cp - 0xD800) << 10) + (lo - 0xDC00);
}
arena_ensure(4);
m_arena_write = put_utf8(m_arena_write, cp);
aw = put_utf8(aw, cp);
}
else if (NLOHMANN_VIEW_LIKELY(d < 128 && simple_escape[d] != 0))
{
arena_ensure(1);
*m_arena_write++ = simple_escape[d];
*aw++ = simple_escape[d];
}
else
{
--p;
return failed(error_code::string_escape, p);
}
if (p != m_end && *p == '\\')
if (p != e && *p == '\\')
{
continue; // consecutive escapes ("\u00e4\u00f6"): no run in between
}
const unsigned char* const r = p;
p = scan_string_run(p, m_end);
p = scan_string_run(p, e);
arena_run(r, static_cast<std::size_t>(p - r));
}
}
@@ -369,17 +355,10 @@ class builder
return v > (std::numeric_limits<FloatType>::max)() || v < -(std::numeric_limits<FloatType>::max)();
}
/// do the `digits` digits of a magnitude at d exceed `limit`, the largest
/// magnitude that fits (a string literal: N counts its terminating NUL, so
/// it has N - 1 digits)?
template<std::size_t N>
static NLOHMANN_VIEW_ALWAYS_INLINE bool digits_exceed(const unsigned char* d, std::size_t digits, const char (&limit)[N]) noexcept // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays): a string literal
/// does the magnitude digits [d, d + n) exceed the given limit (same length)?
static bool digits_exceed(const unsigned char* d, const char* limit, std::size_t n) noexcept
{
if (NLOHMANN_VIEW_LIKELY(digits < N - 1))
{
return false;
}
return digits > N - 1 || std::memcmp(d, limit, N - 1) > 0;
return std::memcmp(d, limit, n) > 0;
}
static bool is_hex(unsigned char c) noexcept
@@ -387,19 +366,19 @@ class builder
return (c >= '0' && c <= '9') || (c >= 'A' && c <= 'F') || (c >= 'a' && c <= 'f');
}
// decode arena: the vector in the document, written through a raw
// decode arena: the std::string in the document, written through a raw
// pointer (resized ahead in large steps; trimmed when parsing succeeds)
char* m_arena_write = nullptr;
char* m_arena_end = nullptr;
char* aw = nullptr;
char* aend = nullptr;
NLOHMANN_VIEW_ALWAYS_INLINE std::size_t arena_used() const noexcept
{
return m_arena_write != nullptr ? static_cast<std::size_t>(m_arena_write - m_doc.arena.data()) : 0;
return aw != nullptr ? static_cast<std::size_t>(aw - doc.arena.data()) : 0;
}
NLOHMANN_VIEW_ALWAYS_INLINE void arena_ensure(std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(m_arena_end - m_arena_write) < n))
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(aend - aw) < n))
{
arena_grow(n);
}
@@ -408,31 +387,28 @@ class builder
NLOHMANN_VIEW_NOINLINE void arena_grow(std::size_t n)
{
const std::size_t used = arena_used();
m_doc.arena.resize((std::max)(m_doc.arena.size() * 2, used + n + 256));
m_arena_write = m_doc.arena.data() + used;
m_arena_end = m_doc.arena.data() + m_doc.arena.size();
doc.arena.resize((std::max)(doc.arena.size() * 2, used + n + 256));
aw = &doc.arena[0] + used; // NOLINT(readability-container-data-pointer): data() is const before C++17
aend = &doc.arena[0] + doc.arena.size(); // NOLINT(readability-container-data-pointer)
}
/// append the run [r, r + n) to the arena; short runs as one fixed-size
/// 16-byte move when both sides have the room (no library call)
NLOHMANN_VIEW_ALWAYS_INLINE void arena_run(const unsigned char* r, std::size_t n)
{
if (n <= 16 && m_end - r >= 16)
arena_ensure(n + 16);
if (n <= 16 && e - r >= 16)
{
arena_ensure(16);
std::memcpy(m_arena_write, r, 16);
std::memcpy(aw, r, 16);
}
else
{
arena_ensure(n);
std::memcpy(m_arena_write, r, n);
std::memcpy(aw, r, n);
}
m_arena_write += n;
aw += n;
}
/// UTF-8 encoding of cp at w (room for 4 bytes); returns the position after
/// it. (A returned pointer stays in a register; writing through a
/// reference to m_arena_write would store and reload it around every byte.)
/// UTF-8 encoding of cp at w (room for 4 bytes)
static char* put_utf8(char* w, int cp) noexcept
{
if (cp < 0x80)
@@ -477,9 +453,9 @@ class builder
{
explicit cursor(builder& owner) noexcept
: cold(owner)
, b(owner.m_begin)
, p(owner.m_begin)
, e(owner.m_end)
, b(owner.b)
, p(owner.b)
, e(owner.e)
{}
builder& cold; ///< out-of-line helpers and state that needs no registers
@@ -495,14 +471,13 @@ class builder
std::uint32_t cur_count = 0;
bool cur_is_object = false;
std::size_t depth = 0;
std::size_t max_depth = 0;
NLOHMANN_VIEW_ALWAYS_INLINE bool run()
{
cold.m_doc.reserve(estimate_nodes(reinterpret_cast<const char*>(b), static_cast<std::size_t>(e - b)));
base = cold.m_doc.nodes;
cold.doc.reserve(estimate_nodes(reinterpret_cast<const char*>(b), static_cast<std::size_t>(e - b)));
base = cold.doc.tape;
out = base;
cap = base + cold.m_doc.node_capacity;
cap = base + cold.doc.tape_cap;
if (e - p >= 3 && p[0] == 0xEF && p[1] == 0xBB && p[2] == 0xBF)
{
@@ -538,40 +513,39 @@ class builder
// value dispatch, expanded once for array elements and once for member
// values: each jump has its own history (arrays tend to hold one kind of
// value), and the continuation needs no branch on the container kind;
// after a scalar, the switch is left to the label that follows it
#define NLOHMANN_VIEW_VALUE \
switch (cur()) \
{ \
case '"': \
if (NLOHMANN_VIEW_UNLIKELY(!string<true>())) { return false; } \
break; \
case '{': \
open(value_t::object); \
++p; \
goto obj_first; \
case '[': \
open(value_t::array); \
++p; \
goto arr_first; \
case '-': \
if (NLOHMANN_VIEW_UNLIKELY(!number<true>())) { return false; } \
break; \
case '0': case '1': case '2': case '3': case '4': \
case '5': case '6': case '7': case '8': case '9': \
if (NLOHMANN_VIEW_UNLIKELY(!number<false>())) { return false; } \
break; \
case 't': \
if (NLOHMANN_VIEW_UNLIKELY(!literal_true())) { return false; } \
break; \
case 'f': \
if (NLOHMANN_VIEW_UNLIKELY(!literal_false())) { return false; } \
break; \
case 'n': \
if (NLOHMANN_VIEW_UNLIKELY(!literal_null())) { return false; } \
break; \
default: \
return fail(error_code::unexpected_value); \
// value), and the continuation needs no branch on the container kind
#define NLOHMANN_VIEW_VALUE(NEXT) \
switch (cur()) \
{ \
case '"': \
if (NLOHMANN_VIEW_UNLIKELY(!string<true>())) { return false; } \
goto NEXT; \
case '{': \
open(value_t::object); \
++p; \
goto obj_first; \
case '[': \
open(value_t::array); \
++p; \
goto arr_first; \
case '-': \
if (NLOHMANN_VIEW_UNLIKELY(!number<true>())) { return false; } \
goto NEXT; \
case '0': case '1': case '2': case '3': \
case '4': case '5': case '6': case '7': case '8': case '9': \
if (NLOHMANN_VIEW_UNLIKELY(!number<false>())) { return false; } \
goto NEXT; \
case 't': \
if (NLOHMANN_VIEW_UNLIKELY(!literal("true", 4, value_t::boolean, node_flags::is_true))) { return false; } \
goto NEXT; \
case 'f': \
if (NLOHMANN_VIEW_UNLIKELY(!literal_false())) { return false; } \
goto NEXT; \
case 'n': \
if (NLOHMANN_VIEW_UNLIKELY(!literal("null", 4, value_t::null, 0))) { return false; } \
goto NEXT; \
default: \
return fail(error_code::unexpected_value); \
}
arr_first:
@@ -585,7 +559,7 @@ arr_first:
goto close_container;
}
value:
NLOHMANN_VIEW_VALUE
NLOHMANN_VIEW_VALUE(arr_next)
arr_next:
++cur_count;
if (!ws())
@@ -652,8 +626,7 @@ obj_key:
{
return false;
}
NLOHMANN_VIEW_VALUE
#undef NLOHMANN_VIEW_VALUE
NLOHMANN_VIEW_VALUE(obj_next)
obj_next:
++cur_count;
if (!ws())
@@ -681,6 +654,8 @@ obj_next:
}
return fail(error_code::expected_object_end);
#undef NLOHMANN_VIEW_VALUE
close_object:
// a large object gets a hash index (objects only, so that closing
// an array pays nothing for this)
@@ -710,11 +685,8 @@ root_done:
{
return fail(error_code::trailing_characters);
}
cold.m_doc.node_count = static_cast<std::size_t>(out - base);
cold.m_doc.max_depth = max_depth;
// (so that size() is the number of bytes used: the arena grows ahead
// in large steps, and shrink_to_fit() keeps size() bytes)
cold.m_doc.arena.resize(cold.arena_used());
cold.doc.tape_size = static_cast<std::size_t>(out - base);
cold.doc.arena.resize(cold.arena_used());
return true;
}
@@ -743,11 +715,11 @@ root_done:
case '"':
return string<true>();
case 't':
return literal_true();
return literal("true", 4, value_t::boolean, node_flags::is_true);
case 'f':
return literal_false();
case 'n':
return literal_null();
return literal("null", 4, value_t::null, 0);
case '-':
return number<true>();
case '0':
@@ -766,19 +738,9 @@ root_done:
}
}
NLOHMANN_VIEW_ALWAYS_INLINE bool literal_true()
{
return literal("true", value_t::boolean, node_flags::is_true);
}
NLOHMANN_VIEW_ALWAYS_INLINE bool literal_false()
{
return literal("false", value_t::boolean, 0);
}
NLOHMANN_VIEW_ALWAYS_INLINE bool literal_null()
{
return literal("null", value_t::null, 0);
return literal("false", 5, value_t::boolean, 0);
}
/// skip whitespace (and comments); false on a malformed comment
@@ -851,7 +813,7 @@ indent_done:
const auto n = static_cast<std::size_t>(out - base);
base = cold.grow(n, p);
out = base + n;
cap = base + cold.m_doc.node_capacity;
cap = base + cold.doc.tape_cap;
}
node* n = out++;
#if NLOHMANN_VIEW_LITTLE_ENDIAN
@@ -885,18 +847,17 @@ indent_done:
// field by field: a frame put together on the stack and
// copied would be read back wider than it was written,
// and that load waits until the stores are done
frame& f = cold.m_shallow[depth - 1];
frame& f = cold.shallow[depth - 1];
f.idx = cur_idx;
f.count = cur_count;
f.is_object = cur_is_object;
}
else
{
cold.m_deep.push_back(frame{cur_idx, cur_count, cur_is_object});
cold.deep.push_back(frame{cur_idx, cur_count, cur_is_object});
}
}
++depth;
max_depth = (std::max)(max_depth, depth);
cur_idx = idx;
cur_count = 0;
cur_is_object = k == value_t::object;
@@ -911,15 +872,15 @@ indent_done:
{
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
{
const frame& f = cold.m_shallow[depth - 1];
const frame& f = cold.shallow[depth - 1];
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
}
else
{
const frame f = cold.m_deep.back();
cold.m_deep.pop_back();
const frame f = cold.deep.back();
cold.deep.pop_back();
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
@@ -927,11 +888,8 @@ indent_done:
}
}
/// the literal `text` at p (N counts the terminating NUL)
template<std::size_t N>
NLOHMANN_VIEW_ALWAYS_INLINE bool literal(const char (&text)[N], value_t k, std::uint8_t flags) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays): a string literal
NLOHMANN_VIEW_ALWAYS_INLINE bool literal(const char* text, std::size_t n, value_t k, std::uint8_t flags)
{
const std::size_t n = N - 1;
if (NLOHMANN_VIEW_UNLIKELY(e - p < static_cast<std::ptrdiff_t>(n) || std::memcmp(p, text, n) != 0))
{
return fail(error_code::invalid_literal);
@@ -1010,15 +968,25 @@ indent_done:
value_t kind = value_t::number_float;
if (!is_float)
{
kind = negative ? value_t::number_integer : value_t::number_unsigned;
}
if (!is_float)
{
// integers that do not fit become floats, as in parse()
if (negative)
if (NLOHMANN_VIEW_UNLIKELY(int_digits >= 19))
{
kind = digits_exceed(int_start, int_digits, "9223372036854775808") ? value_t::number_float : value_t::number_integer;
}
else
{
kind = digits_exceed(int_start, int_digits, "18446744073709551615") ? value_t::number_float : value_t::number_unsigned;
if (negative)
{
if (int_digits > 19 || (int_digits == 19 && digits_exceed(int_start, "9223372036854775808", 19)))
{
kind = value_t::number_float;
}
}
else if (int_digits > 20 || (int_digits == 20 && digits_exceed(int_start, "18446744073709551615", 20)))
{
kind = value_t::number_float;
}
}
}
// parse() rejects floats that overflow; only numbers whose magnitude
@@ -1039,9 +1007,6 @@ indent_done:
// integers are converted now, while their digits are in cache
const std::uint64_t m = int_digits <= 19 ? parse_upto19(int_start, static_cast<unsigned>(int_digits), e)
: (parse_upto19(int_start, 19, e) * 10) + static_cast<std::uint64_t>(int_start[19] - '0');
// (0 - m rather than -m: unary minus on an unsigned type is
// MSVC's C4146, an error under /sdl; both give the bit pattern
// of the negative std::int64_t)
second = negative ? 0 - m : m;
}
emit(kind, 0, layout, static_cast<std::size_t>(s - b), second);
@@ -1074,7 +1039,7 @@ indent_done:
};
/// run the builder with compile-time options
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd>
template<typename FloatType, bool NulIsEnd, bool Comments, bool TrailingCommas>
inline bool build_with(document_data& d, const char* src, std::size_t size, bool sentinel, parse_failure& failure)
{
if (sentinel)
@@ -1097,11 +1062,11 @@ inline bool build(document_data& d, const char* src, std::size_t size, bool comm
{
if (comments)
{
return trailing_commas ? build_with<FloatType, true, true, NulIsEnd>(d, src, size, sentinel, failure)
: build_with<FloatType, true, false, NulIsEnd>(d, src, size, sentinel, failure);
return trailing_commas ? build_with<FloatType, NulIsEnd, true, true>(d, src, size, sentinel, failure)
: build_with<FloatType, NulIsEnd, true, false>(d, src, size, sentinel, failure);
}
return trailing_commas ? build_with<FloatType, false, true, NulIsEnd>(d, src, size, sentinel, failure)
: build_with<FloatType, false, false, NulIsEnd>(d, src, size, sentinel, failure);
return trailing_commas ? build_with<FloatType, NulIsEnd, false, true>(d, src, size, sentinel, failure)
: build_with<FloatType, NulIsEnd, false, false>(d, src, size, sentinel, failure);
}
} // namespace view
+28 -32
View File
@@ -35,15 +35,14 @@ struct document_data
{
const char* src = nullptr;
std::size_t size = 0;
node* nodes = nullptr; ///< the node index (see node.hpp)
std::size_t node_count = 0;
std::size_t node_capacity = 0;
node* inline_nodes = nullptr; ///< node array allocated together with this header
std::size_t inline_capacity = 0;
std::size_t max_depth = 0; ///< deepest nesting of arrays and objects (the root container is depth 1)
std::vector<char> arena; ///< decoded strings that contained escapes (no terminator: they are handed out with their length)
node* tape = nullptr;
std::size_t tape_size = 0;
std::size_t tape_cap = 0;
node* inline_tape = nullptr; ///< node array allocated together with this header
std::size_t inline_cap = 0;
std::string arena; ///< decoded strings that contained escapes
std::size_t arena_size = 0; ///< bytes of decoded strings at base[1] (the arena, or those of a loaded image)
std::string owned; ///< owned copy of the input, if any (a std::string, so that its NUL serves as the parser's sentinel)
std::string owned; ///< owned copy of the input, if any
std::vector<std::uint8_t> owned_image; ///< a loaded image the document owns (the text and the decoded strings point into it)
// hash indexes of large objects (see object_index.hpp)
@@ -80,21 +79,18 @@ struct document_data
};
std::unique_ptr<edit_state> edits; ///< created by the first edit
/// One allocation for the header and room for `count` nodes, so that a
/// small document needs one allocation instead of two. Large documents get
/// a separate node array instead (so it can be trimmed). When a document
/// outgrows the inline array, that array stays unused: at most 256 nodes
/// (4 KiB), and only for documents whose estimate was too small.
static document_data* create(std::size_t count)
/// one allocation for the header and room for `nodes` nodes; large
/// documents get a separate node array instead (so it can be trimmed)
static document_data* create(std::size_t nodes)
{
count = count <= 256 ? count : 0;
void* mem = ::operator new (sizeof(document_data) + (count * sizeof(node)));
nodes = nodes <= 256 ? nodes : 0;
void* mem = ::operator new (sizeof(document_data) + (nodes * sizeof(node)));
auto* d = new (mem) document_data(); // NOLINT(cppcoreguidelines-owning-memory): owned by the returned pointer, freed by deleter
// (aligned: sizeof is a multiple of the alignment; through void*, as GCC's -Wcast-align wants)
d->inline_nodes = static_cast<node*>(static_cast<void*>(static_cast<char*>(mem) + sizeof(document_data))); // NOLINT(bugprone-casting-through-void)
d->inline_capacity = count;
d->nodes = d->inline_nodes;
d->node_capacity = count;
d->inline_tape = static_cast<node*>(static_cast<void*>(static_cast<char*>(mem) + sizeof(document_data))); // NOLINT(bugprone-casting-through-void)
d->inline_cap = nodes;
d->tape = d->inline_tape;
d->tape_cap = nodes;
return d;
}
@@ -113,8 +109,8 @@ struct document_data
/// implicit one); they cannot take default member initializers, which old
/// Clang (3.4-3.6) rejects. The other members have default member
/// initializers. noexcept: create() constructs into raw memory and could not
/// release it if this threw (the std::vector and std::string default
/// constructors do not allocate).
/// release it if this threw (the std::string default constructors do not
/// allocate).
document_data() noexcept
: arena() // NOLINT(readability-redundant-member-init)
, owned() // NOLINT(readability-redundant-member-init)
@@ -135,12 +131,12 @@ struct document_data
void release() noexcept
{
if (nodes != inline_nodes)
if (tape != inline_tape)
{
::operator delete (nodes);
::operator delete (tape);
}
nodes = inline_nodes;
node_capacity = inline_capacity;
tape = inline_tape;
tape_cap = inline_cap;
}
/// the largest node count whose size in bytes fits a std::size_t
@@ -154,12 +150,12 @@ struct document_data
NLOHMANN_VIEW_THROW(std::bad_alloc());
}
/// make room for n nodes; keeps the first node_count nodes (throws
/// make room for n nodes; keeps the first tape_size nodes (throws
/// std::bad_alloc for a count that does not fit the address space,
/// instead of wrapping around in n * sizeof(node))
void reserve(std::size_t n)
{
if (n <= node_capacity)
if (n <= tape_cap)
{
return;
}
@@ -168,13 +164,13 @@ struct document_data
throw_bad_alloc();
}
node* fresh = static_cast<node*>(::operator new (n * sizeof(node)));
if (node_count != 0)
if (tape_size != 0)
{
std::memcpy(fresh, nodes, node_count * sizeof(node));
std::memcpy(fresh, tape, tape_size * sizeof(node));
}
release();
nodes = fresh;
node_capacity = n;
tape = fresh;
tape_cap = n;
}
const char* str(const node& n) const noexcept
+1 -1
View File
@@ -363,7 +363,7 @@ class editor
const std::uint8_t linked = slot->flags & node_flags::linked;
if (e.region == nullptr)
{
if (is_container(*slot) && slot->next > 1 && slot != m_doc.nodes && linked == 0)
if (is_container(*slot) && slot->next > 1 && slot != m_doc.tape && linked == 0)
{
// The slot spans its old elements in the enclosing sequence, but
// a scalar is one node: the enclosing container first switches to
@@ -202,9 +202,9 @@ inline node* block_of(document_data& d, node* n, std::size_t extra)
inline node* find_parent(const document_data& d, const node* target)
{
const std::less<const node*> lt{};
const node* lo = d.nodes;
const node* hi = d.nodes + d.node_count;
const node* c = d.nodes;
const node* lo = d.tape;
const node* hi = d.tape + d.tape_size;
const node* c = d.tape;
if (lt(target, lo) || !lt(target, hi))
{
if (!d.edits)
+4 -87
View File
@@ -51,96 +51,14 @@ namespace view
NLOHMANN_VIEW_THROW(parse_error::create(id, 0, msg, nullptr));
}
/// SAX handler of the error replay: accepts every event without storing it,
/// and throws the exception the parser reports
template<typename BasicJsonType>
struct throwing_sax
{
using number_integer_t = typename BasicJsonType::number_integer_t;
using number_unsigned_t = typename BasicJsonType::number_unsigned_t;
using number_float_t = typename BasicJsonType::number_float_t;
using string_t = typename BasicJsonType::string_t;
using binary_t = typename BasicJsonType::binary_t;
bool null()
{
return true;
}
bool boolean(bool /*unused*/)
{
return true;
}
bool number_integer(number_integer_t /*unused*/)
{
return true;
}
bool number_unsigned(number_unsigned_t /*unused*/)
{
return true;
}
bool number_float(number_float_t /*unused*/, const string_t& /*unused*/)
{
return true;
}
bool string(string_t& /*unused*/)
{
return true;
}
bool binary(binary_t& /*unused*/)
{
return true;
}
bool start_object(std::size_t /*unused*/)
{
return true;
}
bool key(string_t& /*unused*/)
{
return true;
}
bool end_object()
{
return true;
}
bool start_array(std::size_t /*unused*/)
{
return true;
}
bool end_array()
{
return true;
}
/// (a template, as in json_sax_dom_parser: the parser passes the concrete
/// exception type, which is thrown as is)
template<class Exception>
[[noreturn]] bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const Exception& ex)
{
NLOHMANN_VIEW_THROW(ex);
}
};
/*!
@brief throw the exception BasicJsonType::parse would throw for this input
The view accepts exactly the inputs parse() accepts, so on a failure the
library parser is run on the same bytes: it throws the exception parse() would
throw, with the same message, position, and "last read" token. The replay
uses a SAX handler that stores nothing, so it only lexes and parses up to the
error, without building values. The error path is cold, so this costs nothing
on valid input. Should parse() accept the input nevertheless (a bug), the
view's own failure is reported.
throw, with the same message, position, and "last read" token. The error path
is cold, so this costs nothing on valid input. Should parse() accept the input
nevertheless (a bug), the view's own failure is reported.
*/
template<typename BasicJsonType>
[[noreturn]] NLOHMANN_VIEW_NOINLINE void throw_parse_failure(const parse_failure& f, const char* src, std::size_t size,
@@ -151,8 +69,7 @@ template<typename BasicJsonType>
// (the limit is detail::view::max_input_size(): 4 GiB minus 16 bytes)
NLOHMANN_VIEW_THROW(out_of_range::create(416, "input of 4294967280 bytes or more is not supported by json_document", nullptr));
}
throwing_sax<BasicJsonType> sax;
const bool accepted = BasicJsonType::sax_parse(src, src + size, &sax, input_format_t::json, true, ignore_comments, ignore_trailing_commas);
const BasicJsonType accepted = BasicJsonType::parse(src, src + size, nullptr, true, ignore_comments, ignore_trailing_commas);
// LCOV_EXCL_START (only if parse() accepts what the view rejects: a bug)
static_cast<void>(accepted);
+8 -8
View File
@@ -159,7 +159,7 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
}
out.push_back(r);
};
emit(d.nodes);
emit(d.tape);
while (!stack.empty())
{
frame& top = stack.back();
@@ -195,8 +195,8 @@ inline std::vector<std::uint8_t> save_image(const document_data& d)
throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
#endif
const std::size_t arena_size = d.arena_size;
const node* nodes = d.nodes;
std::size_t count = d.node_count;
const node* nodes = d.tape;
std::size_t count = d.tape_size;
std::vector<node> compacted;
std::string text_tail;
std::string arena_tail;
@@ -717,10 +717,10 @@ inline void load_image(document_data& d, const std::uint8_t* image, std::size_t
d.indexes.clear();
d.index_slots.clear();
d.large_objects.clear();
d.node_count = 0;
d.tape_size = 0;
d.reserve(count);
std::memcpy(d.nodes, image + sizeof(h), count * sizeof(node));
d.node_count = count;
std::memcpy(d.tape, image + sizeof(h), count * sizeof(node));
d.tape_size = count;
const std::uint8_t* const text = image + text_at;
const std::uint8_t* const arena = text + text_size + 1;
d.src = reinterpret_cast<const char*>(text); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
@@ -728,14 +728,14 @@ inline void load_image(document_data& d, const std::uint8_t* image, std::size_t
d.base[0] = d.src;
d.base[1] = reinterpret_cast<const char*>(arena); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
d.arena_size = arena_size;
if (check != image_check::none && !check_image(d.nodes, count, text, text_size, arena, arena_size, check == image_check::full))
if (check != image_check::none && !check_image(d.tape, count, text, text_size, arena, arena_size, check == image_check::full))
{
throw_invalid_image("the check failed");
}
// the hash indexes of large objects, as after parsing
for (std::size_t i = 0; i < count; ++i)
{
node& n = d.nodes[i];
node& n = d.tape[i];
if (n.kind == static_cast<std::uint8_t>(value_t::object))
{
n.extra = 0;
+13 -78
View File
@@ -8,11 +8,8 @@
#pragma once
#include <algorithm> // max
#include <cstddef> // size_t
#include <istream> // istream, streambuf
#include <string> // basic_string, char_traits, string
#include <type_traits> // decay, integral_constant, is_array, is_const, is_integral, is_lvalue_reference, is_pointer, is_same, remove_cv, remove_extent, remove_pointer, remove_reference
#include <type_traits> // decay, integral_constant, is_array, is_const, is_integral, is_lvalue_reference, is_pointer, is_same, remove_reference
#include <utility> // forward
#include <nlohmann/json.hpp>
@@ -31,31 +28,19 @@ namespace view
/// how a document takes its input
enum class input_kind
{
move_string, ///< non-const rvalue std::string: owned without a copy
c_string, ///< pointer to single-byte characters (NUL-terminated): borrowed
char_array, ///< array of single-byte characters (e.g. a string literal): borrowed
borrow_range, ///< lvalue contiguous byte container, or std::string_view: borrowed
copy_range, ///< rvalue contiguous byte container (a const rvalue std::string too): copied
wide_c_string, ///< pointer to wide characters (NUL-terminated): converted to UTF-8 into a buffer
adapter, ///< streams, FILE*, wide strings and arrays, and the rest of what the library's input adapter reads: read into a buffer
move_string, ///< non-const rvalue std::string: owned without a copy
c_string, ///< const char* (NUL-terminated): borrowed
char_array, ///< char array (e.g. a string literal): borrowed
borrow_range, ///< lvalue contiguous byte container, or std::string_view: borrowed
copy_range, ///< rvalue contiguous byte container (a const rvalue std::string too): copied
adapter, ///< streams, wide strings, and the rest of what the library's input adapter reads: read into a buffer
};
/// a single-byte character type (char, signed/unsigned char, char8_t, std::uint8_t, ...)
template<typename T>
struct is_byte_char : std::integral_constant < bool, std::is_integral<T>::value && sizeof(T) == 1 > {}; // NOLINT(bugprone-sizeof-expression): T is any pointee or element type
/// a character type the library's input adapter converts to UTF-8
template<typename T>
struct is_wide_char : std::integral_constant < bool, std::is_same<T, wchar_t>::value || std::is_same<T, char16_t>::value
|| std::is_same<T, char32_t>::value > {};
template<typename InputType>
struct classify_input
{
using R = typename std::remove_reference<InputType>::type;
using D = typename std::decay<InputType>::type;
using element = typename std::remove_cv<typename std::remove_extent<R>::type>::type;
using pointee = typename std::remove_cv<typename std::remove_pointer<D>::type>::type;
static constexpr bool is_rvalue = !std::is_lvalue_reference<InputType>::value;
static constexpr bool is_bytes = is_contiguous_byte_container<D>::value;
#if NLOHMANN_VIEW_HAS_CPP_17
@@ -64,14 +49,9 @@ struct classify_input
static constexpr bool is_string_view = false;
#endif
// NOLINTBEGIN(readability-avoid-nested-conditional-operator): a constant expression of C++11
// (arrays and pointers of other types, such as wide-character arrays and
// FILE*, go to the library's input adapter)
static constexpr input_kind value =
(std::is_array<R>::value && is_byte_char<element>::value) ? input_kind::char_array
: std::is_array<R>::value ? input_kind::adapter
: (std::is_pointer<D>::value && is_byte_char<pointee>::value) ? input_kind::c_string
: (std::is_pointer<D>::value && is_wide_char<pointee>::value) ? input_kind::wide_c_string
: std::is_pointer<D>::value ? input_kind::adapter
std::is_array<R>::value ? input_kind::char_array
: std::is_pointer<D>::value ? input_kind::c_string
: (is_rvalue && !std::is_const<R>::value && std::is_same<D, std::string>::value) ? input_kind::move_string
: (is_bytes && (!is_rvalue || is_string_view)) ? input_kind::borrow_range
: is_bytes ? input_kind::copy_range
@@ -93,10 +73,11 @@ struct is_std_string : std::false_type {};
template<typename Traits, typename Alloc>
struct is_std_string<std::basic_string<char, Traits, Alloc>> : std::true_type {};
/// append what a json input adapter yields (UTF-16/32 inputs arrive as UTF-8)
/// drain a json input adapter (UTF-16/32 inputs arrive as UTF-8)
template<typename Adapter>
void append_adapter(std::string& buf, Adapter ia)
std::string collect_adapter(Adapter ia)
{
std::string buf;
for (;;)
{
const auto ch = ia.get_character();
@@ -106,53 +87,7 @@ void append_adapter(std::string& buf, Adapter ia)
}
buf.push_back(static_cast<char>(ch));
}
}
/// an input adapter made by the document for an input that the library's
/// input_adapter() does not take as one argument (an iterator range, a wide C
/// string), to be appended with append_adapter()
template<typename Adapter>
struct adapter_input
{
Adapter ia;
};
template<typename T>
struct is_adapter_input : std::false_type {};
template<typename Adapter>
struct is_adapter_input<adapter_input<Adapter>> : std::true_type {};
template<typename Adapter>
adapter_input<typename std::decay<Adapter>::type> make_adapter_input(Adapter&& ia)
{
return {std::forward<Adapter>(ia)};
}
/// append the rest of a stream: its characters need no conversion, so they are
/// read in blocks from the stream buffer, straight into buf (reusing its
/// capacity). As the library's stream adapter, this reads the stream buffer
/// directly and sets eofbit at the end.
inline void append_stream(std::string& buf, std::istream& is)
{
std::streambuf* const sb = is.rdbuf();
if (sb == nullptr)
{
return;
}
for (;;)
{
const std::size_t used = buf.size();
const std::size_t room = (std::max)(buf.capacity() - used, std::size_t{4096});
buf.resize(used + room);
const std::streamsize got = sb->sgetn(&buf[used], static_cast<std::streamsize>(room));
buf.resize(used + static_cast<std::size_t>(got > 0 ? got : 0));
if (got <= 0)
{
break;
}
}
is.clear(is.rdstate() | std::ios::eofbit);
return buf;
}
} // namespace view
+6 -16
View File
@@ -8,8 +8,6 @@
#pragma once
#include <algorithm> // max
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t
#include <string> // string
#include <vector> // vector
@@ -43,12 +41,9 @@ BasicJsonType materialize(const document_data& d, const node* n)
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>>;
using nav = navigation<Editable>;
// the open containers: their next element (or key of the next member),
// their end, and whether they are objects; sized once from the
// document's deepest nesting (for a subtree an upper bound)
struct frame
{
const node* pos;
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
};
@@ -56,8 +51,7 @@ BasicJsonType materialize(const document_data& d, const node* n)
BasicJsonType result;
sax_t sax(result, true);
const string_t no_token{};
std::vector<frame> open((std::max)(d.max_depth, std::size_t{1}));
std::size_t depth = 0;
std::vector<frame> open;
for (;;)
{
// false positive: n comes from nav::value(), which never returns null for a valid index
@@ -76,11 +70,7 @@ BasicJsonType materialize(const document_data& d, const node* n)
{
sax.start_array(n->len);
}
if (NLOHMANN_VIEW_UNLIKELY(depth == open.size()))
{
open.resize(depth * 2); // (deeper than the recorded nesting, e.g. after edits)
}
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object};
open.push_back(frame{nav::first(d, n), nav::end(d, n), object});
break;
}
case value_t::string:
@@ -112,11 +102,11 @@ BasicJsonType materialize(const document_data& d, const node* n)
// the next value: close finished containers, then read the key
for (;;)
{
if (depth == 0)
if (open.empty())
{
return result;
}
frame& f = open[depth - 1];
frame& f = open.back();
if (f.pos == f.end)
{
if (f.object)
@@ -127,7 +117,7 @@ BasicJsonType materialize(const document_data& d, const node* n)
{
sax.end_array();
}
--depth;
open.pop_back();
continue;
}
const node* entry = f.pos;
+8 -8
View File
@@ -21,6 +21,14 @@ namespace detail
namespace view
{
// the node kinds are value_t values; the tests of is_container() and of the
// number kinds depend on this numbering
static_assert(static_cast<std::uint8_t>(value_t::null) == 0 && static_cast<std::uint8_t>(value_t::object) == 1
&& static_cast<std::uint8_t>(value_t::array) == 2 && static_cast<std::uint8_t>(value_t::string) == 3
&& static_cast<std::uint8_t>(value_t::boolean) == 4 && static_cast<std::uint8_t>(value_t::number_integer) == 5
&& static_cast<std::uint8_t>(value_t::number_unsigned) == 6 && static_cast<std::uint8_t>(value_t::number_float) == 7,
"the node format depends on the numbering of value_t");
/// The largest input a document accepts, in bytes. Offsets and node counts are
/// 32 bits wide; the limit keeps 16 bytes (the width of the scanner's steps)
/// below 2^32, so that a position one step past the end of the text fits.
@@ -60,14 +68,6 @@ struct node
};
static_assert(sizeof(node) == 16, "node must stay 16 bytes");
// the node kinds are value_t values; the test below (object and array are 1
// and 2) and those of the number kinds depend on this numbering
static_assert(static_cast<std::uint8_t>(value_t::null) == 0 && static_cast<std::uint8_t>(value_t::object) == 1
&& static_cast<std::uint8_t>(value_t::array) == 2 && static_cast<std::uint8_t>(value_t::string) == 3
&& static_cast<std::uint8_t>(value_t::boolean) == 4 && static_cast<std::uint8_t>(value_t::number_integer) == 5
&& static_cast<std::uint8_t>(value_t::number_unsigned) == 6 && static_cast<std::uint8_t>(value_t::number_float) == 7,
"the node format depends on the numbering of value_t");
NLOHMANN_VIEW_ALWAYS_INLINE bool is_container(const node& n) noexcept
{
return static_cast<unsigned>(n.kind) - 1u <= 1u;
@@ -125,7 +125,7 @@ inline void build_object_indexes(document_data& d)
{
for (const std::uint32_t i : d.large_objects)
{
build_object_index(d, d.nodes + i);
build_object_index(d, d.tape + i);
}
}
+3 -20
View File
@@ -180,26 +180,9 @@ inline std::uint64_t int_pow10(unsigned k) noexcept
static const std::array<std::uint64_t, 20> table =
{
{
1u,
10u,
100u,
1000u,
10000u,
100000u,
1000000u,
10000000u,
100000000u,
1000000000u,
10000000000u,
100000000000u,
1000000000000u,
10000000000000u,
100000000000000u,
1000000000000000u,
10000000000000000u,
100000000000000000u,
1000000000000000000u,
10000000000000000000u
1u, 10u, 100u, 1000u, 10000u, 100000u, 1000000u, 10000000u, 100000000u, 1000000000u,
10000000000u, 100000000000u, 1000000000000u, 10000000000000u, 100000000000000u, 1000000000000000u,
10000000000000000u, 100000000000000000u, 1000000000000000000u, 10000000000000000000u
}
};
return table[k];
+33 -90
View File
@@ -31,7 +31,6 @@
#include <cstddef> // nullptr_t, size_t // IWYU pragma: keep
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy, strlen
#include <istream> // istream
#include <iterator> // distance, input_iterator_tag, iterator_traits
#include <map> // map
#include <memory> // unique_ptr
@@ -749,9 +748,9 @@ class basic_json_view
if (editable() && m_doc->edits != nullptr)
{
// positions of moved and new values are not source offsets
return m_node == m_doc->nodes ? m_doc->size + m_doc->edits->text_used : 64;
return m_node == m_doc->tape ? m_doc->size + m_doc->edits->text_used : 64;
}
const node* const end = m_doc->nodes + m_doc->node_count;
const node* const end = m_doc->tape + m_doc->tape_size;
if ((m_node->flags & detail::view::node_flags::storage) != 0)
{
return m_node->len;
@@ -941,9 +940,7 @@ class basic_json_document
const bool ignore_trailing_commas = false)
{
basic_json_document d;
std::string buf;
append_input(buf, std::forward<InputType>(input));
d.build_owned(std::move(buf), allow_exceptions, ignore_comments, ignore_trailing_commas);
d.build_owned(collect(std::forward<InputType>(input)), allow_exceptions, ignore_comments, ignore_trailing_commas);
return d;
}
@@ -992,7 +989,7 @@ class basic_json_document
{
return view_type();
}
return view_type(m_data.get(), m_data->nodes);
return view_type(m_data.get(), m_data->tape);
}
/// deleted: the view of a temporary document would dangle
@@ -1018,7 +1015,7 @@ class basic_json_document
/// number of index nodes (values plus object keys)
std::size_t node_count() const noexcept
{
return m_data ? m_data->node_count : 0;
return m_data ? m_data->tape_size : 0;
}
/// bytes held by the document (index, decoded strings, owned text or image)
@@ -1028,8 +1025,8 @@ class basic_json_document
{
return 0;
}
return sizeof(document_data) + (m_data->inline_capacity * sizeof(detail::view::node))
+ (m_data->nodes != m_data->inline_nodes ? m_data->node_capacity * sizeof(detail::view::node) : 0)
return sizeof(document_data) + (m_data->inline_cap * sizeof(detail::view::node))
+ (m_data->tape != m_data->inline_tape ? m_data->tape_cap * sizeof(detail::view::node) : 0)
+ m_data->arena.capacity() + m_data->owned.capacity() + m_data->owned_image.capacity()
+ (m_data->indexes.capacity() * sizeof(document_data::object_index)) + (m_data->index_slots.capacity() * sizeof(std::uint32_t))
+ (m_data->large_objects.capacity() * sizeof(std::uint32_t))
@@ -1053,20 +1050,20 @@ class basic_json_document
// (the decoded strings of a loaded image stay in the image)
const bool arena_in_use = d.base[1] == d.arena.data();
const bool shrink_arena = d.arena.capacity() > d.arena.size();
std::vector<char> arena(shrink_arena && arena_in_use ? d.arena : std::vector<char>());
std::string arena(shrink_arena && arena_in_use ? d.arena : std::string());
// (edits link to the nodes of the index, which then stays in place)
const bool shrink_nodes = d.nodes != d.inline_nodes && d.node_count != d.node_capacity && d.edits == nullptr;
const bool into_header = d.node_count <= d.inline_capacity;
const bool shrink_tape = d.tape != d.inline_tape && d.tape_size != d.tape_cap && d.edits == nullptr;
const bool into_header = d.tape_size <= d.inline_cap;
std::vector<document_data::object_index> indexes(d.indexes.capacity() > d.indexes.size() ? d.indexes : std::vector<document_data::object_index>());
std::vector<std::uint32_t> index_slots(d.index_slots.capacity() > d.index_slots.size() ? d.index_slots : std::vector<std::uint32_t>());
node* fresh = (shrink_nodes && !into_header) ? static_cast<node*>(::operator new (d.node_count * sizeof(node))) : d.inline_nodes;
node* fresh = (shrink_tape && !into_header) ? static_cast<node*>(::operator new (d.tape_size * sizeof(node))) : d.inline_tape;
if (shrink_nodes)
if (shrink_tape)
{
std::memcpy(fresh, d.nodes, d.node_count * sizeof(node));
::operator delete (d.nodes);
d.nodes = fresh;
d.node_capacity = into_header ? d.inline_capacity : d.node_count;
std::memcpy(fresh, d.tape, d.tape_size * sizeof(node));
::operator delete (d.tape);
d.tape = fresh;
d.tape_cap = into_header ? d.inline_cap : d.tape_size;
}
if (shrink_arena)
{
@@ -1328,8 +1325,7 @@ class basic_json_document
}
}
/// parse a buffer the document takes ownership of (it replaces the
/// previous owned copy)
/// parse a buffer the document takes ownership of
void build_owned(std::string&& buf, bool allow_exceptions, bool comments, bool trailing_commas)
{
ensure_data(buf.data(), buf.size());
@@ -1337,29 +1333,6 @@ class basic_json_document
build(m_data->owned.data(), m_data->owned.size(), allow_exceptions, comments, trailing_commas, true, true);
}
/// parse a copy of the input; the copy is appended to the cleared owned
/// buffer, so that its capacity is reused
template<typename T>
void build_copy(T&& input, bool allow_exceptions, bool comments, bool trailing_commas)
{
if (!m_data)
{
std::string buf;
append_input(buf, std::forward<T>(input));
build_owned(std::move(buf), allow_exceptions, comments, trailing_commas);
return;
}
document_data& d = *m_data;
// (the previous text is gone from here on: should reading the input
// throw, the document is discarded)
d.discarded = true;
d.src = "";
d.size = 0;
d.owned.clear();
append_input(d.owned, std::forward<T>(input));
build(d.owned.data(), d.owned.size(), allow_exceptions, comments, trailing_commas, true, true);
}
/// sentinel: src[size] is readable and 0 (std::string, C strings)
void build(const char* src, std::size_t size, bool allow_exceptions, bool comments, bool trailing_commas, bool owned, bool sentinel)
{
@@ -1372,7 +1345,7 @@ class basic_json_document
d.owned_image.clear();
d.src = src;
d.size = size;
d.node_count = 0;
d.tape_size = 0;
d.edits.reset(); // (views of the previous text end here anyway)
d.base[2] = nullptr;
d.arena.clear();
@@ -1417,6 +1390,7 @@ class basic_json_document
template<typename CharT>
void read_kind(CharT* s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::c_string> /*unused*/)
{
static_assert(sizeof(CharT) == 1 && std::is_integral<typename std::remove_cv<CharT>::type>::value, "json_view parses byte (char-like) input");
if (s == nullptr)
{
build("", 0, ae, c, tc, false, true);
@@ -1432,6 +1406,7 @@ class basic_json_document
void read_kind(Array& a, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::char_array> /*unused*/)
{
using CharT = typename std::remove_cv<typename std::remove_extent<Array>::type>::type;
static_assert(sizeof(CharT) == 1 && std::is_integral<CharT>::value, "json_view parses byte (char-like) input");
const std::size_t n = std::extent<Array>::value;
// a trailing NUL (string literals) is not part of the text, as for
// parse(), and serves as sentinel
@@ -1451,25 +1426,13 @@ class basic_json_document
template<typename T>
void read_kind(const T& s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::copy_range> /*unused*/)
{
build_copy(s, ae, c, tc);
}
template<typename CharT>
void read_kind(CharT* s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::wide_c_string> /*unused*/)
{
if (s == nullptr)
{
build("", 0, ae, c, tc, false, true);
return;
}
// NUL-terminated, as for the single-byte C strings
build_copy(detail::view::make_adapter_input(detail::input_adapter(s, s + std::char_traits<typename std::remove_cv<CharT>::type>::length(s))), ae, c, tc);
build_owned(std::string(reinterpret_cast<const char*>(s.data()), static_cast<std::size_t>(s.size())), ae, c, tc); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
template<typename T>
void read_kind(T&& input, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::adapter> /*unused*/)
{
build_copy(std::forward<T>(input), ae, c, tc);
build_owned(detail::view::collect_adapter(detail::input_adapter(std::forward<T>(input))), ae, c, tc);
}
template<typename IteratorType>
@@ -1493,45 +1456,25 @@ class basic_json_document
template<typename IteratorType>
void read_range_impl(IteratorType first, IteratorType last, bool ae, bool c, bool tc, std::false_type /*other*/)
{
build_copy(detail::view::make_adapter_input(detail::input_adapter(first, last)), ae, c, tc);
}
/// append the text of an input to buf: a contiguous byte input as one
/// block, a stream in blocks, an input adapter (made here for any other
/// input) character by character
template<typename T>
static void append_input(std::string& buf, T&& input)
{
using D = typename std::decay<T>::type;
// NOLINTBEGIN(readability-avoid-nested-conditional-operator): a constant expression of C++11
append_input_impl(buf, std::forward<T>(input), std::integral_constant < int, detail::is_contiguous_byte_container<D>::value ? 0
: std::is_base_of<std::istream, D>::value ? 1
: detail::view::is_adapter_input<D>::value ? 2 : 3 > {});
// NOLINTEND(readability-avoid-nested-conditional-operator)
build_owned(detail::view::collect_adapter(detail::input_adapter(first, last)), ae, c, tc);
}
template<typename T>
static void append_input_impl(std::string& buf, const T& s, std::integral_constant<int, 0> /*contiguous bytes*/)
static std::string collect(T&& input)
{
buf.append(reinterpret_cast<const char*>(s.data()), static_cast<std::size_t>(s.size())); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
template<typename Stream>
static void append_input_impl(std::string& buf, Stream&& is, std::integral_constant<int, 1> /*stream*/) // NOLINT(cppcoreguidelines-missing-std-forward): read in place, as by the library's stream adapter
{
detail::view::append_stream(buf, is);
}
template<typename Adapter>
static void append_input_impl(std::string& buf, detail::view::adapter_input<Adapter>&& input, std::integral_constant<int, 2> /*input adapter*/)
{
detail::view::append_adapter(buf, std::move(input.ia));
return collect_impl(std::forward<T>(input), std::integral_constant<bool, detail::is_contiguous_byte_container<typename std::decay<T>::type>::value> {});
}
template<typename T>
static void append_input_impl(std::string& buf, T&& input, std::integral_constant<int, 3> /*other*/)
static std::string collect_impl(const T& s, std::true_type /*contiguous*/)
{
detail::view::append_adapter(buf, detail::input_adapter(std::forward<T>(input)));
return {reinterpret_cast<const char*>(s.data()), static_cast<std::size_t>(s.size())}; // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
template<typename T>
static std::string collect_impl(T&& s, std::false_type /*other*/)
{
return detail::view::collect_adapter(detail::input_adapter(std::forward<T>(s)));
}
std::unique_ptr<document_data, document_data::deleter> m_data{}; // NOLINT(readability-redundant-member-init)
+248 -519
View File
File diff suppressed because it is too large. Load diff
-108
View File
@@ -28,8 +28,6 @@ using json_view_test::has_duplicate_keys;
#include <cstdio>
#include <cstring>
#include <iomanip>
#include <ios>
#include <istream>
#include <iterator>
#include <limits>
#include <list>
@@ -358,112 +356,6 @@ TEST_CASE("json_view")
CHECK(materialized<json_document>(wide) == json::parse(wide));
CHECK(json_document::parse(static_cast<const char*>(nullptr), false).is_discarded());
CHECK(json_document::parse("", false).is_discarded());
// wide-character arrays and pointers are converted to UTF-8, as by
// parse(); the pointers are NUL-terminated
CHECK(materialized<json_document>(L"[\"ä€\", 1]") == json::parse(L"[\"ä€\", 1]"));
CHECK(materialized<json_document>(u"[\"ä€\", 1]") == json::parse(u"[\"ä€\", 1]"));
CHECK(materialized<json_document>(U"[\"ä€\", 1]") == json::parse(U"[\"ä€\", 1]"));
const wchar_t* const wide_pointer = wide.c_str();
const json_document from_wide_pointer = json_document::parse(wide_pointer);
CHECK(from_wide_pointer.owns_source());
CHECK(from_wide_pointer.root().materialize() == json::parse(wide));
const std::u16string wide16 = u"{\"\U0001F600\": [true]}";
CHECK(materialized<json_document>(wide16.c_str()) == json::parse(wide16));
const std::u32string wide32 = U"{\"\U0001F600\": [true]}";
CHECK(materialized<json_document>(wide32.c_str()) == json::parse(wide32));
CHECK(json_document::parse(static_cast<const wchar_t*>(nullptr), false).is_discarded());
// FILE*: read to its end
std::FILE* file = std::tmpfile(); // NOLINT(cppcoreguidelines-owning-memory)
if (file != nullptr)
{
CHECK(std::fputs(text.c_str(), file) >= 0);
CHECK(std::fseek(file, 0, SEEK_SET) == 0);
const json_document from_file = json_document::parse(file);
CHECK(from_file.owns_source());
CHECK(from_file.root().materialize() == expected);
CHECK(std::fclose(file) == 0); // NOLINT(cppcoreguidelines-owning-memory)
}
}
SECTION("streams")
{
// longer than one block of the bulk read
std::string text = "[";
for (int i = 0; i < 5000; ++i)
{
text += (i != 0 ? ",\"" : "\"") + std::to_string(i) + "\"";
}
text += ']';
std::istringstream stream(text);
const json_document from_stream = json_document::parse(stream);
CHECK(from_stream.source() == text);
CHECK(from_stream.root().materialize() == json::parse(text));
// the stream is read to its end, and eofbit set, as by parse()
CHECK(stream.eof());
CHECK(!stream.bad());
std::istringstream empty_stream;
CHECK(json_document::parse(empty_stream, false).is_discarded());
CHECK(empty_stream.eof());
std::istringstream rvalue_source("[1]");
CHECK(json_document::parse_copy(std::move(rvalue_source)).owns_source());
// a stream without a buffer is an empty input
std::istream no_buffer(nullptr);
CHECK(json_document::parse(no_buffer, false).is_discarded());
#if !defined(JSON_NOEXCEPTION)
// an exception while reading the input leaves a discarded document
json_document d = json_document::parse(std::vector<char>(text.begin(), text.end()));
std::istringstream throwing("[1]");
throwing.exceptions(std::ios::eofbit);
CHECK_THROWS_AS(d.read(throwing), std::ios_base::failure&);
CHECK(d.is_discarded());
CHECK(d.root().is_discarded());
d.read("[2]");
CHECK(d.root().materialize() == json::parse("[2]"));
#endif
}
SECTION("read() reuses the owned copy")
{
std::string big = "[";
for (int i = 0; i < 1000; ++i)
{
big += (i != 0 ? ",\"" : "\"") + std::to_string(i) + "\"";
}
big += ']';
const std::string small = "{\"a\": [1, 2]}";
// copied inputs are written into the existing buffer
json_document d = json_document::parse(std::vector<char>(big.begin(), big.end()));
REQUIRE(d.owns_source());
const char* const buffer = d.source().data();
d.read(std::vector<char>(small.begin(), small.end()));
CHECK(d.owns_source());
CHECK(d.source().data() == buffer);
CHECK(d.root().materialize() == json::parse(small));
std::istringstream stream(small);
d.read(stream);
CHECK(d.source().data() == buffer);
CHECK(d.root().materialize() == json::parse(small));
d.read(std::wstring(L"[3]"));
CHECK(d.source().data() == buffer);
CHECK(d.root().materialize() == json::parse("[3]"));
// a moved-in string replaces it (long enough to be on the heap)
std::string moved = big;
const char* const moved_buffer = moved.data();
d.read(std::move(moved));
CHECK(d.source().data() == moved_buffer);
CHECK(d.root().materialize() == json::parse(big));
// a borrowed input leaves nothing owned
d.read(big);
CHECK(!d.owns_source());
CHECK(d.source().data() == big.data());
}
SECTION("integer arguments do not compile")
+14 -27
View File
@@ -114,9 +114,9 @@ json value_of(const document_data& d, const node*& n)
json value_of(const built& b)
{
const node* n = b.data->nodes;
const node* n = b.data->tape;
json v = value_of(*b.data, n);
CHECK(n == b.data->nodes + b.data->node_count);
CHECK(n == b.data->tape + b.data->tape_size);
return v;
}
@@ -367,17 +367,6 @@ TEST_CASE("json_view builder")
SECTION("deep nesting")
{
// the deepest nesting is recorded (materialize() sizes its stack from it)
CHECK(build("1", false, false, false).data->max_depth == 0);
CHECK(build("[]", false, false, false).data->max_depth == 1);
CHECK(build(R"([[1],{"a":[[]]},[]])", false, false, false).data->max_depth == 4);
// a stack that turns out too small still grows
const std::string nested = R"([[1],{"a":[[{"b":[]}]]},[]])";
const built shallow = build(nested, false, false, false);
REQUIRE(shallow.ok);
shallow.data->max_depth = 1;
CHECK(nlohmann::detail::view::materialize<json, false>(*shallow.data, shallow.data->nodes) == json::parse(nested));
// the open containers beyond 64 levels live on the heap
for (const std::size_t depth :
{
@@ -387,9 +376,8 @@ TEST_CASE("json_view builder")
const std::string arrays = std::string(depth, '[') + std::string(depth, ']');
const built b = build(arrays, false, false, false);
REQUIRE(b.ok);
CHECK(b.data->node_count == depth);
CHECK(b.data->nodes[0].next == depth);
CHECK(b.data->max_depth == depth);
CHECK(b.data->tape_size == depth);
CHECK(b.data->tape[0].next == depth);
std::string objects;
for (std::size_t i = 0; i < depth; ++i)
{
@@ -398,8 +386,7 @@ TEST_CASE("json_view builder")
objects += '1' + std::string(depth, '}');
const built o = build(objects, false, false, false);
REQUIRE(o.ok);
CHECK(o.data->node_count == (2 * depth) + 1);
CHECK(o.data->max_depth == depth);
CHECK(o.data->tape_size == (2 * depth) + 1);
CHECK(!build(std::string(depth, '[') + std::string(depth - 1, ']'), false, false, false).ok);
}
}
@@ -572,7 +559,7 @@ TEST_CASE("json_view node integer bits")
CAPTURE(c.text)
const built b = build(c.text, false, false, true);
REQUIRE(b.ok);
const node& n = b.data->nodes[1];
const node& n = b.data->tape[1];
CHECK(integer_bits(n) == c.bits);
CHECK(n.len == static_cast<std::uint32_t>(c.bits));
CHECK(n.next == static_cast<std::uint32_t>(c.bits >> 32));
@@ -587,23 +574,23 @@ TEST_CASE("json_view node array size limit")
// for absurd counts everywhere)
std::unique_ptr<document_data, document_data::deleter> d(document_data::create(0));
d->reserve(8);
REQUIRE(d->node_capacity >= 8);
d->node_count = 2;
const std::size_t cap = d->node_capacity;
node* const nodes = d->nodes;
REQUIRE(d->tape_cap >= 8);
d->tape_size = 2;
const std::size_t cap = d->tape_cap;
node* const tape = d->tape;
#if !defined(JSON_NOEXCEPTION)
const std::size_t too_many = document_data::max_nodes() + 1;
CHECK_THROWS_AS(d->reserve(too_many), std::bad_alloc&);
CHECK_THROWS_AS(d->reserve((std::numeric_limits<std::size_t>::max)()), std::bad_alloc&);
// the array is unchanged
CHECK(d->nodes == nodes);
CHECK(d->node_capacity == cap);
CHECK(d->node_count == 2);
CHECK(d->tape == tape);
CHECK(d->tape_cap == cap);
CHECK(d->tape_size == 2);
#endif
// the largest count that fits is not refused by the check (nothing is
// allocated for a count that is already there)
d->reserve(cap);
CHECK(d->nodes == nodes);
CHECK(d->tape == tape);
}