wt/json

0 Fork 0

mirror of https://github.com/nlohmann/json.git synced 2026-10-21 11:29:53 +00:00

Files

T

Niels Lohmann 823eee3b59 Add editable json_documents: set, push_back, insert, and erase (#5630 )

* Add editable json_documents: set, push_back, insert, and erase

basic_json_document gets a second template parameter, Editable
(false by default), plus the aliases json_editable_document,
json_editable_view, ordered_json_editable_document and
ordered_json_editable_view.

Editable documents can change values and structure without
rewriting the source text: set()/push_back() on values, keys,
array indices and JSON pointers; insert() before an array
element; erase() of an object key, array index or JSON pointer.

New values and element sequences go into edit storage that the
document owns and never moves, so views keep referring to their
value across edits and a parsed node never moves. Read-only
documents walk the plain node array and are unaffected.

Strings are checked for UTF-8 on entry, so dump() of an editable
document never throws type_error.316. Binary values cannot be
stored (type_error.319).

A seeded differential test applies random edits to an editable
document and to the equivalent ordered_json and compares both
after every step.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix old clang and MSVC C4127 in editable json documents

Value-initialize the const std::less in find_parent (clang 3.4/3.6 do not
implement DR 253), and test the Editable template argument through a
function to avoid MSVC C4127.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Copy nested values into a document without recursion

Counting and copying the nodes of a view or a basic_json value into an
editable document recursed once per nesting level, so that a deeply
nested value overflowed the stack. The four functions now walk the value
with an explicit stack, as materialize() does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not leave a half-assigned container when set_moved() throws

assign() rewrote the kind, length and extent of a slot before set_moved()
registered its new element sequence. set_moved() can throw
(std::bad_alloc from reserving the bookkeeping vectors) for a slot that
is not moved yet, which left a container without the moved flag that
showed its old children. The bookkeeping is now reserved first
(reserve_moved()), so that nothing after the first write to the slot can
throw. block_of() reserves before it marks anything for the same reason.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Check strings copied from another document for valid UTF-8

An editable document only holds valid UTF-8, but copy_scalar() copied the
strings and keys of a view of another document unchecked. A document of
a weaker check (or a borrowed text that changed after parsing) could
therefore bring ill-formed UTF-8 into it. The copy is checked now, with
the error that dump() reports for the string; copies within the same
document stay unchecked.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Allow the edit arena to grow up to 4 GiB - 1 bytes

append_text() doubled the capacity of the arena and rejected the result
if it exceeded 4 GiB, so that an arena of more than 2 GiB could not grow
even though the 32-bit offsets of nodes address 4 GiB - 1 bytes. The
capacity is now clamped to that limit (text_capacity()), and an append
is rejected only if the bytes themselves do not fit.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Set a null value below a pointer as basic_json does

set(json_pointer, value) turned a null parent into an object for every
last reference token, so "/a/0" and "/a/-" below {"a":null} made
{"a":{"0":1}}, where basic_json's operator[](json_pointer) makes an
array. A null parent now becomes an array for "-" and for digit tokens
(filled with nulls up to the index) and an object otherwise. An invalid
index is reported before the parent changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Replace a container by a scalar in constant time once it is linked

Replacing an array or object that spans several nodes by a scalar looks
up its parent from the root (find_parent), which is linear in the size
of the document: setting every element of a 20,000 element array took
more than half a second. The parent only has to switch to links once;
afterward the extent of the replaced value no longer matters. Nodes that
an entry of a moved sequence links to are now marked (node_flags::linked),
and assign() skips the lookup for them.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Update the json_view banner and type_error.319 for editable documents

The banner still described a read-only view, and the documentation of
type_error.319 listed set and push_back only, although insert rejects
binary values as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Assign the member that lookups find when set() meets duplicate keys

An editable document reads the last member of a repeated key (as the
read-only document does), but set(object, key, value) assigned the first
one: a view taken from object[key] before the call did not show the new
value. It now assigns the last member's value, keeps the key at the
position of its first occurrence (where materialize() puts it), and drops
the other members.

The seeded differential test now also edits documents whose objects
repeat keys and compares every lookup (operator[], at, find, value,
contains, count, and JSON pointers) with the parsed basic_json value.
A targeted test covers duplicates before and after edits that move the
object, in large objects with an index, in moved arrays, and in values
copied from other documents.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Assign the first member when set() meets duplicate keys again

Lookups find the first member of a repeated key again, so set(object, key,
value) assigns that member (keeping the key where it is) and drops the
others, as documented before, and a view taken from object[key] shows the
new value. This undoes the code and documentation changes of 5cbb8e6d6.
Lookups in edited objects (navigation of editable documents) stop at the
first match, like those of parsed objects.

The tests that commit added expect the first member from lookups now. In the
seeded differential test, the documents with repeated keys repeat them after
the real members; the reference holds the first members, which the edits
address and the lookups are compared with, while materialize() is compared
with what parse() makes of the document's dump.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC C4127 in the lookups of editable json documents

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy 22 findings in the json_view headers

Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast
would trip -Wcast-align=strict), use auto for a cast initialiser, add
parentheses to a mixed expression, and name bugprone-std-namespace-modification
in the NOLINTs of the tuple_size/tuple_element specialisations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
(cherry picked from commit 0effbaa7a2)

* Fix clang-tidy 22 findings in unit-json_view_edit.cpp and edit.hpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep a raw string with a backslash out of a CHECK (MSVC C2017)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make json_view.hpp pass include-what-you-use

ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), and the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), plus <version> for
std::nullptr_t, and removing <cstddef> and edit_storage.hpp.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- drop edit_storage.hpp, which edit.hpp includes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the view's edit headers clean for clang-tidy in C++17 mode

clang-tidy 22 checks the headers with -std=gnu++17 as well, where it asks
for a transparent comparator and std::make_unique. The regions map uses its
default comparator, and the allocation of the edit state keeps new (the
library is C++11) with a NOLINT.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Follow the renames from the review of the view builder

The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Expect count() to count every member in the edit tests

Since #5622, count() counts every member with the key, so documents
with duplicate keys no longer give 1.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Count matching keys as unsigned in the json_view edit tests

Like in the access test, adding the int result of the conditional to a
std::size_t fails -Werror=sign-conversion with GCC.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the review of erase() in editable json documents

From gregmarr's review of #5630:

- invalid_iterator.202 is thrown by overloads 1 and 2 only; the page said
  "every overload" and then excluded 3 in parentheses.
- The note on links says what moves (the links after the removed element,
  as insert() moves them the other way) and that every iterator over the
  container is invalidated, not only those after the removed element.
- edit.hpp compares keys with document_data::key_equals().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the review of editable json documents

From gregmarr's review of #5630:

- set() with a JSON pointer below a null value checks for "-" before
  scanning the token for digits.
- node_flags::storage is defined as escaped | edited, and the comment on
  document_data::base says that the fourth slot is unused.
- source_extent() names its root check.
- The integer-index overloads of set(), insert(), and erase() use a named
  trait instead of a negated condition, which astyle formats like the
  library's other enable_ifs.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the rest of the review of editable json documents

From gregmarr's review of #5630:

- alloc_nodes() gives a request of half of the next chunk or more a chunk
  of its own, so that the rest of the current chunk stays available for
  smaller requests instead of being abandoned.
- The comments on edit_state, append_text(), and reserve_moved() say that
  views stay valid across every edit (not just the first), that n is 0 for
  empty strings, and why both capacities are checked.
- The architecture page names flag bits 3, 4, and 5 (linked) separately
  and says that a discarded view has no node; erase.md rewords overload 3
  and says why it never throws invalid_iterator.202.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Call alloc_nodes() before reading chunks.back() in the json_view chunk test

The operands of == are unsequenced, and MSVC evaluated
d->edits->chunks.back() before alloc_nodes() added the new chunk, so the
check compared the result with the previous chunk and failed on
msvc-arm64 (Debug and Release).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

2026-10-21 10:21:29 +02:00

7.0 KiB

Raw Blame History

nlohmann::basic_json_view

Defined in header <nlohmann/json_view.hpp>

template<typename BasicJsonType, bool Editable = false>
class basic_json_view;

A read-only handle to one value of a basic_json_document: two pointers (a pointer to the document and a pointer into its index), trivially copyable. A view is valid as long as

the document is alive,
the document has not been re-parsed with read() (or parse() into it) or shrunk with shrink_to_fit() since the view was taken, and
if the document borrows its source text, that text is still alive and unmodified (floating-point values and unescaped strings are read from it lazily).

Moving the document itself does not invalidate its views: the index is heap-allocated independently of the basic_json_document object.

basic_json_view provides the read-only part of the BasicJsonType interface:

the type-inspection functions
element access
lookup
iteration
conversion: get<T>(), get_string(), number_token(), and materialize() to build the BasicJsonType value of a subtree on demand
comparison: operator== and operator!= compare two views, or a view and a BasicJsonType value, without ever building a BasicJsonType value for a view; no ordering comparison (#!cpp operator<) is provided

operator[], at, contains, and value also accept a json_pointer. Unlike basic_json, there are no implicit conversion operators (no #!cpp operator ValueType()), and no #!cpp get_ptr() or #!cpp get_ref(); see get.

basic_json_view itself is always read-only -- it never has a set or push_back of its own. A view of an editable document (#!cpp Editable == true) sees every edit made through basic_json_document::set and basic_json_document::push_back: once a value is changed, every view that still refers to it -- including ones taken before the change -- reads the new value. See Edits.

Template parameters

BasicJsonType: a specialization of basic_json, matching the basic_json_document the view was taken from.
Editable: whether the view is of an editable document, matching the basic_json_document it was taken from (optional, #!cpp false by default). See Edits.

Specializations

json_view - views of a json_document
ordered_json_view - views of an ordered_json_document
json_editable_view - views of a json_editable_document
ordered_json_editable_view - views of an ordered_json_editable_document

Member types

value_t - the JSON type enumeration, see basic_json::value_t
string_t, number_integer_t, number_unsigned_t, number_float_t, json_pointer - the corresponding member types of BasicJsonType
size_type - #!cpp std::size_t
string_view_t - #!cpp std::string_view on C++17 and newer, a minimal internal substitute otherwise
iterator, const_iterator - a forward iterator over the elements of an array or the member values of an object, in document order; both names refer to the same type, since a view is always read-only
number_format - how dump() writes numbers

Member functions

(constructor)

Object inspection

type - return the type of the value
type_name - return the type as string
is_null - return whether the value is null
is_boolean - return whether the value is a boolean
is_number - return whether the value is a number
is_number_integer - return whether the value is an integer number
is_number_unsigned - return whether the value is an unsigned integer number
is_number_float - return whether the value is a floating-point number
is_string - return whether the value is a string
is_array - return whether the value is an array
is_object - return whether the value is an object
is_binary - return whether the value is a binary array (always #!cpp false)
is_primitive - return whether the type is primitive
is_structured - return whether the type is structured
is_discarded - return whether the view is invalid

Element access

at - access specified element with bounds checking
operator[] - access specified element
value - access specified element with default value
front - access the first element
back - access the last element

Lookup

find - find an element in an object
count - returns the number of occurrences of a key in an object
contains - check the existence of an element in an object

Iterators

begin - returns an iterator to the first element
cbegin - returns a const iterator to the first element
end - returns an iterator to one past the last element
cend - returns a const iterator to one past the last element
items - wrapper to access iterator member functions in range-based for

Capacity

size - return the number of elements
empty - return whether the value has no elements

Conversion

get - get a value
get_to - get a value and write it to a destination
get_string - get a string value without a copy
number_token - get a number's token text without a copy
materialize - build the BasicJsonType value of this subtree

Comparison

operator== - comparison: equal
operator!= - comparison: not equal

Serialization

dump - serialize to a JSON-formatted string
operator<< - serialize to stream

Source access

source_offset - byte offset of this value in the document's source text

Version history

Added in version 3.13.0.

7.0 KiB Raw Blame History