wt/json
1
0 Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-26 12:14:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add editable json_documents: set, push_back, insert, and erase (#5630)

Browse Source 
* Add editable json_documents: set, push_back, insert, and erase

basic_json_document gets a second template parameter, Editable
(false by default), plus the aliases json_editable_document,
json_editable_view, ordered_json_editable_document and
ordered_json_editable_view.

Editable documents can change values and structure without
rewriting the source text: set()/push_back() on values, keys,
array indices and JSON pointers; insert() before an array
element; erase() of an object key, array index or JSON pointer.

New values and element sequences go into edit storage that the
document owns and never moves, so views keep referring to their
value across edits and a parsed node never moves. Read-only
documents walk the plain node array and are unaffected.

Strings are checked for UTF-8 on entry, so dump() of an editable
document never throws type_error.316. Binary values cannot be
stored (type_error.319).

A seeded differential test applies random edits to an editable
document and to the equivalent ordered_json and compares both
after every step.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix old clang and MSVC C4127 in editable json documents

Value-initialize the const std::less in find_parent (clang 3.4/3.6 do not
implement DR 253), and test the Editable template argument through a
function to avoid MSVC C4127.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Copy nested values into a document without recursion

Counting and copying the nodes of a view or a basic_json value into an
editable document recursed once per nesting level, so that a deeply
nested value overflowed the stack. The four functions now walk the value
with an explicit stack, as materialize() does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not leave a half-assigned container when set_moved() throws

assign() rewrote the kind, length and extent of a slot before set_moved()
registered its new element sequence. set_moved() can throw
(std::bad_alloc from reserving the bookkeeping vectors) for a slot that
is not moved yet, which left a container without the moved flag that
showed its old children. The bookkeeping is now reserved first
(reserve_moved()), so that nothing after the first write to the slot can
throw. block_of() reserves before it marks anything for the same reason.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Check strings copied from another document for valid UTF-8

An editable document only holds valid UTF-8, but copy_scalar() copied the
strings and keys of a view of another document unchecked. A document of
a weaker check (or a borrowed text that changed after parsing) could
therefore bring ill-formed UTF-8 into it. The copy is checked now, with
the error that dump() reports for the string; copies within the same
document stay unchecked.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Allow the edit arena to grow up to 4 GiB - 1 bytes

append_text() doubled the capacity of the arena and rejected the result
if it exceeded 4 GiB, so that an arena of more than 2 GiB could not grow
even though the 32-bit offsets of nodes address 4 GiB - 1 bytes. The
capacity is now clamped to that limit (text_capacity()), and an append
is rejected only if the bytes themselves do not fit.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Set a null value below a pointer as basic_json does

set(json_pointer, value) turned a null parent into an object for every
last reference token, so "/a/0" and "/a/-" below {"a":null} made
{"a":{"0":1}}, where basic_json's operator[](json_pointer) makes an
array. A null parent now becomes an array for "-" and for digit tokens
(filled with nulls up to the index) and an object otherwise. An invalid
index is reported before the parent changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Replace a container by a scalar in constant time once it is linked

Replacing an array or object that spans several nodes by a scalar looks
up its parent from the root (find_parent), which is linear in the size
of the document: setting every element of a 20,000 element array took
more than half a second. The parent only has to switch to links once;
afterward the extent of the replaced value no longer matters. Nodes that
an entry of a moved sequence links to are now marked (node_flags::linked),
and assign() skips the lookup for them.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Update the json_view banner and type_error.319 for editable documents

The banner still described a read-only view, and the documentation of
type_error.319 listed set and push_back only, although insert rejects
binary values as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Assign the member that lookups find when set() meets duplicate keys

An editable document reads the last member of a repeated key (as the
read-only document does), but set(object, key, value) assigned the first
one: a view taken from object[key] before the call did not show the new
value. It now assigns the last member's value, keeps the key at the
position of its first occurrence (where materialize() puts it), and drops
the other members.

The seeded differential test now also edits documents whose objects
repeat keys and compares every lookup (operator[], at, find, value,
contains, count, and JSON pointers) with the parsed basic_json value.
A targeted test covers duplicates before and after edits that move the
object, in large objects with an index, in moved arrays, and in values
copied from other documents.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Assign the first member when set() meets duplicate keys again

Lookups find the first member of a repeated key again, so set(object, key,
value) assigns that member (keeping the key where it is) and drops the
others, as documented before, and a view taken from object[key] shows the
new value. This undoes the code and documentation changes of 5cbb8e6d6.
Lookups in edited objects (navigation of editable documents) stop at the
first match, like those of parsed objects.

The tests that commit added expect the first member from lookups now. In the
seeded differential test, the documents with repeated keys repeat them after
the real members; the reference holds the first members, which the edits
address and the lookups are compared with, while materialize() is compared
with what parse() makes of the document's dump.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC C4127 in the lookups of editable json documents

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy 22 findings in the json_view headers

Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast
would trip -Wcast-align=strict), use auto for a cast initialiser, add
parentheses to a mixed expression, and name bugprone-std-namespace-modification
in the NOLINTs of the tuple_size/tuple_element specialisations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
(cherry picked from commit 0effbaa7a2)

* Fix clang-tidy 22 findings in unit-json_view_edit.cpp and edit.hpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep a raw string with a backslash out of a CHECK (MSVC C2017)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make json_view.hpp pass include-what-you-use

ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), and the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), plus <version> for
std::nullptr_t, and removing <cstddef> and edit_storage.hpp.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- drop edit_storage.hpp, which edit.hpp includes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the view's edit headers clean for clang-tidy in C++17 mode

clang-tidy 22 checks the headers with -std=gnu++17 as well, where it asks
for a transparent comparator and std::make_unique. The regions map uses its
default comparator, and the allocation of the edit state keeps new (the
library is C++11) with a NOLINT.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Follow the renames from the review of the view builder

The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Expect count() to count every member in the edit tests

Since #5622, count() counts every member with the key, so documents
with duplicate keys no longer give 1.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Count matching keys as unsigned in the json_view edit tests

Like in the access test, adding the int result of the conditional to a
std::size_t fails -Werror=sign-conversion with GCC.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the review of erase() in editable json documents

From gregmarr's review of #5630:

- invalid_iterator.202 is thrown by overloads 1 and 2 only; the page said
  "every overload" and then excluded 3 in parentheses.
- The note on links says what moves (the links after the removed element,
  as insert() moves them the other way) and that every iterator over the
  container is invalidated, not only those after the removed element.
- edit.hpp compares keys with document_data::key_equals().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the review of editable json documents

From gregmarr's review of #5630:

- set() with a JSON pointer below a null value checks for "-" before
  scanning the token for digits.
- node_flags::storage is defined as escaped | edited, and the comment on
  document_data::base says that the fourth slot is unused.
- source_extent() names its root check.
- The integer-index overloads of set(), insert(), and erase() use a named
  trait instead of a negated condition, which astyle formats like the
  library's other enable_ifs.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the rest of the review of editable json documents

From gregmarr's review of #5630:

- alloc_nodes() gives a request of half of the next chunk or more a chunk
  of its own, so that the rest of the current chunk stays available for
  smaller requests instead of being abandoned.
- The comments on edit_state, append_text(), and reserve_moved() say that
  views stay valid across every edit (not just the first), that n is 0 for
  empty strings, and why both capacities are checked.
- The architecture page names flag bits 3, 4, and 5 (linked) separately
  and says that a discarded view has no node; erase.md rewords overload 3
  and says why it never throws invalid_iterator.202.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Call alloc_nodes() before reading chunks.back() in the json_view chunk test

The operands of == are unsequenced, and MSVC evaluated
d->edits->chunks.back() before alloc_nodes() added the new chunk, so the
check compared the result with the previous chunk and failed on
msvc-arm64 (Debug and Release).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann authored and GitHub committed 2026-10-21 10:21:29 +02:00
1 parent 3ff37ead75
commit 823eee3b59
44 files changed
+6081 -584
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/mkdocs/docs/api/basic_json_document/set.md
+203
View File
@@ -0,0 +1,203 @@
# <small>nlohmann::basic_json_document::</small>set
```cpp
// (1)
template<typename V>
view_type set(view_type target, V&& value);
// (2)
template<typename V>
view_type set(view_type object, string_view_t key, V&& value);
// (3)
template<typename I, typename V>
view_type set(view_type array, I idx, V&& value);
// (4)
template<typename V>
view_type set(const json_pointer& ptr, V&& value);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `set`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
1. Replaces the value `target` refers to with `value`.
2. Sets the member `key` of the object `object` to `value`: assigns it if `object` already has a member with this
key -- the first one, should the key occur more than once, and the later duplicates are then dropped (see the
[Notes](#notes) below) -- or appends a new member at the end otherwise. A [null](../basic_json_view/is_null.md)
`object` first becomes an empty object.
3. Assigns `value` to the element at index `idx` of the array `array`, which must already exist (`#!cpp idx <
array.size()`).
4. Sets the value the JSON pointer `ptr` refers to, relative to [`root()`](root.md), to `value`. The *parent* of the
target must already exist: an object member is set as in 2. (added if it does not exist yet), an array element is
assigned as in 3., and a last reference token of `#!cpp "-"`, or equal to the size of the array, appends `value`
instead, exactly as [`push_back`](push_back.md) would. A [null](../basic_json_view/is_null.md) parent becomes what
[`basic_json::operator[]`](../basic_json/operator%5B%5D.md) with a JSON pointer makes of it: an array if the last
reference token is `#!cpp "-"` or consists of digits only (for an index beyond 0, the array is first filled with
null values up to that index), an object otherwise. An empty `ptr` sets [`root()`](root.md) itself, as in 1.
In every overload, `value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any*
document -- read-only or editable, and it does not have to be `target`'s/`object`'s/`array`'s own document -- which
is copied so that nothing is shared with the source document afterward; a `BasicJsonType` value; or anything
`BasicJsonType` can be constructed from (numbers, strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
## Template parameters
`V`
: the type of `value`, deduced; see above for what is accepted.
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
## Parameters
`target` (in)
: the value to replace
`object` (in)
: the object (or null value) whose member to set
`array` (in)
: the array whose element to assign
`key` (in)
: the key of the member to set
`idx` (in)
: the index of the element to assign; a negative value throws (see [Exceptions](#exceptions))
`ptr` (in)
: a JSON pointer to the value to set, relative to `root()`
`value` (in)
: the new value
## Return value
1. a view of `target`, now holding `value`
2. a view of the member `key` of `object`, now holding `value`
3. a view of the element `idx` of `array`, now holding `value`
4. a view of the value `ptr` refers to, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while an edited array or object switches
from its parsed layout to a growable block, see [Notes](#notes) -- can still leave a partial effect, such as a
[null](../basic_json_view/is_null.md) `object`/`array` argument already turned into an empty object/array even
though `value` itself was not linked in. Likewise, a failure of `value` in 4. leaves a null parent that is set with an
index beyond 0 already filled with the null values before the index.
## Exceptions
1. Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view, or a [discarded](../basic_json/is_discarded.md)
`BasicJsonType` value (e.g. `#!cpp BasicJsonType(value_t::discarded)`) -- an object or array `value`, of either
kind, is fine and is encoded as a whole subtree.
2. Throws [`type_error.305`](../../home/exceptions.md#jsonexceptiontype_error305) if `object` is neither an object
nor null -- the same message [`operator[]`](../basic_json_view/operator%5B%5D.md) throws for a string argument on
such a value. Throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `key` is not
valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
Also throws what 1. throws for `value`.
3. Throws `type_error.305` if `array` is not an array -- the same message `operator[]` throws for a numeric argument
on such a value. Throws [`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is
negative, or if `#!cpp idx >= array.size()`. Also throws what 1. throws for `value`.
4. Throws what [`at`](../basic_json_view/at.md) throws (overload 3) for resolving `ptr`'s parent, except that a
missing object member or an array index equal to the array's size at the very last reference token is not an
error there (it becomes a new member or an appended element) instead of
[`out_of_range.403`](../../home/exceptions.md#jsonexceptionout_of_range403)/[`out_of_range.402`](../../home/exceptions.md#jsonexceptionout_of_range402).
For the last reference token itself: if the parent is an object (or a primitive value, where it throws
`type_error.305`), throws what 2. throws; if the parent is an array, throws what 3. throws for an index that is
out of range, or, for a token that is not a valid array index,
[`parse_error.106`](../../home/exceptions.md#jsonexceptionparse_error106) (a leading `#!cpp '0'`),
[`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) (not a number),
[`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) (too large for `size_type`), or
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) (an empty token); the same errors are
thrown for a null parent and a token of digits (the parent is not changed then), and
[`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if the index is 4294967295 or more.
Also throws what 1. throws for `value`.
Every overload also throws [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or
contains) a binary value -- `BasicJsonType` can hold one, but a `json_document` cannot -- and
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `target`/`object`/`array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a
*different* document (overloads 1-3 only; overload 4 always starts from this document's own [`root()`](root.md)).
## Complexity
1. Linear in the size of `value` (encoding it into the document's storage): constant for a scalar, linear in the
number of nested values for an array or object. If `target` is itself an array or object that spans more than one
node in its parent's original, unedited layout, and `value` is a scalar, replacing it additionally costs time
linear in the size of the document, the *first* time (the parent of `target` is looked up from
[`root()`](root.md), and then switches to links) -- see [Notes](#notes). Once the parent has links, `target` is
replaced in constant time: setting every element of a large array one after the other is linear overall. To avoid
the lookup altogether, use 3. (or 2. for an object), which know the parent.
2. Linear in the number of members of `object`, to find an existing member with `key`, plus the complexity of 1. for
`value`.
3. Constant, plus the complexity of 1. for `value`.
4. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array (as [`at`](../basic_json_view/at.md)), plus the complexity of 2. or 3. for
the last token; for a null parent and an index, linear in the index.
## Notes
!!! info "Duplicate keys"
If `object` already has more than one member with `key` (2.), the *first* one is assigned `value` and every
later member with the same key is removed -- so that a lookup, an iteration, and
[`materialize()`](../basic_json_view/materialize.md) of `object` afterward all agree on a single value for
`key`, the same way [`operator[]`](../basic_json_view/operator%5B%5D.md) already picks the first occurrence of a
duplicate key for reading. See the [Notes on duplicate keys](../basic_json_view/operator%5B%5D.md#notes) of
`operator[]`.
Setting a member (2.) or an element (3., through 4.) of an array or object whose elements have not been edited
before switches it from its parsed layout to a growable block holding links to its elements; a later
[`push_back`](push_back.md) or `set` on the same container reuses that block, growing it (amortized constant time)
only once it runs out of room. This never moves an element itself -- only where the container's *links* to its
elements live -- so a view of an element stays valid, but any iterator already taken over the container is
invalidated, since it was walking the old layout. See [Edits](index.md#edits) for what stays valid across an edit in
general.
The same switch happens, for the same reason, when overload 1. replaces a multi-node array/object value with a
scalar: the *parent's* element sequence is what has to switch to links, not `target` itself, because the parent
originally stepped over `target`'s whole subtree by its node count, which no longer applies once `target` is a
one-node scalar.
## Examples
??? example "Example: (1)/(2)/(3)/(4) replace a value, set a member, assign an element, set via a JSON pointer"
The example below edits a small configuration document -- replacing a value, adding an object member, assigning
an array element, and reaching a field through a JSON pointer -- and shows what
[`dump()`](../basic_json_view/dump.md) preserves that is lost once the same edits are made on a `BasicJsonType`
value instead: the order object members were written in, and the exact spelling of a number that was never
touched.
```cpp
--8<-- "examples/basic_json_document__set.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__set.output"
```
## See also
- [push_back](push_back.md) - append to an array
- [insert](insert.md) - insert an element into an array
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [root](root.md) - the view of the root value, the starting point of overload 4
- [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document, keeping an untouched number's
spelling with `#!cpp number_format::source`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
- [Editing a document](../../features/json_view.md#editing-a-document) - why editable documents keep the source
text's order and number spelling
## Version history
- Added in version 3.13.0.