* Make unflatten() linear in the length of the keys
unflatten() decides in a first pass which prefixes of the keys denote arrays
(those that have a reference token 0 among their children), and looks up
prefixes again in the second pass. A prefix was identified by hashing it, and
by comparing it byte by byte with the entry in the set, from its beginning.
A key with n tokens therefore cost n^2/2 bytes: 100,000 tokens "/0" took
2.4 s (3.12.0: 5 ms), and so did "/0" followed by "/1" tokens, because every
new level looks its prefix up. Keys in the order of an unsorted object type
(ordered_json) defeated the shortcut for keys that share a beginning with the
previous one: four keys were enough to be quadratic.
Prefixes of less than 64 bytes are kept in a hash table and compared as
before, which costs a bounded number of bytes. Longer prefixes are kept in a
tree whose nodes are looked up by their parent and the text of one token. The
node of a prefix is found by walking the tokens of the key from the root, so
all prefixes of a key together cost one pass over the key, and no prefix is
compared as a whole. The nodes along the key walked last are kept, so keys
that share a beginning do not walk it again. The array/object decision and
the independence from the order of the keys are unchanged.
The hash function is seeded with a value derived from the address of a local
variable, which address space layout randomization changes between
processes, so keys cannot be crafted to collide in advance: with the fixed
constants, 20,000 keys of 16 + 2 bytes with the same hash took 0.5 s (random
keys, and 3.12.0: 7 ms). Where addresses are not randomized, or when the seed
is known, k short prefixes can still collide and cost k^2 / 2 comparisons of
fewer than 64 bytes. Results do not depend on the seed.
Measurements (Apple clang -O2, arm64), unflatten() of one key with n tokens:
tokens "/0" n=25k n=50k n=100k n=200k
before 175 ms 617 ms 2437 ms 9729 ms
after 4 ms 6 ms 12 ms 27 ms
3.12.0 1 ms 3 ms 5 ms 10 ms
"/0" then "/1" n=25k n=50k n=100k n=200k
before 153 ms 603 ms 2436 ms 9678 ms
after 1 ms 3 ms 5 ms 10 ms
3.12.0 1 ms 3 ms 5 ms 11 ms
ordered_json, 4 keys /0*m/a0, /10, /0*m/a1, /11
m=25k m=50k m=100k m=200k
before 397 ms 1402 ms 5191 ms 20723 ms
after 4 ms 7 ms 15 ms 33 ms
3.12.0 2 ms 3 ms 8 ms 17 ms
Ordinary documents (flatten() then unflatten(), instructions retired): within
0.3 % of before for canada, citm_catalog, twitter, jeopardy, mesh,
update-center and random from the benchmark corpus.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix -Wuseless-cast in array_prefix_set::reseed
uintptr_t is already uint64_t on LP64 Linux, so the explicit cast is
useless there; rely on the implicit (widening) conversion instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Throw type_error.313 when an unflatten key is a prefix of an earlier key
With an object type that keeps the insertion order (ordered_json), a key
such as "/a" can come after "/a/b". The final assignment then replaced the
object built for "/a/b" without an error, so {"/a/b":2,"/a":1} gave
{"a":1}. json never saw this because std::map visits the shorter key first.
Throw type_error.313 if the target of the final assignment is no longer
null, as json does for the same members. A null value keeps the container,
so {"/a/b":2,"/a":null} gives {"a":{"b":2}} for both object types.
Fixes #6105.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
4.7 KiB
nlohmann::basic_json::unflatten
basic_json unflatten() const;
The function restores the arbitrary nesting of a JSON value that has been flattened before using the
flatten() function. The JSON value must meet certain constraints:
- The value must be an object.
- The keys must be JSON pointers (see RFC 6901)
- The mapped values must be primitive JSON types.
Return value
the original JSON from a flattened version
Exception safety
Strong exception safety: if an exception occurs, the original value stays intact.
Exceptions
The function can throw the following exceptions:
- Throws
type_error.314if value is not an object - Throws
type_error.315if object values are not primitive - Throws
type_error.313if a key (JSON pointer) leads to a conflicting nesting, for example if a key with a value other than#!json nullis a prefix of another key (as/aand/a/b), in whichever order the keys come; example:"invalid value to unflatten" - Throws
parse_error.106if an array index in a key begins with '0'; example:"array index '01' must not begin with '0'" - Throws
parse_error.107if a key is not empty and does not begin with a slash (/); example:"JSON pointer must be empty or begin with '/' - was: 'a'" - Throws
parse_error.108if a tilde (~) in a key is not followed by0or1; example:"escape character '~' must be followed with '0' or '1'" - Throws
parse_error.109if an array index in a key is not a number; example:"array index 'one' is not a number" - Throws
out_of_range.404if a level becomes an array (because one of its keys is0) and another key at that level is empty (as in/a/), is-, or begins with a digit but is not a valid array index (such as1a); examples:"unresolved reference token '-'","unresolved reference token ''"
Complexity
Linear in the size of the JSON value.
Notes
Empty objects and arrays are flattened by flatten() to #!json null values and cannot be unflattened to
their original type.
A flattened array and a flattened object whose keys are array indices are indistinguishable, because both are
described by the same JSON pointers. A value is therefore restored as an array if and only if one of its keys is the
reference token 0, and as an object otherwise: #!json {"2": 1} is restored unchanged, whereas #!json {"0": 1} is
restored as #!json [1]. This decision does not depend on the order in which the flattened object is iterated.
A third case is an object that has the key 0 next to a key that is not a valid array index: #!json {"0": 1, "b": 2}
is flattened to #!json {"/0": 1, "/b": 2}, and unflatten treats the level as an array and throws
parse_error.109 because b is not a number. (For the keys
-, 1a, or the empty key instead of b, it throws out_of_range.404.)
Apart from these three cases, for a JSON value j, the following is always true:
#!cpp j == j.flatten().unflatten().
Examples
??? example
The following code shows how a flattened JSON object is unflattened into the original nested JSON object.
```cpp
--8<-- "examples/unflatten.cpp"
```
Output:
```json
--8<-- "examples/unflatten.output"
```
See also
flatten- the reverse function- JSON Pointer - the article on JSON Pointer and flattening
Version history
- Added in version 2.0.0.
- Made the array/object decision independent of the object's iteration order in version 3.13.0.
- Throws
parse_error.109instead ofout_of_range.404for a one-character array index that is not a digit (e.g.,/x) in version 3.13.0, as it already did for longer ones. - Throws
type_error.313for a key that is a prefix of an earlier key in version 3.13.0; before, an object type that keeps the insertion order (such asordered_json) silently replaced the nested value.