Files
json/docs/mkdocs/docs/api/basic_json/unflatten.md
T
Niels Lohmann 99adff001d Make unflatten() linear in the length of the keys (#6076)
* Make unflatten() linear in the length of the keys

unflatten() decides in a first pass which prefixes of the keys denote arrays
(those that have a reference token 0 among their children), and looks up
prefixes again in the second pass. A prefix was identified by hashing it, and
by comparing it byte by byte with the entry in the set, from its beginning.
A key with n tokens therefore cost n^2/2 bytes: 100,000 tokens "/0" took
2.4 s (3.12.0: 5 ms), and so did "/0" followed by "/1" tokens, because every
new level looks its prefix up. Keys in the order of an unsorted object type
(ordered_json) defeated the shortcut for keys that share a beginning with the
previous one: four keys were enough to be quadratic.

Prefixes of less than 64 bytes are kept in a hash table and compared as
before, which costs a bounded number of bytes. Longer prefixes are kept in a
tree whose nodes are looked up by their parent and the text of one token. The
node of a prefix is found by walking the tokens of the key from the root, so
all prefixes of a key together cost one pass over the key, and no prefix is
compared as a whole. The nodes along the key walked last are kept, so keys
that share a beginning do not walk it again. The array/object decision and
the independence from the order of the keys are unchanged.

The hash function is seeded with a value derived from the address of a local
variable, which address space layout randomization changes between
processes, so keys cannot be crafted to collide in advance: with the fixed
constants, 20,000 keys of 16 + 2 bytes with the same hash took 0.5 s (random
keys, and 3.12.0: 7 ms). Where addresses are not randomized, or when the seed
is known, k short prefixes can still collide and cost k^2 / 2 comparisons of
fewer than 64 bytes. Results do not depend on the seed.

Measurements (Apple clang -O2, arm64), unflatten() of one key with n tokens:

  tokens "/0"          n=25k  n=50k  n=100k  n=200k
  before              175 ms 617 ms 2437 ms 9729 ms
  after                 4 ms   6 ms   12 ms   27 ms
  3.12.0                1 ms   3 ms    5 ms   10 ms

  "/0" then "/1"       n=25k  n=50k  n=100k  n=200k
  before              153 ms 603 ms 2436 ms 9678 ms
  after                 1 ms   3 ms    5 ms   10 ms
  3.12.0                1 ms   3 ms    5 ms   11 ms

  ordered_json, 4 keys /0*m/a0, /10, /0*m/a1, /11
                       m=25k  m=50k  m=100k  m=200k
  before              397 ms 1402 ms 5191 ms 20723 ms
  after                 4 ms   7 ms   15 ms   33 ms
  3.12.0                2 ms   3 ms    8 ms   17 ms

Ordinary documents (flatten() then unflatten(), instructions retired): within
0.3 % of before for canada, citm_catalog, twitter, jeopardy, mesh,
update-center and random from the benchmark corpus.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix -Wuseless-cast in array_prefix_set::reseed

uintptr_t is already uint64_t on LP64 Linux, so the explicit cast is
useless there; rely on the implicit (widening) conversion instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Throw type_error.313 when an unflatten key is a prefix of an earlier key

With an object type that keeps the insertion order (ordered_json), a key
such as "/a" can come after "/a/b". The final assignment then replaced the
object built for "/a/b" without an error, so {"/a/b":2,"/a":1} gave
{"a":1}. json never saw this because std::map visits the shorter key first.

Throw type_error.313 if the target of the final assignment is no longer
null, as json does for the same members. A null value keeps the container,
so {"/a/b":2,"/a":null} gives {"a":{"b":2}} for both object types.

Fixes #6105.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-15 17:02:29 +01:00

4.7 KiB

nlohmann::basic_json::unflatten

basic_json unflatten() const;

The function restores the arbitrary nesting of a JSON value that has been flattened before using the flatten() function. The JSON value must meet certain constraints:

  1. The value must be an object.
  2. The keys must be JSON pointers (see RFC 6901)
  3. The mapped values must be primitive JSON types.

Return value

the original JSON from a flattened version

Exception safety

Strong exception safety: if an exception occurs, the original value stays intact.

Exceptions

The function can throw the following exceptions:

  • Throws type_error.314 if value is not an object
  • Throws type_error.315 if object values are not primitive
  • Throws type_error.313 if a key (JSON pointer) leads to a conflicting nesting, for example if a key with a value other than #!json null is a prefix of another key (as /a and /a/b), in whichever order the keys come; example: "invalid value to unflatten"
  • Throws parse_error.106 if an array index in a key begins with '0'; example: "array index '01' must not begin with '0'"
  • Throws parse_error.107 if a key is not empty and does not begin with a slash (/); example: "JSON pointer must be empty or begin with '/' - was: 'a'"
  • Throws parse_error.108 if a tilde (~) in a key is not followed by 0 or 1; example: "escape character '~' must be followed with '0' or '1'"
  • Throws parse_error.109 if an array index in a key is not a number; example: "array index 'one' is not a number"
  • Throws out_of_range.404 if a level becomes an array (because one of its keys is 0) and another key at that level is empty (as in /a/), is -, or begins with a digit but is not a valid array index (such as 1a); examples: "unresolved reference token '-'", "unresolved reference token ''"

Complexity

Linear in the size of the JSON value.

Notes

Empty objects and arrays are flattened by flatten() to #!json null values and cannot be unflattened to their original type.

A flattened array and a flattened object whose keys are array indices are indistinguishable, because both are described by the same JSON pointers. A value is therefore restored as an array if and only if one of its keys is the reference token 0, and as an object otherwise: #!json {"2": 1} is restored unchanged, whereas #!json {"0": 1} is restored as #!json [1]. This decision does not depend on the order in which the flattened object is iterated.

A third case is an object that has the key 0 next to a key that is not a valid array index: #!json {"0": 1, "b": 2} is flattened to #!json {"/0": 1, "/b": 2}, and unflatten treats the level as an array and throws parse_error.109 because b is not a number. (For the keys -, 1a, or the empty key instead of b, it throws out_of_range.404.)

Apart from these three cases, for a JSON value j, the following is always true: #!cpp j == j.flatten().unflatten().

Examples

??? example

The following code shows how a flattened JSON object is unflattened into the original nested JSON object.

```cpp
--8<-- "examples/unflatten.cpp"
```

Output:

```json
--8<-- "examples/unflatten.output"
```

See also

Version history

  • Added in version 2.0.0.
  • Made the array/object decision independent of the object's iteration order in version 3.13.0.
  • Throws parse_error.109 instead of out_of_range.404 for a one-character array index that is not a digit (e.g., /x) in version 3.13.0, as it already did for longer ones.
  • Throws type_error.313 for a key that is a prefix of an earlier key in version 3.13.0; before, an object type that keeps the insertion order (such as ordered_json) silently replaced the nested value.