Compare commits

..
Author SHA1 Message Date
Niels Lohmann 23f999547e Mark the parameters of the binary writers' throw helpers as used
With JSON_NOEXCEPTION, JSON_THROW() drops its argument, so the
parameters that only build the message were unused, which Clang's
-Wunused-parameter turned into an error in test-disabled_exceptions.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 11:45:57 +02:00
Niels Lohmann adceb69a77 Test that the writers' loops for arrays and objects keep the range checks
Runs of numbers and scalar object members are encoded by loops of their own
for the number types up to 64 bits. A number type wider than that has to take
the generic path, which throws out_of_range.407 for an integer that does not
fit. The test writes arrays and objects with such an integer in every format,
checks the partial output of a failing CBOR and BON8 write, and compares the
bytes of the values that do fit with those of the 64-bit number types.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:39 +02:00
Niels Lohmann b0a7d5f6ff Test that number runs, string heads and adapter flushing keep the output
The writers encode a run of numbers in an array, and a string with its head,
straight into space reserved from the sink, and the adapter sink buffers its
writes. These tests pin the output to what it was: an array is its head and
the encodings of its elements one by one for every boundary of the integer
encodings and every kind of float, strings and keys of every head length and
large arrays read back, and a write that throws leaves the same bytes in a
vector, a string and a stream.

The tests pass with and without the changes of this series.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:36 +02:00
Niels Lohmann 70fdb359f0 Describe the sinks of the binary writer in the architecture page
The page listed the output adapters only. The binary writer writes through a
sink: a non-virtual one that appends to the vector of the vector-returning
to_* functions, and one that collects the output for the adapters of the other
overloads and passes it on in blocks.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:34 +02:00
Niels Lohmann 7717853976 Write the digits of a high-precision UBJSON number without basic_json
An unsigned integer above the int64 range is a high-precision number in
UBJSON: its decimal digits as a string. They were found by building a
basic_json from the integer and calling dump() on it, an allocation and a
serializer for 20 digits.

An unsigned integer of at most 64 bits now writes its digits from a small
buffer. Other integer types (wider than 64 bits) keep dump().

Instructions per call, 3000 integers above the int64 range (Linux, GCC 13, thousands): to_ubjson 3968 -> 1825, to_bjdata 180 -> 180. (to_bjdata writes these as uint64 and is unchanged.)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:31 +02:00
Niels Lohmann bc94203dbc Throw, and write high-precision numbers, from functions of their own
Functions that check something that almost never fails kept the code that
builds the exception in their own body: check_utf8() of BON8 (a std::string
for the message), the NUL check of a BSON key, the length checks of
MessagePack and BSON, the subtype checks, the size marker check of UBJSON, the
integer range check of BON8, and the high-precision number of UBJSON (a
basic_json and a string, for every integer that picks the marker `H`). That
gave them a frame with a stack protector under GCC, and made the compiler
spill registers for a path that is not taken.

Each of these is now thrown, or written, by a function that is not inlined.
The messages and exception ids are the same.

Instructions per call (Linux, GCC 13, thousands), to_bson: 20000 small objects 31714 -> 29614, twitter 4930 -> 4637; to_ubjson: citm_catalog 7020 -> 6597; to_bjdata: citm_catalog 7192 -> 6789; to_bon8: citm_catalog 7107 -> 6790.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:29 +02:00
Niels Lohmann b85d1ca043 Write the one-byte head of an array or object with a single sink write
The head of an array or object of up to 23 elements (15 for MessagePack) is a
single byte, but it was written through write_cbor_head()/write_number(),
which GCC keeps out of line, and a std::array.

The CBOR array and object writers now write that byte directly, and
MessagePack does the same in its array and map prefixes.

Instructions per call (Linux, GCC 13, thousands), to_cbor: citm_catalog 5588 -> 5020, canada 15840 -> 14499, 20000 small objects 13162 -> 12582; to_msgpack: citm_catalog 5920 -> 5676, canada 16344 -> 15676.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:26 +02:00
Niels Lohmann 26b4716740 Write the count of a UBJSON or BJData container in one reservation
With use_size, a container is announced by the marker `#` and its element
count as an integer, three small sink writes for the many containers of up to
127 elements (an empty array, an object with a few members).

A count of up to 127 is now written as `#`, `i` and one byte in a single
reservation.

Instructions per call (Linux, GCC 13, thousands), to_ubjson with use_size and use_type: canada 25813 -> 22060, citm_catalog 10656 -> 9192, 20000 small objects 18178 -> 16798; to_bjdata with use_size and use_type: canada 26341 -> 22588, citm_catalog 10904 -> 9440.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:24 +02:00
Niels Lohmann a3b9e9c712 Write the members of a BON8 object with scalar values without a call each
As for the other formats: a member with a null, boolean, number, or string
value is written by the loop over the members, without a call of
write_bon8_value() for its value. A number above the int64 range, which BON8
cannot write (also with a number type wider than 64 bits), still goes through
write_bon8_value(), which throws as before.

Instructions per call (Linux, GCC 13, thousands), to_bon8: 20000 small objects 20741 -> 18164, twitter 5526 -> 5123, citm_catalog 7335 -> 7107.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:21 +02:00
Niels Lohmann 985fa5cb2e Keep a cursor and an end pointer in the vector sink
The vector sink kept the vector and a position, so that every write or
reservation read the vector's begin and end, subtracted them and the position,
and added the position to the data pointer again for the store.

It now keeps the write pointer and the end of the vector. A capacity check is
a subtraction of two members, and the pointer is the one to store to. Both are
set again whenever the vector grows; the destructor still trims the vector to
what was written, also when a write throws.

Instructions per call (Linux, GCC 13, thousands), to_bon8: 20000 small objects 23279 -> 20741, citm_catalog 7974 -> 7335; to_msgpack: 20000 small objects 13641 -> 12702; to_ubjson: 20000 small objects 14620 -> 13620; to_bjdata: 20000 small objects 14641 -> 13641.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:19 +02:00
Niels Lohmann f41f6408eb Write a MessagePack float that is not in an array of numbers into reserved space
A float that was not part of a run in an array (an object member, a float in a
mixed array) went through write_compact_float(), with a sink call for its
marker and one through a std::array for its bytes.

It is now encoded by the function that encodes the floats of a run, into space
reserved from the sink.

CBOR keeps the previous code for these floats: giving them the same treatment
made GCC 13 stop inlining the encoder into the loop for the floats of an array,
which cost 4 percent more instructions for canada and citm_catalog.

Instructions per call (Linux, GCC 13, thousands), to_msgpack: 20000 small objects 14601 -> 13641.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:16 +02:00
Niels Lohmann ff2d59d3f1 Write the elements of a BJData ND-array in blocks
An ND-array annotation object is checked and then written from its _ArrayData_
in three passes. Every element went through basic_json::get<>() twice or three
times (the range check of the dry run, the conversion for the write) and
through write_number() with a std::array and a sink call of its own, with a
switch on the number kind for each.

The range check and the conversion read the value directly (the elements are
known to be integers or floats by then), and the real pass switches on the
number kind once and encodes its elements in blocks into space reserved from
the sink.

to_bjdata into a std::vector, instructions per call (Linux, GCC 13, thousands): an ND-array of 120000 doubles 14588 -> 9975, an ND-array of 120000 uint8 16583 -> 8790, an ND-array of 120000 doubles with use_size and use_type 14630 -> 10017.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:14 +02:00
Niels Lohmann ceb3c92709 Reserve at least 64 bytes for the result of a binary serialization
The result of to_cbor() and its siblings reserved the lower bound of
binary_reserve_hint(), for a small object a few bytes, and the first write of
the sink replaced it by an allocation of 64 bytes, as an empty or too small
vector is grown to at least that. A small document therefore cost two
allocations and a free.

The result now reserves at least min_vector_sink_capacity (64) characters at
once. binary_reserve_hint() itself, a lower bound that is tested as such, is
unchanged.

Instructions to serialize a small object (six members, a nested array and object), Linux, GCC 13: to_cbor 1700 -> 1500, to_msgpack 1800 -> 1600, to_ubjson 1800 -> 1700, to_bson 2700 -> 2500. Documents larger than 64 bytes of hint are unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:11 +02:00
Niels Lohmann 7241ba94c8 Encode runs of numbers in BON8 arrays without a call each
A number in a BON8 array went through write_bon8_value(), the ladder of
write_bon8_integer() and one sink write per byte of its encoding (through a
std::array for each).

An array that starts with a number now encodes its runs of integers and floats
straight into space reserved from the sink, as the other formats do. An
unsigned integer above the int64 range, which BON8 cannot write, is passed to
write_bon8_value(), which throws exactly as before after the same bytes. So
is an integer of a number type wider than 64 bits that does not fit.

Instructions per call (Linux, GCC 13, thousands), to_bon8: 20000 integers 2375 -> 1002, 20000 floats 3173 -> 1607, canada 22130 -> 15558.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:09 +02:00
Niels Lohmann f9e2302af9 Write the members of an object with scalar values without a call each
Every member of an object was a call of write_cbor()/write_msgpack()/
write_ubjson() for its value, however simple: a function with a large frame
that saves and restores many registers around a single sink write.

Objects are now written by a function of their own, like arrays, in which
members with a null, boolean, integer, or string value (and a float, in UBJSON
and BJData) are written in place; anything else is passed on to the general
function. The depth limit and the recursion for nested values are unchanged.

Instructions per call (Linux, GCC 13, thousands), to_ubjson: 20000 small objects 17861 -> 14661, twitter 3271 -> 2798; to_bjdata: 20000 small objects 17881 -> 14681, twitter 3277 -> 2804; to_msgpack: twitter 2948 -> 2646.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:06 +02:00
Niels Lohmann eb251df17c Size and write BSON arrays of numbers without a frame on the stack
Every array nested in a BSON document pushed a frame of about 100 bytes on a
vector, replaced the current one by moving, and popped it again, once for the
size pass and once for the write pass. For the many small arrays of numbers
(coordinates, vectors) that was more than the work on their entries.

An array whose elements are all booleans, numbers, or null is now sized by a
loop of its own, and written in the loop for fixed-size entries on a frame
that is not pushed. The entries of other arrays go to the fixed-size loop only
when the element is such a value, which also saves the call for an array of
strings. An array with an integer that does not fit 64 bits (possible with
number types wider than that) is left to the generic path, which throws.

Instructions per call (Linux, GCC 13, thousands), to_bson: canada 46833 -> 30575, 20000 integers 2680 -> 1839, 20000 floats 3050 -> 2209, citm_catalog 14536 -> 13235, 20000 small objects 33294 -> 31954.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:04 +02:00
Niels Lohmann dd3ed1ef86 Check and sanitize strings out of line
The string writers kept the string_t that holds a sanitized copy for the other
error handlers in the function that also has the fast path for the default
one, so that every string paid for its construction check, a stack frame with
a stack protector under GCC, and the registers saved for the slow path.

The slow path is now a function of its own that the fast path does not call.

Instructions per call (Linux, GCC 13, thousands), to_ubjson: twitter 3867 -> 3274, 20000 small objects 20981 -> 17901; to_bjdata: twitter 3873 -> 3281, 20000 small objects 21001 -> 17921; to_msgpack: 20000 small objects 18342 -> 15922.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:29:01 +02:00
Niels Lohmann a517b8ea8b Write a UBJSON, BJData or BSON string and its head in one reservation
Same as for CBOR and MessagePack: with the default error handler (keep) a
string or object key of up to 253 characters (UBJSON, BJData: the `S` marker,
the length as `i` or `U` and one byte, and the characters), or a BSON string
entry that fits the reservation limit (type, name, length, characters and null
byte), is written into space reserved from the sink in one go, instead of by
four to six sink writes, a string_t for the error handler, and a call of
sanitize_utf8_for_write(). The name of a BSON entry is copied with the short-
string copy of the sinks.

Instructions per call (Linux, GCC 13, thousands), to_ubjson: 20000 small objects 28241 -> 20981, twitter 5175 -> 3867, citm_catalog 10681 -> 8857; to_bjdata: 20000 small objects 28261 -> 21001, twitter 5181 -> 3873.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:28:59 +02:00
Niels Lohmann ecc2f511c7 Write a CBOR or MessagePack string with its head in one reservation
A string or object key took a string_t constructed for the UTF-8 error
handler, a call of the (out of line) sanitize_utf8_for_write() that returns
the string unchanged for the default handler, a call that builds the head from
a range-checked length and writes it with a std::array and a sink call of its
own, and a sink write for the characters, which copied it with a call of
memcpy() through a stub with its own dispatch on the length.

With the default error handler (keep) and a string of up to 254 characters,
the head and the characters are now written into space reserved from the sink
in one go. Longer strings and the other handlers take the previous code. The
sinks copy up to 32 characters with fixed-size loads and stores - two possibly
overlapping 16, 8 or 4 byte copies, or the first, middle and last character of
a string of 1 to 3 - instead of calling memcpy(); nothing outside of the
source or destination range is touched.

Instructions per call (Linux, GCC 13, thousands), to_msgpack: twitter 4414 -> 3383, 20000 small objects 23895 -> 18342, citm_catalog 8575 -> 7031; to_cbor: 20000 small objects 20695 -> 17182, twitter 3842 -> 3204.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:28:56 +02:00
Niels Lohmann 4ee05e4d1f Encode runs of floats in CBOR and MessagePack arrays without a call each
Integers in an array were already encoded in runs; a float went through
write_cbor()/write_msgpack(), the NaN/infinity checks, write_compact_float()
and, for the marker and the 4 or 8 bytes of the number, two sink writes, one
of them through a std::array that was reversed and copied.

A run of floats is now encoded straight into space reserved from the sink,
like a run of integers, by a function of its own so that the loop of integers
is unchanged. The float32 check is shared with write_compact_float(). Arrays
that are empty are returned from before the elements loop, which also saves
its prologue for the many empty arrays of documents like citm_catalog.json.

Instructions per call (Linux, GCC 13, thousands), to_cbor: 20000 floats 2753 -> 1202, canada 21529 -> 16232; to_msgpack: 20000 floats 2633 -> 1202, canada 21595 -> 16791.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:28:54 +02:00
Niels Lohmann 38536b6b4d Write the bytes of an unoptimized UBJSON and BON8 binary value in blocks
Without use_type, a binary value is an array of its bytes, and was written one
sink call per marker and per byte (UBJSON, BJData), or through the integer
ladder for each byte (BON8).

The bytes are now encoded in blocks into space reserved from the sink, by a
function that is not inlined. BON8 stores both bytes of an integer above 39
and keeps the second only if it is needed, so that the code does not branch on
the value.

Instructions per call for a 1 MB binary value (Linux, GCC 13, thousands): to_ubjson 34253 -> 9521, to_bjdata 34253 -> 9521, to_bon8 53088 -> 17333.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:28:51 +02:00
Niels Lohmann 7d3412a559 Encode runs of numbers in UBJSON and BJData arrays without a call each
An array of numbers was written one write_ubjson() call per element, each
picking the marker of an integer by a ladder of range checks and writing it
with a switch and several small sink writes.

An array that starts with a number now encodes its runs of numbers straight
into space reserved from the sink, as the CBOR and MessagePack writers do for
integers. The marker and size of an integer come from a table indexed by the
bit length of its magnitude, and its bytes are stored as one word, so that the
code does not branch on their size; floats are stored the same way. Arrays
that do not start with a number, and a single integer, keep the previous code.
An unsigned integer UBJSON can only write as a high-precision number is left
to write_ubjson().

Instructions per call (Linux, GCC 13, thousands), to_bjdata: 20000 integers 3278 -> 1086, 20000 floats 2193 -> 1000, canada 20829 -> 16805; to_ubjson: 20000 integers 3282 -> 1246, 20000 floats 2213 -> 1020.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:28:49 +02:00
Niels Lohmann e82ee7375a Collect the writes of the adapter sink and pass them on in blocks
to_cbor(j, o) and its siblings with a std::string, a std::vector or a
std::ostream go through output_adapter_sink, where every write is a virtual
call, and for a stream a call with a sentry on top. That costs more than
encoding the few bytes of most values, which is why those overloads were 1.8
to 3 times slower than the vector-returning ones.

The sink now keeps a 1 KiB buffer and passes it on when it is full, when a
write is larger than the space left (a payload of a KiB or more goes
directly), or when binary_writer::flush() is called at the end of the to_*
function. Everything reaches the adapter in the same order. The destructor
flushes what is left, so a value that throws (invalid UTF-8 with the strict
handler) leaves the same partial output as before; only flush() can report an
error of the adapter, as a destructor must not throw.

Instructions per call (Linux, GCC 13, thousands), to_*(j, std::string&): twitter cbor 4952 -> 3356, twitter ubjson 6762 -> 4647, twitter bson 5518 -> 4704, citm_catalog cbor 10404 -> 7090; to_*(j, std::ostream&): twitter cbor 9178 -> 3844, twitter ubjson 11974 -> 5129, citm_catalog cbor 18374 -> 7554.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:28:46 +02:00
Niels Lohmann ef2fee20d8 Write BSON entries into space reserved from the sink
An entry was written as up to five separate sink writes (type, name, null
byte, then the number as a std::array that was reversed and copied) and, in an
array, from a string_t built for the index. The size pass looked for U+0000 in
every key with std::string::find() (a call that calls memchr()), and built a
string_t and called sanitize_utf8_for_write() for the default error handler,
which returns the key unchanged.

Entries of booleans, numbers and null are now encoded straight into space
reserved from the sink, as the CBOR and MessagePack writers do for integer
runs. A run of such elements in an array keeps the write pointer, the index,
and the digits of its name in locals, so that nothing is reloaded after each
character store. The entry of an object member does the same for one entry,
and a string or binary value writes its header the same way. The name is
searched for U+0000 eight characters at a time, and the string_t for the other
error handlers is built in a function of its own.

Instructions per call (Linux, GCC 13, thousands), to_bson: 20000 integers 6486 -> 2680, 20000 floats 6562 -> 3050, canada 55282 -> 47186, twitter 5865 -> 5140, 20000 small objects 37171 -> 33559.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:28:44 +02:00
Niels Lohmann 135095d692 Name BSON array elements without a std::string per element
BSON stores an array as a document whose keys are the decimal indices. Both
passes built that key with std::to_string() and copied it into a string_t for
every element, and the size pass then validated it like an object key (a
search for U+0000 and the UTF-8 handler), though a decimal number cannot fail
either.

The index name now lives in the array's frame as digits that are incremented
in place, and the size pass adds 2 + the number of digits.

Instructions per call (Linux, GCC 13, thousands), to_bson: 20000 integers 16228 -> 6486, 20000 floats 16304 -> 6562, canada 109155 -> 55282, 20000 small objects 45513 -> 37171, citm_catalog 17269 -> 15026.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:28:41 +02:00
23 changed files with 4441 additions and 1714 deletions

No files matched your search

+1 -2
View File
@@ -80,8 +80,7 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access). Constant if no element of the array is an array
or object, since the elements are then adjacent in the index.
index (unlike `BasicJsonType`'s array, which is random-access).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level (as 1.) or the index into the array (as 2.).
+1 -2
View File
@@ -28,8 +28,7 @@ This exception does not carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics.
Linear in the size of the array or object: unlike [`front()`](front.md), which only ever looks at the first
element, `back()` has to walk every element to find where they end, since elements are not a fixed size in the
index. Constant for an array none of whose elements is an array or object, since the elements are then adjacent
in the index.
index.
## Notes
@@ -86,8 +86,7 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access). Constant if no element of the array is an array
or object, since the elements are then adjacent in the index.
index (unlike `BasicJsonType`'s array, which is random-access).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level (as 1.) or the index into the array (as 2.).
+5
View File
@@ -339,6 +339,11 @@ The `serializer` (used by [`dump`](../api/basic_json/dump.md) and [`operator<<`]
- `output_stream_adapter` writes to a `std::ostream`.
- `output_string_adapter` appends to a string.
The `binary_writer` does not call an adapter for every byte. The `to_*` functions that return a `std::vector` write
into the vector through a sink (`output_vector_sink`) that is not virtual. The functions that take an output adapter
write through a sink (`output_adapter_sink`) that collects the output in a buffer of one kibibyte and passes it to the
adapter in blocks, so that the adapter is called (and a stream locked) once per block instead of once per value.
## Value conversion
Values are converted from and to other types with the `JSONSerializer` template parameter. The default,
File diff suppressed because it is too large. Load diff
@@ -55,6 +55,66 @@ using output_adapter_t = std::shared_ptr<output_adapter_protocol<CharType>>;
/// output_vector_sink also appends writes of at least this size with insert()
constexpr std::size_t max_reserve_size = 256;
/// @brief copy @a length characters, at most 32, without a call of memcpy()
///
/// Short strings are the typical payload of a binary value, and a call of
/// memcpy() costs more than copying them. A copy of 16 to 32 characters is two
/// possibly overlapping 16-byte copies, one of 8 to 15 two 8-byte copies, and
/// so on down to a single character; each is a fixed-size load and store.
/// Nothing outside of [src, src + length) is read or written.
template<typename CharType>
inline void copy_short(CharType* dst, const CharType* src, const std::size_t length, std::true_type /*is_byte*/) noexcept
{
JSON_ASSERT(length <= 32);
if (length >= 16)
{
std::array<char, 16> first; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<char, 16> last; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::memcpy(first.data(), src, 16);
std::memcpy(last.data(), src + (length - 16), 16);
std::memcpy(dst, first.data(), 16);
std::memcpy(dst + (length - 16), last.data(), 16);
}
else if (length >= 8)
{
std::array<char, 8> first; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<char, 8> last; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::memcpy(first.data(), src, 8);
std::memcpy(last.data(), src + (length - 8), 8);
std::memcpy(dst, first.data(), 8);
std::memcpy(dst + (length - 8), last.data(), 8);
}
else if (length >= 4)
{
std::array<char, 4> first; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<char, 4> last; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::memcpy(first.data(), src, 4);
std::memcpy(last.data(), src + (length - 4), 4);
std::memcpy(dst, first.data(), 4);
std::memcpy(dst + (length - 4), last.data(), 4);
}
else if (length != 0)
{
// 1 to 3: the first, the middle, and the last character
const CharType a = src[0];
const CharType b = src[length >> 1];
const CharType c = src[length - 1];
dst[0] = a;
dst[length >> 1] = b;
dst[length - 1] = c;
}
}
/// @brief copy @a length characters, of a character type wider than a byte
template<typename CharType>
inline void copy_short(CharType* dst, const CharType* src, const std::size_t length, std::false_type /*is_byte*/) noexcept
{
std::memcpy(dst, src, length * sizeof(CharType));
}
/// the number of characters an empty vector is grown to by output_vector_sink
constexpr std::size_t min_vector_sink_capacity = 64;
/// @brief non-virtual output sink writing into a std::vector
///
/// This sink is not part of the virtual output_adapter_protocol hierarchy: it is
@@ -78,7 +138,7 @@ class output_vector_sink
{
public:
explicit output_vector_sink(std::vector<CharType, AllocatorType>& vec) noexcept
: v(&vec), position(vec.size())
: v(&vec), cursor(vec.data() + vec.size()), limit(cursor)
{}
output_vector_sink(const output_vector_sink&) = delete;
@@ -86,7 +146,7 @@ class output_vector_sink
output_vector_sink& operator=(output_vector_sink&&) = delete;
output_vector_sink(output_vector_sink&& other) noexcept
: v(other.v), position(other.position)
: v(other.v), cursor(other.cursor), limit(other.limit)
{
other.v = nullptr;
}
@@ -95,7 +155,7 @@ class output_vector_sink
{
if (v != nullptr)
{
v->resize(position);
v->resize(position());
}
}
@@ -104,11 +164,11 @@ class output_vector_sink
JSON_HEDLEY_ALWAYS_INLINE
void write_character(CharType c)
{
if (JSON_HEDLEY_UNLIKELY(position == v->size()))
if (JSON_HEDLEY_UNLIKELY(cursor == limit))
{
grow(1);
}
(*v)[position++] = c;
*cursor++ = c;
}
// no JSON_HEDLEY_NON_NULL here: binary_writer legitimately passes a null
@@ -117,7 +177,7 @@ class output_vector_sink
JSON_HEDLEY_ALWAYS_INLINE
void write_characters(const CharType* s, std::size_t length)
{
if (JSON_HEDLEY_UNLIKELY(v->size() - position < length))
if (JSON_HEDLEY_UNLIKELY(space() < length))
{
if (length >= max_reserve_size)
{
@@ -126,10 +186,15 @@ class output_vector_sink
}
grow(length);
}
if (length != 0)
if (length <= 32)
{
std::memcpy(v->data() + position, s, length * sizeof(CharType));
position += length;
copy_short(cursor, s, length, std::integral_constant<bool, sizeof(CharType) == 1>());
cursor += length;
}
else
{
std::memcpy(cursor, s, length * sizeof(CharType));
cursor += length;
}
}
@@ -144,21 +209,37 @@ class output_vector_sink
CharType* reserve(const std::size_t length)
{
JSON_ASSERT(length <= max_reserve_size);
if (JSON_HEDLEY_UNLIKELY(v->size() - position < length))
if (JSON_HEDLEY_UNLIKELY(space() < length))
{
grow(length);
}
return v->data() + position;
return cursor;
}
/// keep the first @a length characters written to the space from reserve()
JSON_HEDLEY_ALWAYS_INLINE
void commit(const std::size_t length) noexcept
{
position += length;
cursor += length;
}
/// nothing to pass on: the characters are in the vector (see output_adapter_sink)
void flush() noexcept
{}
private:
/// number of characters of *v written so far
std::size_t position() const noexcept
{
return static_cast<std::size_t>(cursor - v->data());
}
/// number of characters that fit the vector behind the write position
std::size_t space() const noexcept
{
return static_cast<std::size_t>(limit - cursor);
}
/// make room for at least @a length more characters after the position;
/// kept out of line so that the write functions stay small enough for
/// GCC to inline them into the writer
@@ -166,12 +247,15 @@ class output_vector_sink
void grow(const std::size_t length)
{
// use the capacity the vector already has, else at least double it;
// never less than 64 characters, so that a small value does not
// never less than min_vector_sink_capacity, so that a small value does not
// reallocate for every few bytes
const std::size_t needed = position + length;
v->resize((std::max)(std::size_t{64}, needed <= v->capacity()
const std::size_t written = position();
const std::size_t needed = written + length;
v->resize((std::max)(min_vector_sink_capacity, needed <= v->capacity()
? v->capacity()
: (std::max)(needed, v->capacity() * 2)));
cursor = v->data() + written;
limit = v->data() + v->size();
}
/// append @a length (at least max_reserve_size) characters that do not fit
@@ -181,20 +265,24 @@ class output_vector_sink
JSON_HEDLEY_NEVER_INLINE
void append(const CharType* s, const std::size_t length)
{
v->resize(position);
if (v->capacity() - position < length)
const std::size_t written = position();
v->resize(written);
if (v->capacity() - written < length)
{
// grow geometrically, with room for the writes that follow, so
// that the next grow() need not reallocate right away
v->reserve((std::max)(v->capacity() * 2, position + length + max_reserve_size));
v->reserve((std::max)(v->capacity() * 2, written + length + max_reserve_size));
}
v->insert(v->end(), s, s + length);
position = v->size();
cursor = v->data() + v->size();
limit = cursor;
}
std::vector<CharType, AllocatorType>* v;
/// number of characters of *v written so far
std::size_t position;
/// the write position: in *v, at the number of characters written so far
CharType* cursor;
/// the end of *v
CharType* limit;
};
/// output adapter for byte vectors
@@ -277,9 +365,17 @@ class output_string_adapter : public output_adapter_protocol<CharType>
///
/// Wraps the polymorphic output_adapter_t so the same binary_writer template can
/// also target arbitrary adapters (output streams, strings, user-provided
/// adapters) via the `output_adapter`-based overloads. Each write still goes
/// through one virtual call, exactly as before; only the concrete sinks above
/// avoid it.
/// adapters) via the `output_adapter`-based overloads.
///
/// A write through the adapter is a virtual call, and for a stream a call with
/// a sentry on top, which costs more than encoding the few bytes of most values.
/// The sink therefore collects what is written in a buffer and passes it on in
/// one piece when it is full, a write is too large for it, or @ref flush is
/// called. Everything written reaches the adapter in the same order and with
/// the same bytes; the destructor passes on what is left, so that a write that
/// throws leaves the adapter with exactly the bytes written before it. Only
/// @ref flush can report an error of the adapter (the destructor must not
/// throw), so it has to be called once the value is written.
template<typename CharType>
class output_adapter_sink
{
@@ -290,36 +386,121 @@ class output_adapter_sink
JSON_ASSERT(oa);
}
output_adapter_sink(const output_adapter_sink&) = delete;
output_adapter_sink& operator=(const output_adapter_sink&) = delete;
output_adapter_sink& operator=(output_adapter_sink&&) = delete;
output_adapter_sink(output_adapter_sink&& other) noexcept
: oa(std::move(other.oa)), used(other.used)
{
std::memcpy(buffer.data(), other.buffer.data(), other.used * sizeof(CharType));
other.used = 0;
}
~output_adapter_sink()
{
// a normal return has flushed already, so this is for a write that threw
if (used != 0)
{
JSON_TRY
{
flush();
}
JSON_CATCH(...)
{
// nothing can be done about an adapter that throws here
}
}
}
JSON_HEDLEY_ALWAYS_INLINE
void write_character(CharType c)
{
oa->write_character(c);
if (JSON_HEDLEY_UNLIKELY(used == buffer_size))
{
flush();
}
buffer[used++] = c;
}
// no JSON_HEDLEY_NON_NULL: forwards (null, 0) for empty payloads, exactly as
// the type-erased path already did before this sink existed
JSON_HEDLEY_ALWAYS_INLINE
void write_characters(const CharType* s, std::size_t length)
{
oa->write_characters(s, length);
if (JSON_HEDLEY_UNLIKELY(buffer_size - used < length))
{
write_large(s, length);
return;
}
if (length <= 32)
{
copy_short(buffer.data() + used, s, length, std::integral_constant<bool, sizeof(CharType) == 1>());
used += length;
}
else
{
std::memcpy(buffer.data() + used, s, length * sizeof(CharType));
used += length;
}
}
/// space for writing up to @a length characters (see
/// output_vector_sink::reserve)
CharType* reserve(const std::size_t length) noexcept
JSON_HEDLEY_ALWAYS_INLINE
CharType* reserve(const std::size_t length)
{
JSON_ASSERT(length <= max_reserve_size);
static_cast<void>(length);
return reserved.data();
if (JSON_HEDLEY_UNLIKELY(buffer_size - used < length))
{
flush();
}
return buffer.data() + used;
}
/// write the first @a length characters written to the space from reserve()
void commit(const std::size_t length)
/// keep the first @a length characters written to the space from reserve()
JSON_HEDLEY_ALWAYS_INLINE
void commit(const std::size_t length) noexcept
{
oa->write_characters(reserved.data(), length);
used += length;
}
/// pass everything written so far on to the adapter
void flush()
{
if (used != 0)
{
// reset first: the characters count as written even if the adapter throws
const std::size_t length = used;
used = 0;
oa->write_characters(buffer.data(), length);
}
}
private:
/// a write that does not fit the space left in the buffer
JSON_HEDLEY_NEVER_INLINE
void write_large(const CharType* s, const std::size_t length)
{
flush();
if (length >= buffer_size)
{
oa->write_characters(s, length);
}
else
{
std::memcpy(buffer.data(), s, length * sizeof(CharType));
used = length;
}
}
/// large enough for reserve() at any time
static constexpr std::size_t buffer_size = 4 * max_reserve_size;
output_adapter_t<CharType> oa;
std::array<CharType, max_reserve_size> reserved{};
/// number of characters of @ref buffer not yet passed on
std::size_t used = 0;
std::array<CharType, buffer_size> buffer{};
};
/// @brief whether std::basic_string<CharType> has a non-deprecated std::char_traits
+46 -131
View File
@@ -275,41 +275,11 @@ class builder
0, 0, '\b', 0, 0, 0, '\f', 0, 0, 0, 0, 0, 0, 0, '\n', 0, 0, 0, '\r', 0, '\t', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
}
};
// The write position is kept in locals: the stores of characters into
// the arena may alias the members of the builder, which would then be
// reloaded after every one. It is stored back where the arena grows
// and at the end.
const unsigned char* const end = m_end;
const std::size_t start = arena_used();
char* w = m_arena_write;
const char* w_end = m_arena_end;
#define NLOHMANN_VIEW_ENSURE(n) \
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(w_end - w) < (n))) \
{ \
m_arena_write = w; \
arena_grow(n); \
w = m_arena_write; \
w_end = m_arena_end; \
}
{
const auto n = static_cast<std::size_t>(p - s);
if (n <= 16 && end - s >= 16)
{
NLOHMANN_VIEW_ENSURE(16)
std::memcpy(w, s, 16);
}
else if (n != 0)
{
// (an empty run leaves an empty arena unallocated, and memcpy()
// must not be passed its null pointer even for zero bytes)
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, s, n);
}
w += n;
}
arena_run(s, static_cast<std::size_t>(p - s));
for (;;)
{
if (p == end)
if (p == m_end)
{
return failed(error_code::string_missing_quote, p);
}
@@ -317,7 +287,6 @@ class builder
if (c == '"')
{
++p;
m_arena_write = w;
return decoded{p, start, arena_used() - start};
}
if (c != '\\')
@@ -328,7 +297,7 @@ class builder
return failed(c < 0x20 ? error_code::string_control_character : error_code::string_utf8, p);
}
++p;
if (p == end)
if (p == m_end)
{
return failed(error_code::string_missing_quote, p);
}
@@ -346,7 +315,7 @@ class builder
{
return failed(error_code::string_surrogate_low, p);
}
if (end - p < 2 || p[0] != '\\' || p[1] != 'u')
if (m_end - p < 2 || p[0] != '\\' || p[1] != 'u')
{
return failed(error_code::string_surrogate_high, p);
}
@@ -362,108 +331,27 @@ class builder
}
cp = 0x10000 + ((cp - 0xD800) << 10) + (lo - 0xDC00);
}
NLOHMANN_VIEW_ENSURE(4)
w = put_utf8(w, cp);
arena_ensure(4);
m_arena_write = put_utf8(m_arena_write, cp);
}
else if (NLOHMANN_VIEW_LIKELY(d < 128 && simple_escape[d] != 0))
{
NLOHMANN_VIEW_ENSURE(1)
*w++ = simple_escape[d];
arena_ensure(1);
*m_arena_write++ = simple_escape[d];
}
else
{
--p;
return failed(error_code::string_escape, p);
}
#if NLOHMANN_VIEW_NEON
if (p != end && *p == '\\')
if (p != m_end && *p == '\\')
{
continue; // consecutive escapes ("\u00e4\u00f6"): no run in between
}
// The first 16 bytes of the run are checked one by one, in predicted branches, and copied
// as one block: where the runs between escapes are short, the vector step below would put
// its longer dependency chain (load, compare, mask, count, add) on p for each of them,
// and be slower than these steps (the scanner of the keys works the same way)
{
if (end - p >= 16)
{
const std::uint8_t* const plain = string_plain();
std::size_t steps = 16;
#define NLOHMANN_VIEW_STEP(i) if (NLOHMANN_VIEW_LIKELY(plain[p[i]] != 0)) {} else { steps = (i); goto run_found; }
NLOHMANN_VIEW_REPEAT16(NLOHMANN_VIEW_STEP)
#undef NLOHMANN_VIEW_STEP
run_found:
NLOHMANN_VIEW_ENSURE(16)
std::memcpy(w, p, 16); // (only the first `steps` bytes are kept)
w += steps;
p += steps;
if (steps < 16)
{
if (*p >= 0x80)
{
// non-ASCII: the UTF-8 check, and the rest of the run
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, r, n);
w += n;
}
continue;
}
}
}
#else
if (p != end && (*p == '\\' || *p == '"'))
{
continue; // consecutive escapes ("\u00e4\u00f6"), or the end of the string: no run in between
}
#endif
#if NLOHMANN_VIEW_VECTOR
// the plain bytes up to the next quote, backslash, or control
// character are copied as they are scanned, 16 bytes at a time
for (;;)
{
if (NLOHMANN_VIEW_UNLIKELY(end - p < 16))
{
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, r, n);
w += n;
break;
}
NLOHMANN_VIEW_ENSURE(16)
const unsigned k = copy_plain_run16(p, w);
w += k;
p += k;
if (k == 16)
{
continue;
}
if (NLOHMANN_VIEW_UNLIKELY(*p >= 0x80))
{
// non-ASCII: the UTF-8 check, and the rest of the run
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, r, n);
w += n;
}
break;
}
#else
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, r, n);
w += n;
#endif
p = scan_string_run(p, m_end);
arena_run(r, static_cast<std::size_t>(p - r));
}
#undef NLOHMANN_VIEW_ENSURE
}
/// does the float token [s, p) overflow FloatType? (parse() rejects it)
@@ -509,6 +397,14 @@ run_found:
return m_arena_write != nullptr ? static_cast<std::size_t>(m_arena_write - m_doc.arena.data()) : 0;
}
NLOHMANN_VIEW_ALWAYS_INLINE void arena_ensure(std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(m_arena_end - m_arena_write) < n))
{
arena_grow(n);
}
}
NLOHMANN_VIEW_NOINLINE void arena_grow(std::size_t n)
{
const std::size_t used = arena_used();
@@ -517,6 +413,25 @@ run_found:
m_arena_end = m_doc.arena.data() + m_doc.arena.size();
}
/// append the run [r, r + n) to the arena; short runs as one fixed-size
/// 16-byte move when both sides have the room (no library call)
NLOHMANN_VIEW_ALWAYS_INLINE void arena_run(const unsigned char* r, std::size_t n)
{
if (n <= 16 && m_end - r >= 16)
{
arena_ensure(16);
std::memcpy(m_arena_write, r, 16);
}
else if (n != 0)
{
// (an empty run leaves an empty arena unallocated, and memcpy()
// must not be passed its null pointer even for zero bytes)
arena_ensure(n);
std::memcpy(m_arena_write, r, n);
}
m_arena_write += n;
}
/// UTF-8 encoding of cp at w (room for 4 bytes); returns the position after
/// it. (A returned pointer stays in a register; writing through a
/// reference to m_arena_write would store and reload it around every byte.)
@@ -1039,11 +954,11 @@ indent_done:
++p;
}
const unsigned char* const int_start = p;
if (cur() == '0')
if (p != e && *p == '0')
{
++p;
}
else if (NLOHMANN_VIEW_LIKELY(cur() >= '1' && cur() <= '9'))
else if (NLOHMANN_VIEW_LIKELY(p != e && *p >= '1' && *p <= '9'))
{
p = skip_digits(p + 1, e);
}
@@ -1054,7 +969,7 @@ indent_done:
const auto int_digits = static_cast<std::size_t>(p - int_start);
std::size_t frac_digits = 0;
bool is_float = false;
if (cur() == '.')
if (p != e && *p == '.')
{
++p;
const unsigned char* const f0 = p;
@@ -1067,20 +982,20 @@ indent_done:
is_float = true;
}
std::int64_t exponent = 0;
if ((cur() | 0x20) == 'e')
if (p != e && (*p | 0x20) == 'e')
{
++p;
bool exp_negative = false;
if (cur() == '+' || cur() == '-')
if (p != e && (*p == '+' || *p == '-'))
{
exp_negative = *p == '-';
++p;
}
if (NLOHMANN_VIEW_UNLIKELY(!is_digit(cur())))
if (NLOHMANN_VIEW_UNLIKELY(p == e || !is_digit(*p)))
{
return fail(error_code::number_after_exponent);
}
while (is_digit(cur()))
while (p != e && is_digit(*p))
{
if (exponent < 100000)
{
@@ -1142,7 +1057,7 @@ indent_done:
++p; // opening quote
const unsigned char* const s = p;
p = scan_string_run<Value>(p, e);
if (NLOHMANN_VIEW_LIKELY(cur() == '"'))
if (NLOHMANN_VIEW_LIKELY(p != e && *p == '"'))
{
push_node(value_t::string, 0, 0, static_cast<std::size_t>(s - b), static_cast<std::uint64_t>(p - s));
++p;
-34
View File
@@ -10,9 +10,6 @@
#include <algorithm> // max
#include <cstddef> // size_t
#ifndef JSON_NO_IO
#include <cstdio> // FILE, fread
#endif
#include <istream> // istream, streambuf
#include <string> // basic_string, char_traits, string
#include <type_traits> // decay, integral_constant, is_array, is_const, is_integral, is_lvalue_reference, is_pointer, is_same, remove_cv, remove_extent, remove_pointer, remove_reference
@@ -157,37 +154,6 @@ inline void append_stream(std::string& buf, std::istream& is)
is.clear(is.rdstate() | std::ios::eofbit);
}
/// a pointer to a C stream (never with JSON_NO_IO, which leaves out <cstdio>)
template<typename T>
struct is_c_stream
#ifndef JSON_NO_IO
: std::is_same<T, std::FILE*>
#else
: std::false_type
#endif
{};
#ifndef JSON_NO_IO
/// append the rest of a C stream: in blocks with fread, not character by
/// character (the library's file adapter reads with fgetc);
/// f is not null
inline void append_file(std::string& buf, std::FILE* f)
{
for (;;)
{
const std::size_t used = buf.size();
const std::size_t room = (std::max)(buf.capacity() - used, std::size_t{4096});
buf.resize(used + room);
const std::size_t got = std::fread(&buf[used], 1, room, f);
buf.resize(used + got);
if (got < room)
{
break; // end of file, or an error (as fgetc, which returns EOF for both)
}
}
}
#endif
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+8 -10
View File
@@ -147,21 +147,20 @@ class view_item
view_item(const iterator& it, std::size_t index)
: m_it(it)
, m_position(index)
{}
{
if (!it.is_object_iterator())
{
m_index = std::to_string(index);
}
}
/// the member key, or the element index for arrays (as text made on the
/// first call, so that iterating over the values pays nothing for it)
/// the member key, or the element index for arrays
string_view_t key() const
{
if (m_it.is_object_iterator())
{
return m_it.key();
}
if (m_index.empty())
{
m_index = std::to_string(m_position);
}
return string_view_t(m_index.data(), m_index.size());
}
@@ -184,8 +183,7 @@ class view_item
private:
iterator m_it;
std::size_t m_position; ///< the index of an array element
mutable std::string m_index{}; ///< the text of m_position, once key() was called // NOLINT(readability-redundant-member-init)
std::string m_index{}; // NOLINT(readability-redundant-member-init)
};
/// the range returned by basic_json_view::items()
-8
View File
@@ -166,10 +166,6 @@ const node* element_at(const document_data& d, const node* array, std::size_t id
{
return e + idx; // a growable block: one link per element
}
if (array->next == array->len + 1u && (!nav::editable() || (array->flags & node_flags::moved) == 0))
{
return e + idx; // no element has a subtree: the elements are the nodes after the array
}
for (std::size_t i = 0; i < idx; ++i)
{
e = document_data::after(e);
@@ -185,10 +181,6 @@ const node* last_child(const document_data& d, const node* container) noexcept
const std::size_t value_offset = container->kind == static_cast<std::uint8_t>(value_t::object) ? 1 : 0;
const node* const end = navigation<Editable>::end(d, container);
const node* last = navigation<Editable>::first(d, container);
if (value_offset == 0 && container->next == container->len + 1u && (!navigation<Editable>::editable() || (container->flags & node_flags::moved) == 0))
{
return last + (container->len - 1u); // an array without subtrees: the elements are the nodes after it
}
for (const node* c = document_data::after(last + value_offset); c != end; c = document_data::after(c + value_offset))
{
last = c;
+6 -44
View File
@@ -8,7 +8,7 @@
#pragma once
#include <array> // array
#include <algorithm> // max
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t
#include <string> // string
@@ -44,9 +44,8 @@ BasicJsonType materialize(const document_data& d, const node* n)
using nav = navigation<Editable>;
// the open containers: their next element (or key of the next member),
// their end, and whether they are objects; for the first 16 levels on the
// stack, else sized once from the document's deepest nesting (for a
// subtree an upper bound)
// their end, and whether they are objects; sized once from the
// document's deepest nesting (for a subtree an upper bound)
struct frame
{
const node* pos;
@@ -54,43 +53,10 @@ BasicJsonType materialize(const document_data& d, const node* n)
bool object;
};
// a scalar needs neither the handler nor the stack of open containers
switch (static_cast<value_t>(n->kind))
{
case value_t::string:
return BasicJsonType(string_t(d.str(*n), n->len));
case value_t::number_integer:
return BasicJsonType(static_cast<typename BasicJsonType::number_integer_t>(static_cast<std::int64_t>(integer_bits(*n))));
case value_t::number_unsigned:
return BasicJsonType(static_cast<typename BasicJsonType::number_unsigned_t>(integer_bits(*n)));
case value_t::number_float:
return BasicJsonType(float_value<typename BasicJsonType::number_float_t>(d, *n));
case value_t::boolean:
return BasicJsonType((n->flags & node_flags::is_true) != 0);
case value_t::null:
return BasicJsonType(nullptr);
case value_t::object:
case value_t::array:
case value_t::binary:
case value_t::discarded:
default:
break;
}
BasicJsonType result;
sax_t sax(result, true);
const string_t no_token{};
// (shallow subtrees on the stack: a call for a small value then allocates only for the value)
std::array<frame, 16> shallow; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled as containers open
std::vector<frame> deep{};
frame* open = shallow.data();
std::size_t capacity = shallow.size();
if (d.max_depth > capacity)
{
deep.resize(d.max_depth);
open = deep.data();
capacity = deep.size();
}
std::vector<frame> open((std::max)(d.max_depth, std::size_t{1}));
std::size_t depth = 0;
for (;;)
{
@@ -110,13 +76,9 @@ BasicJsonType materialize(const document_data& d, const node* n)
{
sax.start_array(n->len);
}
if (NLOHMANN_VIEW_UNLIKELY(depth == capacity))
if (NLOHMANN_VIEW_UNLIKELY(depth == open.size()))
{
// (deeper than the recorded nesting, e.g. after edits)
deep.assign(open, open + depth);
deep.resize(depth * 2);
open = deep.data();
capacity = deep.size();
open.resize(depth * 2); // (deeper than the recorded nesting, e.g. after edits)
}
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object};
break;
+11 -101
View File
@@ -47,96 +47,27 @@ constexpr std::size_t index_max_displacement = 64;
static_assert(document_data::index_min_members >= 2 * index_max_displacement,
"a table must be much larger than the probe window of a lookup");
/// hash of a key: its bytes, eight at a time (in a fixed byte order), where the
/// last word may overlap the one before (so that no loop over single bytes is
/// needed); a key of fewer than eight bytes is read as two overlapping words of
/// four or two bytes (in the native byte order: the hash is not stored, and the
/// same on both sides of a lookup)
/// hash of a key: its bytes, eight at a time, in a fixed byte order
inline std::uint64_t key_hash(const char* s, std::size_t n) noexcept
{
std::uint64_t h = 0x9E3779B97F4A7C15u * (n + 1);
const auto* p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
while (n >= 8)
{
h = (h ^ read_eight_bytes(p)) * 0xBF58476D1CE4E5B9u;
h ^= h >> 29u;
p += 8;
n -= 8;
}
std::uint64_t w = 0;
if (n >= 8)
for (std::size_t i = 0; i < n; ++i)
{
const unsigned char* const last = p + n - 8;
while (p < last)
{
h = (h ^ read_eight_bytes(p)) * 0xBF58476D1CE4E5B9u;
h ^= h >> 29u;
p += 8;
}
w = read_eight_bytes(last);
w |= static_cast<std::uint64_t>(p[i]) << (8u * i);
}
else if (n >= 4)
{
std::uint32_t head = 0;
std::uint32_t tail = 0;
std::memcpy(&head, p, 4);
std::memcpy(&tail, p + n - 4, 4);
w = static_cast<std::uint64_t>(head) | (static_cast<std::uint64_t>(tail) << 32u);
}
else if (n >= 2)
{
std::uint16_t head = 0;
std::uint16_t tail = 0;
std::memcpy(&head, p, 2);
std::memcpy(&tail, p + n - 2, 2);
w = static_cast<std::uint64_t>(head) | (static_cast<std::uint64_t>(tail) << 16u);
}
else if (n == 1)
{
w = p[0];
}
// (a product only carries bits upwards, and the slot is taken from the low
// bits: the high bits of the last word are folded down before the multiplication)
h = (h ^ w ^ (w >> 29u)) * 0x94D049BB133111EBu;
h = (h ^ w) * 0x94D049BB133111EBu;
return h ^ (h >> 31u);
}
/// whether the n <= 16 bytes at a and b are equal, as two overlapping words
/// (the first and the last 8, 4, or 2 bytes) and without a call
NLOHMANN_VIEW_ALWAYS_INLINE bool same_short_bytes(const unsigned char* a, const unsigned char* b, std::size_t n) noexcept
{
if (n >= 8)
{
std::uint64_t x0 = 0;
std::uint64_t x1 = 0;
std::uint64_t y0 = 0;
std::uint64_t y1 = 0;
std::memcpy(&x0, a, 8);
std::memcpy(&x1, a + n - 8, 8);
std::memcpy(&y0, b, 8);
std::memcpy(&y1, b + n - 8, 8);
return x0 == y0 && x1 == y1;
}
if (n >= 4)
{
std::uint32_t x0 = 0;
std::uint32_t x1 = 0;
std::uint32_t y0 = 0;
std::uint32_t y1 = 0;
std::memcpy(&x0, a, 4);
std::memcpy(&x1, a + n - 4, 4);
std::memcpy(&y0, b, 4);
std::memcpy(&y1, b + n - 4, 4);
return x0 == y0 && x1 == y1;
}
if (n >= 2)
{
std::uint16_t x0 = 0;
std::uint16_t x1 = 0;
std::uint16_t y0 = 0;
std::uint16_t y1 = 0;
std::memcpy(&x0, a, 2);
std::memcpy(&x1, a + n - 2, 2);
std::memcpy(&y0, b, 2);
std::memcpy(&y1, b + n - 2, 2);
return x0 == y0 && x1 == y1;
}
return n == 0 || *a == *b;
}
/// build the table of a large object
inline void build_object_index(document_data& d, node* obj)
{
@@ -201,27 +132,6 @@ inline const node* find_indexed(const document_data& d, const node* obj, const c
const document_data::object_index& ix = d.indexes[obj->extra - 1u];
const std::uint32_t* const table = d.index_slots.data() + ix.start;
std::size_t slot = conditional_static_cast<std::size_t>(key_hash(key, n)) & ix.mask;
const auto* const k_bytes = reinterpret_cast<const unsigned char*>(key); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
// (the loop for longer keys comes first, as the code is laid out; it is not the more common case)
if (NLOHMANN_VIEW_UNLIKELY(n <= 16))
{
// (up to 16 bytes, the bytes are compared as two words, without a call)
for (std::size_t distance = 0; distance <= index_max_displacement; ++distance)
{
const std::uint32_t s = table[slot];
if (s == 0)
{
return nullptr;
}
const node* const k = obj + s;
if (k->len == n && same_short_bytes(reinterpret_cast<const unsigned char*>(d.str(*k)), k_bytes, n)) // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
{
return k;
}
slot = (slot + 1) & ix.mask;
}
return nullptr;
}
for (std::size_t distance = 0; distance <= index_max_displacement; ++distance)
{
const std::uint32_t s = table[slot];
-28
View File
@@ -116,34 +116,6 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned
}
return p;
}
/*!
@brief copy 16 bytes from p to w, and find the first quote, backslash,
control character, or non-ASCII byte among them
The bytes are copied whether or not they are all plain (the caller has room
for 16 bytes and keeps only the plain ones).
@return the offset of that byte, or 16 if there is none
*/
NLOHMANN_VIEW_ALWAYS_INLINE unsigned copy_plain_run16(const unsigned char* p, char* w) noexcept
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
vst1q_u8(reinterpret_cast<std::uint8_t*>(w), in); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits) >> 2u) : 16u;
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
_mm_storeu_si128(static_cast<__m128i*>(static_cast<void*>(w)), in); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits)) : 16u;
#endif
}
#endif
#if NLOHMANN_VIEW_SSSE3_DISPATCH
+42 -18
View File
@@ -6321,7 +6321,7 @@ public:
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
std::vector<std::uint8_t> result;
result.reserve(detail::binary_reserve_hint(j));
result.reserve(detail::binary_initial_capacity(j));
vector_writer(result, error_handler).write_cbor(j);
return result;
}
@@ -6331,7 +6331,9 @@ public:
static void to_cbor(const basic_json& j, detail::output_adapter<std::uint8_t> o,
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
binary_writer<std::uint8_t>(o, error_handler).write_cbor(j);
binary_writer<std::uint8_t> writer(o, error_handler);
writer.write_cbor(j);
writer.flush();
}
/// @brief create a CBOR serialization of a given JSON value
@@ -6339,7 +6341,9 @@ public:
static void to_cbor(const basic_json& j, detail::output_adapter<char> o,
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
binary_writer<char>(o, error_handler).write_cbor(j);
binary_writer<char> writer(o, error_handler);
writer.write_cbor(j);
writer.flush();
}
/// @brief create a MessagePack serialization of a given JSON value
@@ -6348,7 +6352,7 @@ public:
const error_handler_t error_handler = error_handler_t::keep)
{
std::vector<std::uint8_t> result;
result.reserve(detail::binary_reserve_hint(j));
result.reserve(detail::binary_initial_capacity(j));
vector_writer(result, error_handler).write_msgpack(j);
return result;
}
@@ -6358,7 +6362,9 @@ public:
static void to_msgpack(const basic_json& j, detail::output_adapter<std::uint8_t> o,
const error_handler_t error_handler = error_handler_t::keep)
{
binary_writer<std::uint8_t>(o, error_handler).write_msgpack(j);
binary_writer<std::uint8_t> writer(o, error_handler);
writer.write_msgpack(j);
writer.flush();
}
/// @brief create a MessagePack serialization of a given JSON value
@@ -6366,7 +6372,9 @@ public:
static void to_msgpack(const basic_json& j, detail::output_adapter<char> o,
const error_handler_t error_handler = error_handler_t::keep)
{
binary_writer<char>(o, error_handler).write_msgpack(j);
binary_writer<char> writer(o, error_handler);
writer.write_msgpack(j);
writer.flush();
}
/// @brief create a UBJSON serialization of a given JSON value
@@ -6377,7 +6385,7 @@ public:
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
std::vector<std::uint8_t> result;
result.reserve(detail::binary_reserve_hint(j));
result.reserve(detail::binary_initial_capacity(j));
vector_writer(result, error_handler).write_ubjson(j, use_size, use_type);
return result;
}
@@ -6388,7 +6396,9 @@ public:
const bool use_size = false, const bool use_type = false,
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
binary_writer<std::uint8_t>(o, error_handler).write_ubjson(j, use_size, use_type);
binary_writer<std::uint8_t> writer(o, error_handler);
writer.write_ubjson(j, use_size, use_type);
writer.flush();
}
/// @brief create a UBJSON serialization of a given JSON value
@@ -6397,7 +6407,9 @@ public:
const bool use_size = false, const bool use_type = false,
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
binary_writer<char>(o, error_handler).write_ubjson(j, use_size, use_type);
binary_writer<char> writer(o, error_handler);
writer.write_ubjson(j, use_size, use_type);
writer.flush();
}
/// @brief create a BJData serialization of a given JSON value
@@ -6409,7 +6421,7 @@ public:
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
std::vector<std::uint8_t> result;
result.reserve(detail::binary_reserve_hint(j));
result.reserve(detail::binary_initial_capacity(j));
vector_writer(result, error_handler).write_ubjson(j, use_size, use_type, true, true, version);
return result;
}
@@ -6421,7 +6433,9 @@ public:
const bjdata_version_t version = bjdata_version_t::draft2,
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
binary_writer<std::uint8_t>(o, error_handler).write_ubjson(j, use_size, use_type, true, true, version);
binary_writer<std::uint8_t> writer(o, error_handler);
writer.write_ubjson(j, use_size, use_type, true, true, version);
writer.flush();
}
/// @brief create a BJData serialization of a given JSON value
@@ -6431,7 +6445,9 @@ public:
const bjdata_version_t version = bjdata_version_t::draft2,
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
binary_writer<char>(o, error_handler).write_ubjson(j, use_size, use_type, true, true, version);
binary_writer<char> writer(o, error_handler);
writer.write_ubjson(j, use_size, use_type, true, true, version);
writer.flush();
}
/// @brief create a BSON serialization of a given JSON value
@@ -6440,7 +6456,7 @@ public:
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
std::vector<std::uint8_t> result;
result.reserve(detail::binary_reserve_hint(j));
result.reserve(detail::binary_initial_capacity(j));
vector_writer(result, error_handler).write_bson(j);
return result;
}
@@ -6450,7 +6466,9 @@ public:
static void to_bson(const basic_json& j, detail::output_adapter<std::uint8_t> o,
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
binary_writer<std::uint8_t>(o, error_handler).write_bson(j);
binary_writer<std::uint8_t> writer(o, error_handler);
writer.write_bson(j);
writer.flush();
}
/// @brief create a BSON serialization of a given JSON value
@@ -6458,7 +6476,9 @@ public:
static void to_bson(const basic_json& j, detail::output_adapter<char> o,
const error_handler_t error_handler = detail::binary_writer_default_error_handler())
{
binary_writer<char>(o, error_handler).write_bson(j);
binary_writer<char> writer(o, error_handler);
writer.write_bson(j);
writer.flush();
}
/// @brief create a BON8 serialization of a given JSON value
@@ -6466,7 +6486,7 @@ public:
static std::vector<std::uint8_t> to_bon8(const basic_json& j)
{
std::vector<std::uint8_t> result;
result.reserve(detail::binary_reserve_hint(j));
result.reserve(detail::binary_initial_capacity(j));
vector_writer(result).write_bon8(j);
return result;
}
@@ -6475,14 +6495,18 @@ public:
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
static void to_bon8(const basic_json& j, detail::output_adapter<std::uint8_t> o)
{
binary_writer<std::uint8_t>(o).write_bon8(j);
binary_writer<std::uint8_t> writer(o);
writer.write_bon8(j);
writer.flush();
}
/// @brief create a BON8 serialization of a given JSON value
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
static void to_bon8(const basic_json& j, detail::output_adapter<char> o)
{
binary_writer<char>(o).write_bon8(j);
binary_writer<char> writer(o);
writer.write_bon8(j);
writer.flush();
}
/// @brief create a JSON value from an input in CBOR format
+10 -67
View File
@@ -295,7 +295,7 @@ class basic_json_view
}
detail::view::throw_type_error(305, "cannot use operator[] with a numeric argument with ", type_name());
}
return idx < m_node->len ? element(idx) : basic_json_view();
return idx < m_node->len ? basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx))) : basic_json_view();
}
/// any other integer type (int, unsigned, long, std::int64_t, ...; a
@@ -358,7 +358,7 @@ class basic_json_view
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return element(idx);
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
}
/// any other integer type, see operator[]; negative values are out of
@@ -392,13 +392,7 @@ class basic_json_view
string_t value(string_view_t key, const char* default_value) const
{
if (NLOHMANN_VIEW_UNLIKELY(!is_object()))
{
detail::view::throw_type_error(306, "cannot use value() with ", type_name());
}
// (the default is made only if there is no such member)
const basic_json_view r = lookup(key);
return r.is_discarded() ? string_t(default_value) : r.template get<string_t>();
return value(key, string_t(default_value));
}
/// the value a JSON pointer refers to converted to T, or the default
@@ -417,12 +411,7 @@ class basic_json_view
string_t value(const json_pointer& ptr, const char* default_value) const
{
if (NLOHMANN_VIEW_UNLIKELY(!is_structured()))
{
detail::view::throw_type_error(306, "cannot use value() with ", type_name());
}
const basic_json_view r = detail::view::resolve_pointer(*this, detail::json_pointer_access::reference_tokens(ptr), detail::view::pointer_mode::value);
return r.is_discarded() ? string_t(default_value) : r.template get<string_t>();
return value(ptr, string_t(default_value));
}
/// the first element or member value; a primitive value itself. Throws
@@ -444,8 +433,7 @@ class basic_json_view
{
if (is_structured() && m_node->len != 0)
{
return unedited() ? basic_json_view(m_doc, detail::view::last_child<false>(*m_doc, m_node) + (is_object() ? 1 : 0))
: basic_json_view(m_doc, navigation::value(detail::view::last_child<Editable>(*m_doc, m_node) + (is_object() ? 1 : 0)));
return basic_json_view(m_doc, navigation::value(detail::view::last_child<Editable>(*m_doc, m_node) + (is_object() ? 1 : 0)));
}
return front();
}
@@ -462,8 +450,7 @@ class basic_json_view
{
return end();
}
const node* const k = unedited() ? detail::view::find_member<false>(*m_doc, m_node, key.data(), key.size())
: detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? iterator(m_doc, k, true) : end();
}
@@ -480,8 +467,7 @@ class basic_json_view
/// whether this is an object with a member with this key
bool contains(string_view_t key) const
{
return is_object() && (unedited() ? detail::view::find_member<false>(*m_doc, m_node, key.data(), key.size()) != nullptr
: detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size()) != nullptr);
return is_object() && detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size()) != nullptr;
}
bool contains(const char* key) const
@@ -641,14 +627,7 @@ class basic_json_view
// the compact writer keeps 64 bytes of slack, so that it does not grow
// the buffer just before the end
const std::size_t estimate = source_extent() + (style.pretty ? source_extent() / 2 : 0) + 160;
if (unedited())
{
detail::view::view_serializer<BasicJsonType, false>(*m_doc, out, estimate, style).dump(m_node);
}
else
{
detail::view::view_serializer<BasicJsonType, Editable>(*m_doc, out, estimate, style).dump(m_node);
}
detail::view::view_serializer<BasicJsonType, Editable>(*m_doc, out, estimate, style).dump(m_node);
return out;
}
@@ -728,7 +707,7 @@ class basic_json_view
{
return BasicJsonType(value_t::discarded);
}
return unedited() ? detail::view::materialize<BasicJsonType, false>(*m_doc, m_node) : detail::view::materialize<BasicJsonType, Editable>(*m_doc, m_node);
return detail::view::materialize<BasicJsonType, Editable>(*m_doc, m_node);
}
/// byte offset of this value in the source text (for strings: of the
@@ -806,32 +785,10 @@ class basic_json_view
/// (object required)
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view lookup(string_view_t key) const noexcept
{
if (unedited())
{
const node* const k = detail::view::find_member<false>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, k + 1) : basic_json_view();
}
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, navigation::value(k + 1)) : basic_json_view();
}
/// whether the document has not been edited yet: until the first edit, its index is walked like that of a
/// read-only document (nothing is moved or linked), which is cheaper
NLOHMANN_VIEW_ALWAYS_INLINE bool unedited() const noexcept
{
return !navigation::editable() || m_doc->edits == nullptr;
}
/// the element of an array at an index below its size
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view element(size_type idx) const
{
if (unedited())
{
return basic_json_view(m_doc, detail::view::element_at<false>(*m_doc, m_node, idx));
}
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
}
// --- get() dispatch ---
bool get_impl(detail::view::value_tag<bool> /*unused*/, detail::priority_tag<2> /*unused*/) const
@@ -1571,8 +1528,7 @@ class basic_json_document
// NOLINTBEGIN(readability-avoid-nested-conditional-operator): a constant expression of C++11
append_input_impl(buf, std::forward<T>(input), std::integral_constant < int, detail::is_contiguous_byte_container<D>::value ? 0
: std::is_base_of<std::istream, D>::value ? 1
: detail::view::is_adapter_input<D>::value ? 2
: detail::view::is_c_stream<D>::value ? 4 : 3 > {});
: detail::view::is_adapter_input<D>::value ? 2 : 3 > {});
// NOLINTEND(readability-avoid-nested-conditional-operator)
}
@@ -1594,19 +1550,6 @@ class basic_json_document
detail::view::append_adapter(buf, std::move(input.ia));
}
#ifndef JSON_NO_IO
static void append_input_impl(std::string& buf, std::FILE* file, std::integral_constant<int, 4> /*C stream*/)
{
if (file == nullptr)
{
// (as before: the library's adapter decides what a null stream yields)
detail::view::append_adapter(buf, detail::input_adapter(file));
return;
}
detail::view::append_file(buf, file);
}
#endif
template<typename T>
static void append_input_impl(std::string& buf, T&& input, std::integral_constant<int, 3> /*other*/)
{
+1975 -220
View File
File diff suppressed because it is too large. Load diff
+81 -423
View File
@@ -756,34 +756,6 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned
}
return p;
}
/*!
@brief copy 16 bytes from p to w, and find the first quote, backslash,
control character, or non-ASCII byte among them
The bytes are copied whether or not they are all plain (the caller has room
for 16 bytes and keeps only the plain ones).
@return the offset of that byte, or 16 if there is none
*/
NLOHMANN_VIEW_ALWAYS_INLINE unsigned copy_plain_run16(const unsigned char* p, char* w) noexcept
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
vst1q_u8(reinterpret_cast<std::uint8_t*>(w), in); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits) >> 2u) : 16u;
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
_mm_storeu_si128(static_cast<__m128i*>(static_cast<void*>(w)), in); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits)) : 16u;
#endif
}
#endif
#if NLOHMANN_VIEW_SSSE3_DISPATCH
@@ -1465,41 +1437,11 @@ class builder
0, 0, '\b', 0, 0, 0, '\f', 0, 0, 0, 0, 0, 0, 0, '\n', 0, 0, 0, '\r', 0, '\t', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
}
};
// The write position is kept in locals: the stores of characters into
// the arena may alias the members of the builder, which would then be
// reloaded after every one. It is stored back where the arena grows
// and at the end.
const unsigned char* const end = m_end;
const std::size_t start = arena_used();
char* w = m_arena_write;
const char* w_end = m_arena_end;
#define NLOHMANN_VIEW_ENSURE(n) \
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(w_end - w) < (n))) \
{ \
m_arena_write = w; \
arena_grow(n); \
w = m_arena_write; \
w_end = m_arena_end; \
}
{
const auto n = static_cast<std::size_t>(p - s);
if (n <= 16 && end - s >= 16)
{
NLOHMANN_VIEW_ENSURE(16)
std::memcpy(w, s, 16);
}
else if (n != 0)
{
// (an empty run leaves an empty arena unallocated, and memcpy()
// must not be passed its null pointer even for zero bytes)
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, s, n);
}
w += n;
}
arena_run(s, static_cast<std::size_t>(p - s));
for (;;)
{
if (p == end)
if (p == m_end)
{
return failed(error_code::string_missing_quote, p);
}
@@ -1507,7 +1449,6 @@ class builder
if (c == '"')
{
++p;
m_arena_write = w;
return decoded{p, start, arena_used() - start};
}
if (c != '\\')
@@ -1518,7 +1459,7 @@ class builder
return failed(c < 0x20 ? error_code::string_control_character : error_code::string_utf8, p);
}
++p;
if (p == end)
if (p == m_end)
{
return failed(error_code::string_missing_quote, p);
}
@@ -1536,7 +1477,7 @@ class builder
{
return failed(error_code::string_surrogate_low, p);
}
if (end - p < 2 || p[0] != '\\' || p[1] != 'u')
if (m_end - p < 2 || p[0] != '\\' || p[1] != 'u')
{
return failed(error_code::string_surrogate_high, p);
}
@@ -1552,108 +1493,27 @@ class builder
}
cp = 0x10000 + ((cp - 0xD800) << 10) + (lo - 0xDC00);
}
NLOHMANN_VIEW_ENSURE(4)
w = put_utf8(w, cp);
arena_ensure(4);
m_arena_write = put_utf8(m_arena_write, cp);
}
else if (NLOHMANN_VIEW_LIKELY(d < 128 && simple_escape[d] != 0))
{
NLOHMANN_VIEW_ENSURE(1)
*w++ = simple_escape[d];
arena_ensure(1);
*m_arena_write++ = simple_escape[d];
}
else
{
--p;
return failed(error_code::string_escape, p);
}
#if NLOHMANN_VIEW_NEON
if (p != end && *p == '\\')
if (p != m_end && *p == '\\')
{
continue; // consecutive escapes ("\u00e4\u00f6"): no run in between
}
// The first 16 bytes of the run are checked one by one, in predicted branches, and copied
// as one block: where the runs between escapes are short, the vector step below would put
// its longer dependency chain (load, compare, mask, count, add) on p for each of them,
// and be slower than these steps (the scanner of the keys works the same way)
{
if (end - p >= 16)
{
const std::uint8_t* const plain = string_plain();
std::size_t steps = 16;
#define NLOHMANN_VIEW_STEP(i) if (NLOHMANN_VIEW_LIKELY(plain[p[i]] != 0)) {} else { steps = (i); goto run_found; }
NLOHMANN_VIEW_REPEAT16(NLOHMANN_VIEW_STEP)
#undef NLOHMANN_VIEW_STEP
run_found:
NLOHMANN_VIEW_ENSURE(16)
std::memcpy(w, p, 16); // (only the first `steps` bytes are kept)
w += steps;
p += steps;
if (steps < 16)
{
if (*p >= 0x80)
{
// non-ASCII: the UTF-8 check, and the rest of the run
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, r, n);
w += n;
}
continue;
}
}
}
#else
if (p != end && (*p == '\\' || *p == '"'))
{
continue; // consecutive escapes ("\u00e4\u00f6"), or the end of the string: no run in between
}
#endif
#if NLOHMANN_VIEW_VECTOR
// the plain bytes up to the next quote, backslash, or control
// character are copied as they are scanned, 16 bytes at a time
for (;;)
{
if (NLOHMANN_VIEW_UNLIKELY(end - p < 16))
{
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, r, n);
w += n;
break;
}
NLOHMANN_VIEW_ENSURE(16)
const unsigned k = copy_plain_run16(p, w);
w += k;
p += k;
if (k == 16)
{
continue;
}
if (NLOHMANN_VIEW_UNLIKELY(*p >= 0x80))
{
// non-ASCII: the UTF-8 check, and the rest of the run
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, r, n);
w += n;
}
break;
}
#else
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
NLOHMANN_VIEW_ENSURE(n)
std::memcpy(w, r, n);
w += n;
#endif
p = scan_string_run(p, m_end);
arena_run(r, static_cast<std::size_t>(p - r));
}
#undef NLOHMANN_VIEW_ENSURE
}
/// does the float token [s, p) overflow FloatType? (parse() rejects it)
@@ -1699,6 +1559,14 @@ run_found:
return m_arena_write != nullptr ? static_cast<std::size_t>(m_arena_write - m_doc.arena.data()) : 0;
}
NLOHMANN_VIEW_ALWAYS_INLINE void arena_ensure(std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(m_arena_end - m_arena_write) < n))
{
arena_grow(n);
}
}
NLOHMANN_VIEW_NOINLINE void arena_grow(std::size_t n)
{
const std::size_t used = arena_used();
@@ -1707,6 +1575,25 @@ run_found:
m_arena_end = m_doc.arena.data() + m_doc.arena.size();
}
/// append the run [r, r + n) to the arena; short runs as one fixed-size
/// 16-byte move when both sides have the room (no library call)
NLOHMANN_VIEW_ALWAYS_INLINE void arena_run(const unsigned char* r, std::size_t n)
{
if (n <= 16 && m_end - r >= 16)
{
arena_ensure(16);
std::memcpy(m_arena_write, r, 16);
}
else if (n != 0)
{
// (an empty run leaves an empty arena unallocated, and memcpy()
// must not be passed its null pointer even for zero bytes)
arena_ensure(n);
std::memcpy(m_arena_write, r, n);
}
m_arena_write += n;
}
/// UTF-8 encoding of cp at w (room for 4 bytes); returns the position after
/// it. (A returned pointer stays in a register; writing through a
/// reference to m_arena_write would store and reload it around every byte.)
@@ -2229,11 +2116,11 @@ indent_done:
++p;
}
const unsigned char* const int_start = p;
if (cur() == '0')
if (p != e && *p == '0')
{
++p;
}
else if (NLOHMANN_VIEW_LIKELY(cur() >= '1' && cur() <= '9'))
else if (NLOHMANN_VIEW_LIKELY(p != e && *p >= '1' && *p <= '9'))
{
p = skip_digits(p + 1, e);
}
@@ -2244,7 +2131,7 @@ indent_done:
const auto int_digits = static_cast<std::size_t>(p - int_start);
std::size_t frac_digits = 0;
bool is_float = false;
if (cur() == '.')
if (p != e && *p == '.')
{
++p;
const unsigned char* const f0 = p;
@@ -2257,20 +2144,20 @@ indent_done:
is_float = true;
}
std::int64_t exponent = 0;
if ((cur() | 0x20) == 'e')
if (p != e && (*p | 0x20) == 'e')
{
++p;
bool exp_negative = false;
if (cur() == '+' || cur() == '-')
if (p != e && (*p == '+' || *p == '-'))
{
exp_negative = *p == '-';
++p;
}
if (NLOHMANN_VIEW_UNLIKELY(!is_digit(cur())))
if (NLOHMANN_VIEW_UNLIKELY(p == e || !is_digit(*p)))
{
return fail(error_code::number_after_exponent);
}
while (is_digit(cur()))
while (p != e && is_digit(*p))
{
if (exponent < 100000)
{
@@ -2332,7 +2219,7 @@ indent_done:
++p; // opening quote
const unsigned char* const s = p;
p = scan_string_run<Value>(p, e);
if (NLOHMANN_VIEW_LIKELY(cur() == '"'))
if (NLOHMANN_VIEW_LIKELY(p != e && *p == '"'))
{
push_node(value_t::string, 0, 0, static_cast<std::size_t>(s - b), static_cast<std::uint64_t>(p - s));
++p;
@@ -3281,96 +3168,27 @@ constexpr std::size_t index_max_displacement = 64;
static_assert(document_data::index_min_members >= 2 * index_max_displacement,
"a table must be much larger than the probe window of a lookup");
/// hash of a key: its bytes, eight at a time (in a fixed byte order), where the
/// last word may overlap the one before (so that no loop over single bytes is
/// needed); a key of fewer than eight bytes is read as two overlapping words of
/// four or two bytes (in the native byte order: the hash is not stored, and the
/// same on both sides of a lookup)
/// hash of a key: its bytes, eight at a time, in a fixed byte order
inline std::uint64_t key_hash(const char* s, std::size_t n) noexcept
{
std::uint64_t h = 0x9E3779B97F4A7C15u * (n + 1);
const auto* p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
while (n >= 8)
{
h = (h ^ read_eight_bytes(p)) * 0xBF58476D1CE4E5B9u;
h ^= h >> 29u;
p += 8;
n -= 8;
}
std::uint64_t w = 0;
if (n >= 8)
for (std::size_t i = 0; i < n; ++i)
{
const unsigned char* const last = p + n - 8;
while (p < last)
{
h = (h ^ read_eight_bytes(p)) * 0xBF58476D1CE4E5B9u;
h ^= h >> 29u;
p += 8;
}
w = read_eight_bytes(last);
w |= static_cast<std::uint64_t>(p[i]) << (8u * i);
}
else if (n >= 4)
{
std::uint32_t head = 0;
std::uint32_t tail = 0;
std::memcpy(&head, p, 4);
std::memcpy(&tail, p + n - 4, 4);
w = static_cast<std::uint64_t>(head) | (static_cast<std::uint64_t>(tail) << 32u);
}
else if (n >= 2)
{
std::uint16_t head = 0;
std::uint16_t tail = 0;
std::memcpy(&head, p, 2);
std::memcpy(&tail, p + n - 2, 2);
w = static_cast<std::uint64_t>(head) | (static_cast<std::uint64_t>(tail) << 16u);
}
else if (n == 1)
{
w = p[0];
}
// (a product only carries bits upwards, and the slot is taken from the low
// bits: the high bits of the last word are folded down before the multiplication)
h = (h ^ w ^ (w >> 29u)) * 0x94D049BB133111EBu;
h = (h ^ w) * 0x94D049BB133111EBu;
return h ^ (h >> 31u);
}
/// whether the n <= 16 bytes at a and b are equal, as two overlapping words
/// (the first and the last 8, 4, or 2 bytes) and without a call
NLOHMANN_VIEW_ALWAYS_INLINE bool same_short_bytes(const unsigned char* a, const unsigned char* b, std::size_t n) noexcept
{
if (n >= 8)
{
std::uint64_t x0 = 0;
std::uint64_t x1 = 0;
std::uint64_t y0 = 0;
std::uint64_t y1 = 0;
std::memcpy(&x0, a, 8);
std::memcpy(&x1, a + n - 8, 8);
std::memcpy(&y0, b, 8);
std::memcpy(&y1, b + n - 8, 8);
return x0 == y0 && x1 == y1;
}
if (n >= 4)
{
std::uint32_t x0 = 0;
std::uint32_t x1 = 0;
std::uint32_t y0 = 0;
std::uint32_t y1 = 0;
std::memcpy(&x0, a, 4);
std::memcpy(&x1, a + n - 4, 4);
std::memcpy(&y0, b, 4);
std::memcpy(&y1, b + n - 4, 4);
return x0 == y0 && x1 == y1;
}
if (n >= 2)
{
std::uint16_t x0 = 0;
std::uint16_t x1 = 0;
std::uint16_t y0 = 0;
std::uint16_t y1 = 0;
std::memcpy(&x0, a, 2);
std::memcpy(&x1, a + n - 2, 2);
std::memcpy(&y0, b, 2);
std::memcpy(&y1, b + n - 2, 2);
return x0 == y0 && x1 == y1;
}
return n == 0 || *a == *b;
}
/// build the table of a large object
inline void build_object_index(document_data& d, node* obj)
{
@@ -3435,27 +3253,6 @@ inline const node* find_indexed(const document_data& d, const node* obj, const c
const document_data::object_index& ix = d.indexes[obj->extra - 1u];
const std::uint32_t* const table = d.index_slots.data() + ix.start;
std::size_t slot = conditional_static_cast<std::size_t>(key_hash(key, n)) & ix.mask;
const auto* const k_bytes = reinterpret_cast<const unsigned char*>(key); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
// (the loop for longer keys comes first, as the code is laid out; it is not the more common case)
if (NLOHMANN_VIEW_UNLIKELY(n <= 16))
{
// (up to 16 bytes, the bytes are compared as two words, without a call)
for (std::size_t distance = 0; distance <= index_max_displacement; ++distance)
{
const std::uint32_t s = table[slot];
if (s == 0)
{
return nullptr;
}
const node* const k = obj + s;
if (k->len == n && same_short_bytes(reinterpret_cast<const unsigned char*>(d.str(*k)), k_bytes, n)) // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
{
return k;
}
slot = (slot + 1) & ix.mask;
}
return nullptr;
}
for (std::size_t distance = 0; distance <= index_max_displacement; ++distance)
{
const std::uint32_t s = table[slot];
@@ -3624,10 +3421,6 @@ const node* element_at(const document_data& d, const node* array, std::size_t id
{
return e + idx; // a growable block: one link per element
}
if (array->next == array->len + 1u && (!nav::editable() || (array->flags & node_flags::moved) == 0))
{
return e + idx; // no element has a subtree: the elements are the nodes after the array
}
for (std::size_t i = 0; i < idx; ++i)
{
e = document_data::after(e);
@@ -3643,10 +3436,6 @@ const node* last_child(const document_data& d, const node* container) noexcept
const std::size_t value_offset = container->kind == static_cast<std::uint8_t>(value_t::object) ? 1 : 0;
const node* const end = navigation<Editable>::end(d, container);
const node* last = navigation<Editable>::first(d, container);
if (value_offset == 0 && container->next == container->len + 1u && (!navigation<Editable>::editable() || (container->flags & node_flags::moved) == 0))
{
return last + (container->len - 1u); // an array without subtrees: the elements are the nodes after it
}
for (const node* c = document_data::after(last + value_offset); c != end; c = document_data::after(c + value_offset))
{
last = c;
@@ -5512,9 +5301,6 @@ NLOHMANN_JSON_NAMESPACE_END
#include <algorithm> // max
#include <cstddef> // size_t
#ifndef JSON_NO_IO
#include <cstdio> // FILE, fread
#endif
#include <istream> // istream, streambuf
#include <string> // basic_string, char_traits, string
#include <type_traits> // decay, integral_constant, is_array, is_const, is_integral, is_lvalue_reference, is_pointer, is_same, remove_cv, remove_extent, remove_pointer, remove_reference
@@ -5660,37 +5446,6 @@ inline void append_stream(std::string& buf, std::istream& is)
is.clear(is.rdstate() | std::ios::eofbit);
}
/// a pointer to a C stream (never with JSON_NO_IO, which leaves out <cstdio>)
template<typename T>
struct is_c_stream
#ifndef JSON_NO_IO
: std::is_same<T, std::FILE*>
#else
: std::false_type
#endif
{};
#ifndef JSON_NO_IO
/// append the rest of a C stream: in blocks with fread, not character by
/// character (the library's file adapter reads with fgetc);
/// f is not null
inline void append_file(std::string& buf, std::FILE* f)
{
for (;;)
{
const std::size_t used = buf.size();
const std::size_t room = (std::max)(buf.capacity() - used, std::size_t{4096});
buf.resize(used + room);
const std::size_t got = std::fread(&buf[used], 1, room, f);
buf.resize(used + got);
if (got < room)
{
break; // end of file, or an error (as fgetc, which returns EOF for both)
}
}
}
#endif
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
@@ -5849,21 +5604,20 @@ class view_item
view_item(const iterator& it, std::size_t index)
: m_it(it)
, m_position(index)
{}
{
if (!it.is_object_iterator())
{
m_index = std::to_string(index);
}
}
/// the member key, or the element index for arrays (as text made on the
/// first call, so that iterating over the values pays nothing for it)
/// the member key, or the element index for arrays
string_view_t key() const
{
if (m_it.is_object_iterator())
{
return m_it.key();
}
if (m_index.empty())
{
m_index = std::to_string(m_position);
}
return string_view_t(m_index.data(), m_index.size());
}
@@ -5886,8 +5640,7 @@ class view_item
private:
iterator m_it;
std::size_t m_position; ///< the index of an array element
mutable std::string m_index{}; ///< the text of m_position, once key() was called // NOLINT(readability-redundant-member-init)
std::string m_index{}; // NOLINT(readability-redundant-member-init)
};
/// the range returned by basic_json_view::items()
@@ -5981,7 +5734,7 @@ NLOHMANN_JSON_NAMESPACE_END
#include <array> // array
#include <algorithm> // max
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t
#include <string> // string
@@ -6021,9 +5774,8 @@ BasicJsonType materialize(const document_data& d, const node* n)
using nav = navigation<Editable>;
// the open containers: their next element (or key of the next member),
// their end, and whether they are objects; for the first 16 levels on the
// stack, else sized once from the document's deepest nesting (for a
// subtree an upper bound)
// their end, and whether they are objects; sized once from the
// document's deepest nesting (for a subtree an upper bound)
struct frame
{
const node* pos;
@@ -6031,43 +5783,10 @@ BasicJsonType materialize(const document_data& d, const node* n)
bool object;
};
// a scalar needs neither the handler nor the stack of open containers
switch (static_cast<value_t>(n->kind))
{
case value_t::string:
return BasicJsonType(string_t(d.str(*n), n->len));
case value_t::number_integer:
return BasicJsonType(static_cast<typename BasicJsonType::number_integer_t>(static_cast<std::int64_t>(integer_bits(*n))));
case value_t::number_unsigned:
return BasicJsonType(static_cast<typename BasicJsonType::number_unsigned_t>(integer_bits(*n)));
case value_t::number_float:
return BasicJsonType(float_value<typename BasicJsonType::number_float_t>(d, *n));
case value_t::boolean:
return BasicJsonType((n->flags & node_flags::is_true) != 0);
case value_t::null:
return BasicJsonType(nullptr);
case value_t::object:
case value_t::array:
case value_t::binary:
case value_t::discarded:
default:
break;
}
BasicJsonType result;
sax_t sax(result, true);
const string_t no_token{};
// (shallow subtrees on the stack: a call for a small value then allocates only for the value)
std::array<frame, 16> shallow; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled as containers open
std::vector<frame> deep{};
frame* open = shallow.data();
std::size_t capacity = shallow.size();
if (d.max_depth > capacity)
{
deep.resize(d.max_depth);
open = deep.data();
capacity = deep.size();
}
std::vector<frame> open((std::max)(d.max_depth, std::size_t{1}));
std::size_t depth = 0;
for (;;)
{
@@ -6087,13 +5806,9 @@ BasicJsonType materialize(const document_data& d, const node* n)
{
sax.start_array(n->len);
}
if (NLOHMANN_VIEW_UNLIKELY(depth == capacity))
if (NLOHMANN_VIEW_UNLIKELY(depth == open.size()))
{
// (deeper than the recorded nesting, e.g. after edits)
deep.assign(open, open + depth);
deep.resize(depth * 2);
open = deep.data();
capacity = deep.size();
open.resize(depth * 2); // (deeper than the recorded nesting, e.g. after edits)
}
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object};
break;
@@ -7875,7 +7590,7 @@ class basic_json_view
}
detail::view::throw_type_error(305, "cannot use operator[] with a numeric argument with ", type_name());
}
return idx < m_node->len ? element(idx) : basic_json_view();
return idx < m_node->len ? basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx))) : basic_json_view();
}
/// any other integer type (int, unsigned, long, std::int64_t, ...; a
@@ -7938,7 +7653,7 @@ class basic_json_view
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return element(idx);
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
}
/// any other integer type, see operator[]; negative values are out of
@@ -7972,13 +7687,7 @@ class basic_json_view
string_t value(string_view_t key, const char* default_value) const
{
if (NLOHMANN_VIEW_UNLIKELY(!is_object()))
{
detail::view::throw_type_error(306, "cannot use value() with ", type_name());
}
// (the default is made only if there is no such member)
const basic_json_view r = lookup(key);
return r.is_discarded() ? string_t(default_value) : r.template get<string_t>();
return value(key, string_t(default_value));
}
/// the value a JSON pointer refers to converted to T, or the default
@@ -7997,12 +7706,7 @@ class basic_json_view
string_t value(const json_pointer& ptr, const char* default_value) const
{
if (NLOHMANN_VIEW_UNLIKELY(!is_structured()))
{
detail::view::throw_type_error(306, "cannot use value() with ", type_name());
}
const basic_json_view r = detail::view::resolve_pointer(*this, detail::json_pointer_access::reference_tokens(ptr), detail::view::pointer_mode::value);
return r.is_discarded() ? string_t(default_value) : r.template get<string_t>();
return value(ptr, string_t(default_value));
}
/// the first element or member value; a primitive value itself. Throws
@@ -8024,8 +7728,7 @@ class basic_json_view
{
if (is_structured() && m_node->len != 0)
{
return unedited() ? basic_json_view(m_doc, detail::view::last_child<false>(*m_doc, m_node) + (is_object() ? 1 : 0))
: basic_json_view(m_doc, navigation::value(detail::view::last_child<Editable>(*m_doc, m_node) + (is_object() ? 1 : 0)));
return basic_json_view(m_doc, navigation::value(detail::view::last_child<Editable>(*m_doc, m_node) + (is_object() ? 1 : 0)));
}
return front();
}
@@ -8042,8 +7745,7 @@ class basic_json_view
{
return end();
}
const node* const k = unedited() ? detail::view::find_member<false>(*m_doc, m_node, key.data(), key.size())
: detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? iterator(m_doc, k, true) : end();
}
@@ -8060,8 +7762,7 @@ class basic_json_view
/// whether this is an object with a member with this key
bool contains(string_view_t key) const
{
return is_object() && (unedited() ? detail::view::find_member<false>(*m_doc, m_node, key.data(), key.size()) != nullptr
: detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size()) != nullptr);
return is_object() && detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size()) != nullptr;
}
bool contains(const char* key) const
@@ -8221,14 +7922,7 @@ class basic_json_view
// the compact writer keeps 64 bytes of slack, so that it does not grow
// the buffer just before the end
const std::size_t estimate = source_extent() + (style.pretty ? source_extent() / 2 : 0) + 160;
if (unedited())
{
detail::view::view_serializer<BasicJsonType, false>(*m_doc, out, estimate, style).dump(m_node);
}
else
{
detail::view::view_serializer<BasicJsonType, Editable>(*m_doc, out, estimate, style).dump(m_node);
}
detail::view::view_serializer<BasicJsonType, Editable>(*m_doc, out, estimate, style).dump(m_node);
return out;
}
@@ -8308,7 +8002,7 @@ class basic_json_view
{
return BasicJsonType(value_t::discarded);
}
return unedited() ? detail::view::materialize<BasicJsonType, false>(*m_doc, m_node) : detail::view::materialize<BasicJsonType, Editable>(*m_doc, m_node);
return detail::view::materialize<BasicJsonType, Editable>(*m_doc, m_node);
}
/// byte offset of this value in the source text (for strings: of the
@@ -8386,32 +8080,10 @@ class basic_json_view
/// (object required)
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view lookup(string_view_t key) const noexcept
{
if (unedited())
{
const node* const k = detail::view::find_member<false>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, k + 1) : basic_json_view();
}
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, navigation::value(k + 1)) : basic_json_view();
}
/// whether the document has not been edited yet: until the first edit, its index is walked like that of a
/// read-only document (nothing is moved or linked), which is cheaper
NLOHMANN_VIEW_ALWAYS_INLINE bool unedited() const noexcept
{
return !navigation::editable() || m_doc->edits == nullptr;
}
/// the element of an array at an index below its size
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view element(size_type idx) const
{
if (unedited())
{
return basic_json_view(m_doc, detail::view::element_at<false>(*m_doc, m_node, idx));
}
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
}
// --- get() dispatch ---
bool get_impl(detail::view::value_tag<bool> /*unused*/, detail::priority_tag<2> /*unused*/) const
@@ -9151,8 +8823,7 @@ class basic_json_document
// NOLINTBEGIN(readability-avoid-nested-conditional-operator): a constant expression of C++11
append_input_impl(buf, std::forward<T>(input), std::integral_constant < int, detail::is_contiguous_byte_container<D>::value ? 0
: std::is_base_of<std::istream, D>::value ? 1
: detail::view::is_adapter_input<D>::value ? 2
: detail::view::is_c_stream<D>::value ? 4 : 3 > {});
: detail::view::is_adapter_input<D>::value ? 2 : 3 > {});
// NOLINTEND(readability-avoid-nested-conditional-operator)
}
@@ -9174,19 +8845,6 @@ class basic_json_document
detail::view::append_adapter(buf, std::move(input.ia));
}
#ifndef JSON_NO_IO
static void append_input_impl(std::string& buf, std::FILE* file, std::integral_constant<int, 4> /*C stream*/)
{
if (file == nullptr)
{
// (as before: the library's adapter decides what a null stream yields)
detail::view::append_adapter(buf, detail::input_adapter(file));
return;
}
detail::view::append_file(buf, file);
}
#endif
template<typename T>
static void append_input_impl(std::string& buf, T&& input, std::integral_constant<int, 3> /*other*/)
{
+272
View File
@@ -492,3 +492,275 @@ TEST_CASE("binary_reserve_hint never over-reserves")
CHECK(nlohmann::detail::binary_reserve_hint(json({{"a", 1}, {"b", 2}})) == 5);
}
}
namespace
{
// numbers around every boundary of the integer encodings, and floats of each
// kind (exact in float, needing double, special values)
std::vector<json> boundary_numbers()
{
std::vector<json> numbers;
for (int k = 0; k < 64; ++k)
{
const std::uint64_t p = std::uint64_t{1} << k;
for (const std::uint64_t d :
{
p - 1, p, p + 1
})
{
numbers.emplace_back(d);
if (d <= static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)()))
{
numbers.emplace_back(static_cast<std::int64_t>(d));
numbers.emplace_back(-static_cast<std::int64_t>(d));
numbers.emplace_back(-static_cast<std::int64_t>(d) - 1);
}
}
}
numbers.emplace_back((std::numeric_limits<std::int64_t>::min)());
numbers.emplace_back((std::numeric_limits<std::uint64_t>::max)());
for (const double f :
{
0.0, -0.0, 1.0, -1.0, 0.5, 0.1, 3.25, 1e300, -1e-300, 3.4028234663852886e38, 1e39,
std::numeric_limits<double>::infinity(), -std::numeric_limits<double>::infinity(),
std::numeric_limits<double>::quiet_NaN()
})
{
numbers.emplace_back(f);
}
return numbers;
}
// the concatenation of the encodings of the elements of @a numbers[first, first + count)
template<typename Single>
std::vector<std::uint8_t> concat_singles(const std::vector<json>& numbers, const std::size_t first, const std::size_t count, Single single)
{
std::vector<std::uint8_t> bytes;
for (std::size_t i = first; i < first + count; ++i)
{
const auto one = single(numbers[i]);
bytes.insert(bytes.end(), one.begin(), one.end());
}
return bytes;
}
} // namespace
// Runs of numbers in arrays are encoded straight into space reserved from the
// sink, by code that is separate from the encoding of a single value. Both have
// to agree on every number: an array is its head and the encodings of its
// elements, one after the other.
TEST_CASE("runs of numbers in arrays are encoded as the numbers one by one")
{
const std::vector<json> numbers = boundary_numbers();
// arrays of every length that a one-byte head covers, so that runs are as
// long as, and longer and shorter than, what fits one reservation
for (const std::size_t count :
{
1u, 2u, 3u, 5u, 14u, 15u, 23u
})
{
for (std::size_t first = 0; first + count <= numbers.size(); first += 7)
{
const json array(std::vector<json>(numbers.begin() + static_cast<std::ptrdiff_t>(first),
numbers.begin() + static_cast<std::ptrdiff_t>(first + count)));
CAPTURE(array.dump(-1, ' ', false, json::error_handler_t::replace))
std::vector<std::uint8_t> expected{static_cast<std::uint8_t>(0x80 + count)};
auto elements = concat_singles(numbers, first, count, [](const json & j)
{
return json::to_cbor(j);
});
expected.insert(expected.end(), elements.begin(), elements.end());
CHECK(json::to_cbor(array) == expected);
expected.assign(1, static_cast<std::uint8_t>(count <= 15 ? 0x90 + count : 0xDC));
if (count > 15)
{
expected.push_back(0);
expected.push_back(static_cast<std::uint8_t>(count));
}
elements = concat_singles(numbers, first, count, [](const json & j)
{
return json::to_msgpack(j);
});
expected.insert(expected.end(), elements.begin(), elements.end());
CHECK(json::to_msgpack(array) == expected);
for (const bool bjdata :
{
false, true
})
{
expected.assign(1, static_cast<std::uint8_t>('['));
elements = concat_singles(numbers, first, count, [bjdata](const json & j)
{
return bjdata ? json::to_bjdata(j) : json::to_ubjson(j);
});
expected.insert(expected.end(), elements.begin(), elements.end());
expected.push_back(static_cast<std::uint8_t>(']'));
CHECK((bjdata ? json::to_bjdata(array) : json::to_ubjson(array)) == expected);
}
// BON8 has no integers above the int64 range
bool representable = true;
for (std::size_t i = first; i < first + count; ++i)
{
representable = representable && bon8_representable(numbers[i]);
}
if (representable)
{
expected.assign(1, static_cast<std::uint8_t>(count <= 4 ? 0x80 + count : 0x85));
elements = concat_singles(numbers, first, count, [](const json & j)
{
return json::to_bon8(j);
});
expected.insert(expected.end(), elements.begin(), elements.end());
if (count > 4)
{
expected.push_back(0xFE);
}
CHECK(json::to_bon8(array) == expected);
}
else
{
CHECK_THROWS_AS(json::to_bon8(array), json::out_of_range&);
}
}
}
}
// An array that is written in more than one reservation (more numbers than fit
// one), and arrays of an index with more digits, round trip for every format.
TEST_CASE("large arrays and many digits of indices round trip")
{
json array = json::array();
for (int i = 0; i < 12000; ++i)
{
switch (i % 4)
{
case 0:
array.push_back(i);
break;
case 1:
array.push_back(-i);
break;
case 2:
array.push_back(i * 0.25);
break;
default:
array.push_back(i % 8 == 3);
break;
}
}
json document = json::object();
document["numbers"] = array;
document["nested"] = json::array({array, "text", array});
CHECK(json::from_cbor(json::to_cbor(document)) == document);
CHECK(json::from_msgpack(json::to_msgpack(document)) == document);
CHECK(json::from_ubjson(json::to_ubjson(document)) == document);
CHECK(json::from_ubjson(json::to_ubjson(document, true, true)) == document);
CHECK(json::from_bjdata(json::to_bjdata(document)) == document);
CHECK(json::from_bson(json::to_bson(document)) == document);
CHECK(json::from_bon8(json::to_bon8(document)) == document);
}
// Strings and keys are written with their head in one reservation up to a
// length; around that length, and with every kind of head, they must read back.
TEST_CASE("strings of every head length round trip")
{
json document = json::object();
json array = json::array();
for (const std::size_t length :
{
0u, 1u, 2u, 3u, 4u, 7u, 8u, 15u, 16u, 17u, 23u, 24u, 25u, 31u, 32u, 33u, 63u, 127u, 128u, 129u, 200u, 245u, 246u, 247u, 248u, 249u, 250u, 251u, 252u, 253u, 254u, 255u, 256u, 257u, 300u, 1000u, 65535u, 65536u, 70000u
})
{
const std::string text(length, 'x');
array.push_back(text);
document[std::string(length, 'k')] = text;
}
document["array"] = array;
CHECK(json::from_cbor(json::to_cbor(document)) == document);
CHECK(json::from_msgpack(json::to_msgpack(document)) == document);
CHECK(json::from_ubjson(json::to_ubjson(document)) == document);
CHECK(json::from_ubjson(json::to_ubjson(document, true, true)) == document);
CHECK(json::from_bjdata(json::to_bjdata(document)) == document);
CHECK(json::from_bson(json::to_bson(document)) == document);
CHECK(json::from_bon8(json::to_bon8(document)) == document);
}
// What a write that throws leaves behind in an output adapter is what was written
// before it, whatever the adapter and however the adapter sink buffers.
TEST_CASE("a write that throws leaves the same bytes in every kind of output")
{
json array = json::array();
for (int i = 0; i < 700; ++i)
{
array.push_back(i * 1000);
array.push_back("text " + std::to_string(i));
}
array.push_back("bad \xFF byte");
json document = json::object();
document["first"] = array;
document["second"] = 1;
const auto strict = json::error_handler_t::strict;
SECTION("cbor")
{
std::vector<std::uint8_t> as_vector;
std::string as_string;
std::ostringstream as_stream;
CHECK_THROWS_AS(json::to_cbor(document, as_vector, strict), json::type_error&);
CHECK_THROWS_AS(json::to_cbor(document, as_string, strict), json::type_error&);
CHECK_THROWS_AS(json::to_cbor(document, as_stream, strict), json::type_error&);
CHECK(!as_vector.empty());
CHECK(std::string(as_vector.begin(), as_vector.end()) == as_string);
CHECK(as_stream.str() == as_string);
}
SECTION("msgpack")
{
std::vector<std::uint8_t> as_vector;
std::string as_string;
std::ostringstream as_stream;
CHECK_THROWS_AS(json::to_msgpack(document, as_vector, strict), json::type_error&);
CHECK_THROWS_AS(json::to_msgpack(document, as_string, strict), json::type_error&);
CHECK_THROWS_AS(json::to_msgpack(document, as_stream, strict), json::type_error&);
CHECK(!as_vector.empty());
CHECK(std::string(as_vector.begin(), as_vector.end()) == as_string);
CHECK(as_stream.str() == as_string);
}
SECTION("ubjson")
{
std::vector<std::uint8_t> as_vector;
std::string as_string;
std::ostringstream as_stream;
CHECK_THROWS_AS(json::to_ubjson(document, as_vector, false, false, strict), json::type_error&);
CHECK_THROWS_AS(json::to_ubjson(document, as_string, false, false, strict), json::type_error&);
CHECK_THROWS_AS(json::to_ubjson(document, as_stream, false, false, strict), json::type_error&);
CHECK(!as_vector.empty());
CHECK(std::string(as_vector.begin(), as_vector.end()) == as_string);
CHECK(as_stream.str() == as_string);
}
SECTION("bson")
{
// the size pass finds the string before anything is written
std::vector<std::uint8_t> as_vector;
std::string as_string;
std::ostringstream as_stream;
CHECK_THROWS_AS(json::to_bson(document, as_vector, strict), json::type_error&);
CHECK_THROWS_AS(json::to_bson(document, as_string, strict), json::type_error&);
CHECK_THROWS_AS(json::to_bson(document, as_stream, strict), json::type_error&);
CHECK(as_vector.empty());
CHECK(as_string.empty());
CHECK(as_stream.str().empty());
}
}
+49
View File
@@ -285,6 +285,55 @@ TEST_CASE("custom number types: integers beyond 64 bits")
CHECK(wide_json::to_bjdata(wide_json::array({uint_big, uint_big}), true, true) == expected_bjdata_array);
}
SECTION("integers in arrays and objects")
{
// runs of numbers and scalar object members are encoded by loops of
// their own for the number types up to 64 bits; a wider number type
// takes the generic path, which checks the range of every integer
const wide_json run = wide_json::array({make_int(1), make_int(-3), make_uint(7), wide_json(1.5), int_big, make_int(5)});
const wide_json run_negative = wide_json::array({make_int(1), make_int(-3), int_big_negative});
const wide_json run_unsigned = wide_json::array({make_uint(1), make_uint(7), uint_big});
const wide_json members = wide_json::object({{"a", make_int(1)}, {"b", wide_json(1.5)}, {"c", int_big}});
const std::vector<wide_json> failing = {run, run_negative, run_unsigned, members, wrap(run), wrap(members)};
for (const auto& value : failing)
{
CHECK_THROWS_AS(_ = wide_json::to_cbor(value), out_of_range&);
CHECK_THROWS_AS(_ = wide_json::to_msgpack(value), out_of_range&);
CHECK_THROWS_AS(_ = wide_json::to_bson(wrap(value)), out_of_range&);
CHECK_THROWS_AS(_ = wide_json::to_bon8(value), out_of_range&);
}
// a failing write leaves no more than the elements before the failing one
std::vector<std::uint8_t> out;
CHECK_THROWS_AS(wide_json::to_cbor(run, out), out_of_range&);
// (the head of the array, 1, -3, 7, and 1.5 as a single-precision float)
CHECK(out == std::vector<std::uint8_t>({0x86, 0x01, 0x22, 0x07, 0xFA, 0x3F, 0xC0, 0x00, 0x00}));
out.clear();
CHECK_THROWS_AS(wide_json::to_bon8(run, out), out_of_range&);
CHECK(!out.empty());
// the same values that fit are written exactly as with 64-bit number types
const wide_json fits = wide_json::array({make_int(1), make_int(-3), make_uint(7), wide_json(1.5), make_int(min_int64), make_int(max_int64), make_int(200), make_int(-70000)});
const nlohmann::json fits64 = nlohmann::json::array({1, -3, 7, 1.5, (std::numeric_limits<std::int64_t>::min)(), (std::numeric_limits<std::int64_t>::max)(), 200, -70000});
CHECK(wide_json::to_cbor(fits) == nlohmann::json::to_cbor(fits64));
CHECK(wide_json::to_msgpack(fits) == nlohmann::json::to_msgpack(fits64));
CHECK(wide_json::to_ubjson(fits) == nlohmann::json::to_ubjson(fits64));
CHECK(wide_json::to_ubjson(fits, true, true) == nlohmann::json::to_ubjson(fits64, true, true));
CHECK(wide_json::to_bjdata(fits) == nlohmann::json::to_bjdata(fits64));
CHECK(wide_json::to_bjdata(fits, true, true) == nlohmann::json::to_bjdata(fits64, true, true));
CHECK(wide_json::to_bon8(fits) == nlohmann::json::to_bon8(fits64));
CHECK(wide_json::to_bson(wrap(fits)) == nlohmann::json::to_bson(nlohmann::json::object({{"v", fits64}})));
const wide_json fit_members = wide_json::object({{"a", make_int(1)}, {"b", make_int(-300)}, {"c", make_uint(max_uint64)}, {"d", wide_json(2.5)}});
const nlohmann::json fit_members64 = nlohmann::json::object({{"a", 1}, {"b", -300}, {"c", (std::numeric_limits<std::uint64_t>::max)()}, {"d", 2.5}});
CHECK(wide_json::to_cbor(fit_members) == nlohmann::json::to_cbor(fit_members64));
CHECK(wide_json::to_msgpack(fit_members) == nlohmann::json::to_msgpack(fit_members64));
CHECK(wide_json::to_ubjson(fit_members) == nlohmann::json::to_ubjson(fit_members64));
CHECK(wide_json::to_bjdata(fit_members) == nlohmann::json::to_bjdata(fit_members64));
CHECK(wide_json::to_bson(fit_members) == nlohmann::json::to_bson(fit_members64));
}
SECTION("BJData ND-array")
{
// an ND-array element beyond 64 bits does not fit any dtype, so the
-123
View File
@@ -185,29 +185,6 @@ TEST_CASE("json_view")
// duplicate keys: the last value, at the position of the first key
CHECK(materialized<json_document>(R"({"a":1,"b":2,"a":3})") == json::parse(R"({"a":1,"b":2,"a":3})"));
CHECK(materialized<ordered_json_document>(R"({"a":1,"b":2,"a":3})").dump() == R"({"a":3,"b":2})");
// scalars (made without the handler), and nesting around the depth up to which the stack of open containers is local
for (const char* scalar :
{"null", "true", "false", "0", "-7", "18446744073709551615", "1.5e300", "-0.0", "\"\"", "\"short\"", "\"a string that is longer than fifteen bytes\"", "\"\\u00e4\\n\""
})
{
CAPTURE(scalar)
CHECK(materialized<json_document>(scalar) == json::parse(scalar));
const std::string in_array = std::string("[1,") + scalar + "]";
const json_document d = json_document::parse(in_array);
CHECK(d.root()[1].materialize() == json::parse(scalar));
CHECK(d.root()[1].get<json>() == json::parse(scalar));
}
for (const std::size_t levels :
{
std::size_t{1}, std::size_t{15}, std::size_t{16}, std::size_t{17}, std::size_t{32}, std::size_t{33}
})
{
CAPTURE(levels)
const std::string nested = std::string(levels, '[') + "{\"k\":[1,{\"j\":2}]}" + std::string(levels, ']');
CHECK(materialized<json_document>(nested) == json::parse(nested));
const std::string wide = "{\"a\":" + nested + ",\"b\":" + nested + "}";
CHECK(materialized<json_document>(wide) == json::parse(wide));
}
// very deep nesting (iterative, as parse())
const std::string deep = std::string(100000, '[') + std::string(100000, ']');
// (one check per step, so that a failure shows which step it is)
@@ -433,31 +410,6 @@ TEST_CASE("json_view")
std::istringstream rvalue_source("[1]");
CHECK(json_document::parse_copy(std::move(rvalue_source)).owns_source());
// a C stream is read in blocks as well (more than one block, and a
// text whose length is a multiple of the block size)
for (const std::size_t extra :
{
std::size_t{0}, std::size_t{1}
})
{
std::FILE* long_file = std::tmpfile(); // NOLINT(cppcoreguidelines-owning-memory)
if (long_file != nullptr)
{
std::string padded = text;
padded.append((4096 - (padded.size() % 4096)) % 4096 + extra, ' ');
CHECK(std::fwrite(padded.data(), 1, padded.size(), long_file) == padded.size());
CHECK(std::fseek(long_file, 0, SEEK_SET) == 0);
const json_document from_long_file = json_document::parse(long_file);
CHECK(from_long_file.source() == padded);
CHECK(from_long_file.root().materialize() == json::parse(text));
CHECK(std::fclose(long_file) == 0); // NOLINT(cppcoreguidelines-owning-memory)
}
}
{
json_document from_null;
CHECK_THROWS_AS(from_null = json_document::parse(static_cast<std::FILE*>(nullptr)), json::parse_error&);
}
// a stream without a buffer is an empty input
std::istream no_buffer(nullptr);
CHECK(json_document::parse(no_buffer, false).is_discarded());
@@ -1054,49 +1006,6 @@ TEST_CASE("json_view element access and iteration")
#endif
}
SECTION("arrays whose elements have no subtrees: index and back()")
{
// (the elements are adjacent in the index; an array, an object, or an
// empty container anywhere makes the lookup skip element by element)
const std::vector<std::string> texts =
{
"[]", "[1]", "[1,2,3]", "[1,\"two\",3.5,true,null,-7,18446744073709551615]", "[[]]", "[{}]", "[1,[2],3]",
"[1,2,{\"a\":3}]", "[[1,2],3,4]", "[1,2,3,[4,5,6,7,8],9,10,11,12]", "[{\"a\":[1]},2]", "[\"a\\u00e4\",\"b\"]"
};
for (const std::string& text : texts)
{
CAPTURE(text)
const json_document d = json_document::parse(text);
const json_view v = d.root();
const json expected = json::parse(text);
for (std::size_t i = 0; i < expected.size(); ++i)
{
CHECK(v[i].materialize() == expected[i]);
CHECK(v.at(i).materialize() == expected[i]);
}
CHECK(v[expected.size()].is_discarded());
if (!expected.empty())
{
CHECK(v.back().materialize() == expected.back());
}
}
// a long array (the direct lookup is not limited to short ones), also with a container at its end
std::string text = "[0";
for (int i = 1; i < 1000; ++i)
{
text += "," + std::to_string(i);
}
const json_document scalars = json_document::parse(text + "]");
CHECK(scalars.root()[999].get<int>() == 999);
CHECK(scalars.root()[500].get<int>() == 500);
CHECK(scalars.root().at(1).get<int>() == 1);
CHECK(scalars.root().back().get<int>() == 999);
const json_document container_last = json_document::parse(text + ",[1000]]");
CHECK(container_last.root()[999].get<int>() == 999);
CHECK(container_last.root()[1000][0].get<int>() == 1000);
CHECK(container_last.root().back()[0].get<int>() == 1000);
}
SECTION("iterators")
{
const json_document d = json_document::parse(R"({"x":[1,{"y":2}],"z":null})");
@@ -1136,28 +1045,6 @@ TEST_CASE("json_view element access and iteration")
CHECK((*first++).key() == "0");
CHECK((*first).key() == "1");
CHECK(++first == items.end());
// the keys of array elements are their indexes, whether or not they are asked for
std::string numbers = "[0";
for (int i = 1; i < 1234; ++i)
{
numbers += "," + std::to_string(i);
}
const json_document array = json_document::parse(numbers + "]");
std::size_t position = 0;
for (const auto& item : array.root().items())
{
CHECK(item.value().get<std::size_t>() == position);
if (position % 3 != 0) // (not for every item: values alone do not make the key)
{
CHECK(item.key() == std::to_string(position));
CHECK(item.key().data() == item.key().data()); // (the same text every time)
const json_view::item copy = item;
CHECK(copy.key() == std::to_string(position));
}
++position;
}
CHECK(position == 1234);
#ifdef JSON_HAS_CPP_17
std::string pairs;
for (const auto [key, value] : d.root().items())
@@ -1496,17 +1383,7 @@ TEST_CASE("json_view values")
CHECK(exception_of([&] { static_cast<void>(v.value("s", 0)); }) == without_path(exception_of([&] { static_cast<void>(j.value("s", 0)); })));
CHECK(exception_of([&] { static_cast<void>(v["n"].value("x", 0)); }) == without_path(exception_of([&] { static_cast<void>(j["n"].value("x", 0)); })));
CHECK(exception_of([&] { static_cast<void>(v["n"].value(json::json_pointer("/x"), 0)); }) == without_path(exception_of([&] { static_cast<void>(j["n"].value(json::json_pointer("/x"), 0)); })));
// string defaults: the member is not a string, or the value is neither an object nor an array
CHECK(exception_of([&] { static_cast<void>(v.value("n", "default")); }) == without_path(exception_of([&] { static_cast<void>(j.value("n", "default")); })));
CHECK(exception_of([&] { static_cast<void>(v["n"].value("x", "default")); }) == without_path(exception_of([&] { static_cast<void>(j["n"].value("x", "default")); })));
CHECK(exception_of([&] { static_cast<void>(v.value(json::json_pointer("/n"), "default")); }) == without_path(exception_of([&] { static_cast<void>(j.value(json::json_pointer("/n"), "default")); })));
CHECK(exception_of([&] { static_cast<void>(v["s"].value(json::json_pointer("/x"), "default")); }) == without_path(exception_of([&] { static_cast<void>(j["s"].value(json::json_pointer("/x"), "default")); })));
#endif
// (defaults and members longer than the short string optimization)
CHECK(v.value("s", "a default longer than fifteen characters") == "text");
CHECK(v.value("missing", "a default longer than fifteen characters") == "a default longer than fifteen characters");
CHECK(v.value(json::json_pointer("/s"), "a default longer than fifteen characters") == "text");
CHECK(v.value(json::json_pointer("/zz"), "a default longer than fifteen characters") == "a default longer than fifteen characters");
}
}
-52
View File
@@ -523,58 +523,6 @@ TEST_CASE("json_view builder: strings across vector blocks")
}
}
TEST_CASE("json_view builder: escapes between runs of text")
{
// The text between escapes is copied 16 bytes at a time where there are
// that many bytes left, and with the UTF-8 check where there is
// non-ASCII text. Runs of every length from 0 to 40 between valid and
// invalid escapes, UTF-8 sequences, and control characters, in strings
// that are cut at all places near the end of the text.
const std::array<const char*, 9> valid = {{"\\n", "\\\"", "\\\\", "\\/", "\\t", "\\u00e4", "\\u20ac", "\\ud83d\\ude00", "\\u0000"}};
const std::array<const char*, 9> invalid = {{"\\x", "\\ud800", "\\udc00", "\\u12G4", "\\u12", "\\ud83dx", "\\ud83d\\u0041", "\\", "\x01"}};
const std::array<const char*, 6> utf8 = {{"\xc3\xa4", "\xe2\x82\xac", "\xf0\x9f\x98\x80", "\xc3", "\xff", "\x80"}};
std::mt19937 rng(2026); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
for (std::size_t run = 0; run <= 40; ++run)
{
for (int i = 0; i < 60; ++i)
{
std::string content = std::string(rng() % 20, 'a');
const auto pieces = 1 + (rng() % 6);
for (unsigned k = 0; k < pieces; ++k)
{
content += valid[rng() % valid.size()];
content += std::string(k == 0 ? run : rng() % 41, static_cast<char>('b' + (rng() % 20)));
switch (rng() % 8)
{
case 0:
content += utf8[rng() % 3];
break;
case 1:
content += utf8[rng() % utf8.size()];
break;
case 2:
content += invalid[rng() % invalid.size()];
break;
default:
break;
}
}
check_string(content);
check_string(content.substr(0, content.size() - (rng() % 4)));
}
}
// the text before the first escape, and runs and escapes at the end of the text
for (std::size_t before = 0; before <= 40; ++before)
{
for (const char* escape : valid)
{
check_string(std::string(before, 'a') + escape);
check_string(std::string(before, 'a') + escape + std::string(before % 19, 'z'));
check_string(std::string(before, 'a') + escape + escape + std::string(before % 19, 'z'));
}
}
}
TEST_CASE("json_view node integer bits")
{
using nlohmann::detail::view::integer_bits;
-138
View File
@@ -21,7 +21,6 @@ using nlohmann::ordered_json_view;
using json_view_test::generator;
using json_view_test::has_duplicate_keys;
#include <algorithm>
#include <array>
#include <cmath>
#include <cstddef>
@@ -30,7 +29,6 @@ using json_view_test::has_duplicate_keys;
#include <iomanip>
#include <map>
#include <random>
#include <set>
#include <sstream>
#include <string>
#include <vector>
@@ -600,142 +598,6 @@ TEST_CASE("json_view large objects")
CHECK(without_collisions.memory_usage() >= with_collisions.memory_usage() + (table_size * sizeof(std::uint32_t)));
}
SECTION("keys that differ in their last bytes are spread over the table")
{
// The hash must spread the usual families of keys (numbered names, hex
// strings, dates, short names): a key farther than index_max_displacement
// from its home slot would leave the object without a table. The keys
// are placed with linear probing, as the table does.
const auto max_displacement = [](const std::vector<std::string>& keys)
{
std::size_t slots = 16;
while (slots < 2 * keys.size())
{
slots *= 2;
}
std::vector<bool> used(slots, false);
std::size_t farthest = 0;
for (const std::string& key : keys)
{
std::size_t i = static_cast<std::size_t>(nlohmann::detail::view::key_hash(key.data(), key.size())) & (slots - 1);
std::size_t distance = 0;
while (used[i])
{
i = (i + 1) & (slots - 1);
++distance;
}
used[i] = true;
farthest = (std::max)(farthest, distance);
}
return farthest;
};
// the keys of a printf-like pattern with one %d, %0Nd or %0Nx, formatted with a stream
// (a format string that is not a literal is a Flawfinder error)
const auto numbered = [](const char* format, int count, int step)
{
const std::string pattern(format);
const std::size_t percent = pattern.find('%');
std::size_t conversion = percent + 1;
std::size_t width = 0;
while (pattern[conversion] >= '0' && pattern[conversion] <= '9')
{
width = width * 10 + static_cast<std::size_t>(pattern[conversion] - '0');
++conversion;
}
const bool hex = pattern[conversion] == 'x';
std::vector<std::string> keys;
for (int i = 0; i < count; ++i)
{
std::ostringstream key;
key << pattern.substr(0, percent) << std::setfill('0') << std::setw(static_cast<int>(width))
<< (hex ? std::hex : std::dec) << i * step << pattern.substr(conversion + 1);
keys.push_back(key.str());
}
return keys;
};
for (const char* format :
{"key%d", "%d", "item_%04d", "a%05d", "%03dxyz", "id%d", "user_%d", "%d_suffix", "0x%04x", "A%07d", "12345678%d", "1234567%d", "123456789%d",
"configuration_parameter_value_%04d_extra", "2026-01-%02dT00:00:00Z"
})
{
CAPTURE(format)
CHECK(max_displacement(numbered(format, std::string(format).find("%02d") != std::string::npos ? 60 : 1000, 1)) < nlohmann::detail::view::index_max_displacement);
CHECK(max_displacement(numbered(format, 600, 7)) < nlohmann::detail::view::index_max_displacement);
}
std::mt19937_64 rng(7);
for (const std::size_t length :
{
std::size_t{1}, std::size_t{2}, std::size_t{3}, std::size_t{5}, std::size_t{7}, std::size_t{8}, std::size_t{9}, std::size_t{12}, std::size_t{16}, std::size_t{17}, std::size_t{33}
})
{
CAPTURE(length)
std::set<std::string> distinct;
while (distinct.size() < (length == 1 ? 26 : (length == 2 ? 500 : 700))) // (there are 26 and 676 keys of one and two letters)
{
std::string key(length, 'a');
for (char& c : key)
{
c = static_cast<char>('a' + (rng() % 26));
}
distinct.insert(key);
}
CHECK(max_displacement(std::vector<std::string>(distinct.begin(), distinct.end())) < nlohmann::detail::view::index_max_displacement);
}
}
SECTION("keys of every length that differ in one byte")
{
// In a large object, a key is found by comparing its bytes with those of the member the
// table points to: up to 16 bytes as two overlapping words, longer ones with memcmp.
// Keys of every length from 0 to 40, and keys that differ from them in one byte.
std::mt19937_64 rng(2026);
std::vector<std::string> keys;
std::set<std::string> distinct;
for (std::size_t length = 0; length <= 40; ++length)
{
for (int copy = 0; copy < 5; ++copy)
{
std::string key(length, 'a');
for (char& c : key)
{
c = static_cast<char>('a' + (rng() % 26));
}
if (distinct.insert(key).second)
{
keys.push_back(key);
}
}
}
std::string text = "{";
for (std::size_t i = 0; i < keys.size(); ++i)
{
text += (i != 0 ? ",\"" : "\"") + keys[i] + "\":" + std::to_string(i);
}
text += "}";
REQUIRE(keys.size() >= 128); // (a table is built for 128 members)
const json_document d = json_document::parse(text);
const json_view v = d.root();
for (std::size_t i = 0; i < keys.size(); ++i)
{
CAPTURE(keys[i])
CHECK(v[keys[i]].get<std::size_t>() == i);
CHECK(v.at(keys[i]).get<std::size_t>() == i);
CHECK(v.contains(keys[i]));
for (std::size_t at = 0; at < keys[i].size(); ++at)
{
std::string other = keys[i];
other[at] = other[at] == 'z' ? 'y' : 'z';
CAPTURE(other)
CHECK(v.contains(other) == (distinct.count(other) != 0));
}
CHECK(v.contains(keys[i] + "a") == (distinct.count(keys[i] + "a") != 0));
if (!keys[i].empty())
{
CHECK(v.contains(keys[i].substr(1)) == (distinct.count(keys[i].substr(1)) != 0));
}
}
}
SECTION("shrink_to_fit releases the tables' spare capacity")
{
const auto make_text = [](int objects, int members)
-109
View File
@@ -661,115 +661,6 @@ TEST_CASE("json_view edits: replacing arrays and objects by scalars")
CHECK(d.root().materialize() == expected);
}
TEST_CASE("json_view edits: arrays whose elements have no subtrees")
{
// such an array is indexed directly while it is in its parsed layout; edits
// that replace elements by scalars keep it, others move it
json_editable_document d = json_editable_document::parse("[10,20,30,40]");
json expected = json::array({10, 20, 30, 40});
const auto check = [&]
{
CHECK(d.root().size() == expected.size());
for (std::size_t i = 0; i < expected.size(); ++i)
{
CHECK(d.root()[i].materialize() == expected[i]);
CHECK(d.root().at(i).materialize() == expected[i]);
}
CHECK(d.root()[expected.size()].is_discarded());
CHECK(d.root().back().materialize() == expected.back());
CHECK(d.root().dump() == expected.dump());
};
check();
d.set(d.root()[1], "x");
expected[1] = "x";
check();
d.set(d.root()[3], 2.5);
expected[3] = 2.5;
check();
d.set(d.root()[2], json::array({1, 2}));
expected[2] = json::array({1, 2});
check();
d.push_back(d.root(), 50);
expected.push_back(50);
check();
d.insert(d.root(), 0, true);
expected.insert(expected.begin(), true);
check();
d.erase(d.root(), 1);
expected.erase(expected.begin() + 1);
check();
d.set(d.root()[2], nullptr);
expected[2] = nullptr;
check();
}
TEST_CASE("json_view edits: materializing values nested deeper than the document was parsed")
{
// the stack of open containers of materialize() is sized from the nesting of the parsed text;
// a value added by an edit can be deeper than that (and than the stack that is local to the call)
json_editable_document d = json_editable_document::parse("[1,[2]]");
json expected = json::array({1, json::array({2})});
json nested = 3;
for (int level = 0; level < 40; ++level)
{
nested = json::array({nested, json{{"k", level}}});
}
d.push_back(d.root(), nested);
expected.push_back(nested);
CHECK(d.root().materialize() == expected);
CHECK(d.root()[2].materialize() == nested);
d.set(d.root()[1][0], nested);
expected[1][0] = nested;
CHECK(d.root().materialize() == expected);
CHECK(d.root().get<json>() == expected);
}
TEST_CASE("json_view edits: an editable document before and after its first edit")
{
// until the first edit, an editable document is read like a read-only one; views made before the edit
// keep working after it
const std::string text = R"({"a":[1,2,{"x":"y"},4],"b":{"c":true,"d":null},"e":"text","f":[]})";
const json_document read_only = json_document::parse(text);
json_editable_document d = json_editable_document::parse(text);
const auto root = d.root();
const auto same_as_read_only = [&]
{
CHECK(root.dump() == read_only.root().dump());
CHECK(root.dump(2) == read_only.root().dump(2));
CHECK(root.materialize() == read_only.root().materialize());
CHECK(root["a"][2]["x"].get_string() == "y");
CHECK(root["a"].at(3).get<int>() == 4);
CHECK(root["a"].back().get<int>() == 4);
CHECK(root.contains("b"));
CHECK(!root.contains("zz"));
CHECK(root.find("e")->get_string() == "text");
CHECK(root.find("zz") == root.end());
CHECK(root["b"]["c"].get<bool>());
CHECK(root.value("e", "none") == "text");
std::size_t count = 0;
for (const auto m : root["a"])
{
CHECK(m.materialize() == read_only.root()["a"][count].materialize());
++count;
}
CHECK(count == 4);
};
same_as_read_only();
// an edit that changes a scalar in place, then one that moves an array
d.set(root["e"], "other");
CHECK(root["e"].get_string() == "other");
CHECK(root["a"][2]["x"].get_string() == "y");
CHECK(root["a"].back().get<int>() == 4);
d.push_back(root["f"], 7);
CHECK(root["f"][0].get<int>() == 7);
CHECK(root["f"].back().get<int>() == 7);
CHECK(root.dump() == R"({"a":[1,2,{"x":"y"},4],"b":{"c":true,"d":null},"e":"other","f":[7]})");
d.set(root["a"], 2, json::array({1, 2}));
CHECK(root["a"][2][1].get<int>() == 2);
CHECK(root["a"].dump() == "[1,2,[1,2],4]");
CHECK(root.materialize() == json::parse(R"({"a":[1,2,[1,2],4],"b":{"c":true,"d":null},"e":"other","f":[7]})"));
}
namespace
{
// reads of "a" in an object that repeats it: the first member wins, however the