Compare commits

..
Author SHA1 Message Date
Niels Lohmann 86049706a0 Rename depth_counter's open() and close() to enter() and leave()
Flawfinder took them for the file functions open() and close().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 13:18:11 +02:00
Niels Lohmann cc946adf68 Find the end of the plain run of a decoded string 16 bytes at a time with SSE2
The strings of a document that had escape sequences are written in runs
without a quote, backslash, or control character; the compact dump found the
end of a run 8 bytes at a time with a SWAR test. With SSE2 (x86-64) it now
compares 16 bytes per step. NEON keeps the SWAR code: there the mask of a
step costs more than it saves on the short runs that are typical here.

Instructions of dump(), Linux (GCC 13 / Clang 18):

  twitter.json                  1.99M -> 1.88M (-5.2 %) / 1.97M -> 1.92M (-3.0 %)
  CJK text with \u escapes      5.98M -> 4.52M (-24.5 %) / 5.30M -> 4.25M (-19.9 %)
  ASCII text with a few escapes 4.28M -> 4.09M (-4.4 %) / 4.24M -> 4.11M (-3.2 %)
  jeopardy (4,000 questions)    5.08M -> 4.78M (-5.8 %) / 4.97M -> 4.80M (-3.4 %)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:30:39 +02:00
Niels Lohmann 272d07b48b Scan the strings of json_view's dump() with ensure_ascii 16 bytes at a time
With ensure_ascii, a string of the source that has a byte >= 0x7F was written
by a generic path: put() per piece and a validate-and-decode step for every
character beyond ASCII. It now goes through a writer with the cursor in a
local variable. The part of the string that has nothing to escape is found
16 bytes at a time (SSE2 or NEON; the existing 8-byte scan without vector
support) and copied; strings that are ASCII throughout are still decided by
the 8-byte check of dump() with ensure_ascii (a 16-byte scan for them was
2-5 % slower for short strings). The two-byte sequences (lead C2..DF) and
the three-byte sequences whose continuation bytes may have any value (lead
E1..EF except ED) are decoded with a few compares; E0, ED, four-byte
sequences, and damaged input go through validate_one_utf8() as before, so
damaged images still throw type_error.316.

dump(-1, ' ', true) instructions per call, macOS arm64 (develop -> now);
the compact dump without ensure_ascii and dumps of ASCII text are unchanged:

  twitter 5.42M -> 3.57M    citm 3.28M -> 3.24M     jeopardy(4k) 5.78M -> 5.65M
  CJK text, raw 66.5M -> 35.3M    escaped \u 32.7M -> 17.0M
  Latin text with accents 11.5M -> 9.4M    ASCII text 4.40M -> 4.40M

Linux instructions (GCC 13 / Clang 18), dump(-1, ' ', true):

  twitter -9 % / -33 %    CJK text -14 % / -47 %    Latin text -8 % / -18 %
  citm -0.2 % / -0.4 %    ASCII text -0.1 % / -0.3 %

The compact dump without ensure_ascii is unchanged (canada 0.0 %).

The unit test covers every character of the Basic Multilingual Plane, and a
character that needs an escape at each position of strings of 0..40 bytes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:30:36 +02:00
Niels Lohmann ef45593bda Write the indented output of json_view's dump() with the fast writer
The compact writer of dump() (cursor in a register, fixed-size copies of
tokens) served dump(-1) only; every dump(indent), and everything with
ensure_ascii that was indented, went through a generic walk that calls put()
for each token and put_repeated() (a memset call) for each line break. The
fast writer now writes indented output, too, with or without ensure_ascii, and
the generic walk is gone. A line break is a copy of a 64-byte ruler (a line
break and 63 indentation characters) when the line is at most 64 bytes, and a
store plus memset beyond that. The depth is counted only in the indented
instantiations (counting it in the compact one cost an editable document 5 %
with Clang). The indented walks are out of line (dump_pretty()): inlined
into dump() next to the compact ones, they made compact dumps of integer
arrays 7 % slower.

A decoded string (one with escape sequences) of a damaged image loaded with
image_check::bounds can contain invalid UTF-8. As before, the compact dump
writes such bytes unchanged, and the indented dump throws type_error.316; its
runs are found by the lexer's validating scan (a scalar validation of every
sequence made dump(2) of decoded CJK text twice as slow as before).

dump(2) instructions per call, macOS arm64 (before -> after):

               twitter         citm            canada          jeopardy(4k)
  json         3.75M -> 1.85M  8.28M -> 3.93M  67.1M -> 56.0M  9.56M -> 4.81M
  editable     3.82M -> 1.84M  8.48M -> 4.02M  68.1M -> 56.2M  9.66M -> 4.75M

Linux instructions (GCC 13 / Clang 18), dump(2) and dump(2, ' ', true):

  twitter      -25 % / -30 %      -11 % / -11 %
  citm         -29 % / -28 %      -24 % / -24 %
  jeopardy(4k) -25 % / -32 %      -20 % / -24 %
  CJK text with \u escapes  -6 % / -11 %   -2 % / -2 %

The compact dump is unchanged (within +0.4 % on Linux, within noise on the Mac).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:30:34 +02:00
7 changed files with 998 additions and 1272 deletions

No files matched your search

+33 -232
View File
@@ -15,7 +15,7 @@
#include <cstddef> // size_t
#include <cstdint> // uint64_t
#include <cstdlib> // strtoull
#include <cstring> // memchr, memcmp, memcpy
#include <cstring> // memcmp, memcpy
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
#endif // JSON_NO_IO
@@ -303,12 +303,6 @@ class json_pointer
exceeds_size_type ///< @a s converts to an integer that exceeds size_type
};
/// whether a reference token is "-" (a plain comparison with a string literal is not always inlined)
static bool is_dash(const string_t& s) noexcept
{
return s.size() == 1 && s[0] == '-';
}
/*!
@param[in] s reference token to be converted into an array index
@param[out] idx the integer representation of @a s if @ref array_index_status::ok
@@ -334,7 +328,7 @@ class json_pointer
// error condition (cf. RFC 6901, Sect. 4); this also covers single-
// character tokens, so "/x" and "/xy" fail alike; "-" and the empty
// token are left to the conversion below and are reported as unresolved
if (JSON_HEDLEY_UNLIKELY(!s.empty() && !is_dash(s) && !(s[0] >= '0' && s[0] <= '9')))
if (JSON_HEDLEY_UNLIKELY(!s.empty() && s != "-" && !(s[0] >= '0' && s[0] <= '9')))
{
return array_index_status::not_a_number;
}
@@ -481,40 +475,9 @@ class json_pointer
return pos == size;
}
/// reserve room for @a n elements of an array type that has reserve()
template<typename ArrayType>
static auto reserve_elements(ArrayType& array, const std::size_t n, detail::priority_tag<1> /*unused*/)
-> decltype(array.reserve(n), void())
{
array.reserve(n);
}
template<typename ArrayType>
static void reserve_elements(ArrayType& /*array*/, const std::size_t /*n*/, detail::priority_tag<0> /*unused*/)
{}
/// @brief object member access with a key that is only copied if it is inserted
///
/// Objects that can find the position of a key and insert there
/// (std::map) search once, others look the key up and add it if it is missing.
template<typename BasicJsonType>
static auto object_member_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace_hint(object.m_data.m_value.object->lower_bound(key), key, nullptr), std::declval<BasicJsonType&>())
{
auto& members = *object.m_data.m_value.object;
auto it = members.lower_bound(key);
if (it != members.end() && !members.key_comp()(key, it->first))
{
return it->second;
}
const auto old_capacity = object.container_capacity();
it = members.emplace_hint(it, key, nullptr);
return object.set_parent_after_insert(it->second, old_capacity);
}
template<typename BasicJsonType>
static BasicJsonType& object_member_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<0> /*unused*/)
static BasicJsonType& object_member(BasicJsonType& object, const string_t& key)
{
auto& members = *object.m_data.m_value.object;
const auto it = members.find(key);
@@ -525,12 +488,6 @@ class json_pointer
return object.operator[](key);
}
template<typename BasicJsonType>
static BasicJsonType& object_member(BasicJsonType& object, const string_t& key)
{
return object_member_impl(object, key, detail::priority_tag<1> {});
}
JSON_PRIVATE_UNLESS_TESTED:
json_pointer top() const
{
@@ -763,103 +720,48 @@ class json_pointer
}
}
/*!
@brief the values along the pointer last resolved by @ref resolve_plain
A patch from @ref basic_json::diff lists its operations in document order,
so consecutive pointers share most of their reference tokens; the values
along the previous pointer are kept to resume from the common prefix.
`steps[k]` is the value after the first `k` reference tokens together with
the length of the prefix of `*path` that holds them. A value stays valid as
long as no operation changes its container: callers keep only values up to
the parent of the pointer they modify, or call `reset()`.
*/
template<typename BasicJsonType>
struct plain_steps
{
const string_t* path = nullptr;
std::vector<std::pair<std::size_t, BasicJsonType*>> steps{};
void reset() noexcept
{
path = nullptr;
steps.clear();
}
};
/*!
@brief resolve a plain JSON pointer without building its reference tokens
@param[in] root the value the pointer starts at
@param[in] ptr the value the pointer starts at
@param[in] path a JSON pointer string
@param[in] length the number of leading bytes of @a path to resolve; 0 or
the position of a slash
@param[in,out] token a string to hold the reference tokens, to reuse its storage
@param[in,out] cache the values along the previously resolved pointer;
@a path must outlive the next call
@return the pointed to value, or `nullptr` if the pointer does not denote
an existing value, or does so in a way that is not plain: an array
index that is malformed or out of range, or a "-", or a reference
token with an escape. No exception is
thrown for these; callers use @ref json_pointer instead, which
handles them.
an existing value, or does so in a way that is not plain: a
reference token with an escape, an array index that is malformed or
out of range, or a "-". No exception is thrown for these; callers
use @ref json_pointer instead, which handles them.
*/
template<typename BasicJsonType>
static BasicJsonType* resolve_plain(BasicJsonType* root, const string_t& path, const std::size_t length, string_t& token, plain_steps<BasicJsonType>& cache)
static BasicJsonType* resolve_plain(BasicJsonType* ptr, const string_t& path, const std::size_t length, string_t& token)
{
auto& steps = cache.steps;
// resume from the deepest step that is also a prefix of this path
std::size_t depth = 1;
if (cache.path != nullptr && !steps.empty() && steps[0].second == root)
{
const string_t& previous = *cache.path;
for (depth = steps.size(); depth > 1; --depth)
{
// a step ends at a token boundary of the previous path
const std::size_t prefix = steps[depth - 1].first;
if (prefix <= length && (prefix == length || path[prefix] == '/') && std::memcmp(path.data(), previous.data(), prefix) == 0)
{
break;
}
}
steps.resize(depth);
}
else
{
steps.assign(1, std::make_pair(std::size_t{0}, root));
}
cache.path = &path;
BasicJsonType* ptr = steps[depth - 1].second;
for (std::size_t pos = steps[depth - 1].first; pos < length;)
for (std::size_t pos = 0; pos < length;)
{
if (JSON_HEDLEY_UNLIKELY(path[pos] != '/'))
{
return nullptr;
}
// the reference token between the slash at pos and the next one,
// which must not have an escape
const char* const begin = path.data() + pos + 1;
const char* const limit = path.data() + length;
const char* end = begin;
while (end < limit && *end != '/')
// the reference token between the slash at pos and the next one
std::size_t end = path.find_first_of('/', pos + 1);
if (end == string_t::npos || end > length)
{
if (JSON_HEDLEY_UNLIKELY(*end == '~'))
{
return nullptr;
}
++end;
end = length;
}
token.clear();
token.append(path.data() + pos + 1, end - pos - 1);
if (JSON_HEDLEY_UNLIKELY(token.find_first_of('~') != string_t::npos))
{
return nullptr;
}
switch (ptr->type())
{
case detail::value_t::object:
{
token.clear();
token.append(begin, static_cast<std::size_t>(end - begin));
const auto it = ptr->m_data.m_value.object->find(token);
if (JSON_HEDLEY_UNLIKELY(it == ptr->m_data.m_value.object->end()))
{
@@ -871,24 +773,8 @@ class json_pointer
case detail::value_t::array:
{
// an index of up to 18 digits without a leading zero fits into size_type
// (9 digits for a 32-bit size_type); everything else is left to json_pointer
constexpr std::size_t max_digits = sizeof(typename BasicJsonType::size_type) >= 8 ? 18 : 9;
const std::size_t digits = static_cast<std::size_t>(end - begin);
if (JSON_HEDLEY_UNLIKELY(digits == 0 || digits > max_digits || (digits > 1 && *begin == '0')))
{
return nullptr;
}
typename BasicJsonType::size_type idx = 0;
for (const char* c = begin; c != end; ++c)
{
if (JSON_HEDLEY_UNLIKELY(*c < '0' || *c > '9'))
{
return nullptr;
}
idx = static_cast<typename BasicJsonType::size_type>((idx * 10) + static_cast<typename BasicJsonType::size_type>(*c - '0'));
}
if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
typename BasicJsonType::size_type idx{};
if (JSON_HEDLEY_UNLIKELY(parse_array_index<BasicJsonType>(token, idx) != array_index_status::ok || idx >= ptr->m_data.m_value.array->size()))
{
return nullptr;
}
@@ -907,8 +793,7 @@ class json_pointer
default:
return nullptr;
}
pos = static_cast<std::size_t>(end - path.data());
steps.emplace_back(pos, ptr);
pos = end;
}
return ptr;
@@ -1022,7 +907,7 @@ class json_pointer
case detail::value_t::array:
{
if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" always fails the range check
JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat(
@@ -1093,7 +978,7 @@ class json_pointer
case detail::value_t::array:
{
if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" cannot be used for const access
JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
@@ -1155,7 +1040,7 @@ class json_pointer
case detail::value_t::array:
{
if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" always fails the range check
return nullptr;
@@ -1232,7 +1117,7 @@ class json_pointer
case detail::value_t::array:
{
if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" always fails the range check
return false;
@@ -1391,52 +1276,6 @@ class json_pointer
obj[key] = std::forward<ValueType>(v);
}
/// the number of entries that flattening @a value adds: its primitive
/// values and empty containers
template<typename BasicJsonType>
static std::size_t count_flattened(const BasicJsonType& value)
{
std::size_t count = 0;
std::vector<const BasicJsonType*> pending(1, &value);
while (!pending.empty())
{
const BasicJsonType* const v = pending.back();
pending.pop_back();
if (v->is_array() && !v->m_data.m_value.array->empty())
{
for (const auto& element : *v->m_data.m_value.array)
{
pending.push_back(&element);
}
}
else if (v->is_object() && !v->m_data.m_value.object->empty())
{
for (const auto& member : *v->m_data.m_value.object)
{
pending.push_back(&member.second);
}
}
else
{
++count;
}
}
return count;
}
// Reserve the storage for @a value of objects that have a reserve(), so
// that it does not have to be moved while it grows
template<typename ObjectType, typename BasicJsonType>
static auto reserve_flattened(ObjectType& obj, const BasicJsonType& value, detail::priority_tag<1> /*unused*/)
-> decltype(obj.reserve(std::size_t {}), void())
{
obj.reserve(count_flattened(value));
}
template<typename ObjectType, typename BasicJsonType>
static void reserve_flattened(ObjectType& /*obj*/, const BasicJsonType& /*value*/, detail::priority_tag<0> /*unused*/)
{}
/// append the decimal representation of @a i to @a out
static void append_index(string_t& out, std::size_t i)
{
@@ -1546,7 +1385,6 @@ class json_pointer
}
};
reserve_flattened(object, value, detail::priority_tag<1> {});
enter(value);
while (!stack.empty())
{
@@ -1554,9 +1392,8 @@ class json_pointer
// frame, which would invalidate a reference to it
const BasicJsonType* const container = stack.back().container;
// the path of the previous child is replaced by the path of the
// next one; for an array element in one resize
const std::size_t path_length = stack.back().path_length;
// drop the path of the previous child
path.resize(stack.back().path_length);
if (container->is_array())
{
@@ -1570,20 +1407,8 @@ class json_pointer
// iterate array and use index as a reference string
++stack.back().index;
std::array<char, 24> digits{};
std::size_t pos = digits.size();
std::size_t rest = i;
do
{
digits[--pos] = static_cast<char>('0' + rest % 10);
rest /= 10;
}
while (rest != 0);
const std::size_t count = digits.size() - pos;
path.resize(path_length + 1 + count);
char* const out = &path[path_length];
out[0] = '/';
std::memcpy(out + 1, digits.data() + pos, count);
path.push_back('/');
append_index(path, i);
enter(array[i]);
}
else
@@ -1597,7 +1422,6 @@ class json_pointer
// iterate object and use keys as reference string
++stack.back().member;
path.resize(path_length);
path.push_back('/');
detail::escape_into(path, it->first);
enter(it->second);
@@ -1693,28 +1517,8 @@ class json_pointer
{
end = key.size();
}
const std::size_t token_size = end - pos - 1;
// a token that is a plain index of up to 9 digits, which is
// the case for almost all array elements, is not copied
std::size_t index = 0;
bool plain_index = current->type() == detail::value_t::array && token_size != 0 && token_size <= 9 && (token_size == 1 || data[pos + 1] != '0');
for (std::size_t k = pos + 1; plain_index && k < end; ++k)
{
plain_index = data[k] >= '0' && data[k] <= '9';
index = (index * 10) + static_cast<std::size_t>(data[k] - '0');
}
if (plain_index)
{
current = &current->operator[](static_cast<typename BasicJsonType::size_type>(index));
values.push_back(current);
ends.push_back(end);
pos = end;
continue;
}
token.clear();
token.append(data + pos + 1, token_size);
token.append(data + pos + 1, end - pos - 1);
if (token.find_first_of('~') != string_t::npos)
{
detail::unescape(token);
@@ -1724,13 +1528,10 @@ class json_pointer
{
case detail::value_t::null:
{
// a first child 0 makes the value an array without a look-up
if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, pos))
if (arrays.contains(data, pos))
{
// some reference token below this position is 0, so the
// value is an array
*current = BasicJsonType(detail::value_t::array);
reserve_elements(*current->m_data.m_value.array, 2, detail::priority_tag<1> {});
current = &current->operator[](array_index<BasicJsonType>(token));
}
else
+361 -247
View File
@@ -23,6 +23,7 @@
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/number.hpp>
#include <nlohmann/detail/view/simd.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -80,13 +81,6 @@ class output_buffer
put(s, N - 1);
}
void put_repeated(char c, std::size_t n)
{
reserve(n);
std::memset(m_pos, c, n);
m_pos += n;
}
/// the write position and the end of the writable space, for a writer
/// that keeps the position in a local variable (set_cursor() hands it back)
char* cursor() const noexcept
@@ -143,9 +137,25 @@ class output_buffer
/// image_check::bounds can have others, which are then written unchanged).
inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexcept
{
std::size_t i = 0;
#if NLOHMANN_VIEW_SSE2
// (16 bytes per step; with NEON, the mask of a step costs more than it saves on the
// short runs that are typical here)
for (; i + 16 <= n; i += 16)
{
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(s + i))); // NOLINT(bugprone-casting-through-void)
// (a byte is below 0x20 if it is its own minimum with 0x1F)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmpeq_epi8(_mm_min_epu8(in, _mm_set1_epi8(0x1F)), in));
const auto mask = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
if (mask != 0)
{
return i + static_cast<std::size_t>(count_trailing_zeros(mask));
}
}
#endif
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
const std::uint64_t v = read_eight_bytes(s + i);
@@ -196,6 +206,70 @@ inline bool is_ascii_without_delete(const char* s, std::size_t n) noexcept
return true;
}
#if NLOHMANN_VIEW_VECTOR
/// the bytes of the 16 at p that ensure_ascii cannot copy (a quote, a
/// backslash, a control character, DEL, or not ASCII; a signed compare with
/// 0x20 finds control characters and non-ASCII bytes at once), one bit per
/// byte (x86) or one nibble per byte (NEON)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t ascii_stop_mask(const unsigned char* p) noexcept
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vorrq_u8(vceqq_u8(in, vdupq_n_u8(0x7F)), vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20))));
return vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F)), _mm_cmplt_epi8(in, _mm_set1_epi8(0x20))));
return static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
#endif
}
/// the width in bits of a byte in the mask of ascii_stop_mask()
constexpr unsigned ascii_stop_bits() noexcept
{
return NLOHMANN_VIEW_NEON ? 4u : 1u;
}
#endif
/// As find_ascii_copyable_run(), 16 bytes per step. end is the end of the
/// buffer that holds the n bytes at p (a short run is read as 16 bytes if
/// they are in the buffer).
inline std::size_t ascii_copyable_run(const unsigned char* p, std::size_t n, const unsigned char* end) noexcept
{
#if NLOHMANN_VIEW_VECTOR
if (n >= 16)
{
std::size_t i = 0;
for (; i + 16 <= n; i += 16)
{
const std::uint64_t mask = ascii_stop_mask(p + i);
if (mask != 0)
{
return i + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits());
}
}
if (i == n)
{
return n;
}
// the last 16 bytes: those before i are known to be copyable
const std::uint64_t mask = ascii_stop_mask(p + n - 16);
return mask != 0 ? n - 16 + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits()) : n;
}
if (end - p >= 16)
{
// (the bytes after the run are masked out)
const std::uint64_t mask = ascii_stop_mask(p) & ((std::uint64_t{1} << (n * ascii_stop_bits())) - 1u);
return mask != 0 ? static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits() : n;
}
#else
static_cast<void>(end);
#endif
return find_ascii_copyable_run(p, n);
}
/// A stack that starts in a buffer of the caller (a local array) and moves to
/// the heap (a vector of the caller) only when that is full, so that dumps of
/// shallow documents need no allocation. The top is a pointer, as in
@@ -275,6 +349,39 @@ NLOHMANN_VIEW_ALWAYS_INLINE char* put_literal(char* w, const char (&s)[N]) noexc
return w + N - 1;
}
/// the depth of the innermost open container, counted only for indented output
template<bool Counted>
struct depth_counter
{
std::size_t value = 0;
void enter() noexcept
{
++value;
}
std::size_t leave() noexcept
{
return --value;
}
std::size_t get() const noexcept
{
return value;
}
};
template<>
struct depth_counter<false>
{
void enter() noexcept {}
std::size_t leave() noexcept
{
return 0;
}
std::size_t get() const noexcept
{
return 0;
}
};
/// how the view's dump() writes a value
struct dump_style
{
@@ -311,139 +418,95 @@ class view_serializer
void dump(const node* root)
{
if (!m_style.pretty)
if (m_style.ensure_ascii)
{
if (m_style.ensure_ascii)
{
dump_compact_ascii(root);
}
else if (m_style.source_numbers)
{
dump_compact<true, false>(root);
}
else
{
dump_compact<false, false>(root);
}
return;
dump_ascii(root);
}
struct frame
else if (m_style.pretty)
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
bool first; ///< nothing written yet
};
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
const node* n = root;
for (;;)
dump_pretty(root);
}
else if (m_style.source_numbers)
{
// write the value at n
if (is_container(*n))
{
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
if (n->len == 0)
{
m_out.put(object ? "{}" : "[]", 2);
}
else
{
m_out.put(object ? '{' : '[');
stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
}
}
else
{
write_scalar(*n);
}
// go to the next value: close finished containers, then separate
for (;;)
{
if (stack.empty())
{
m_out.finish();
return;
}
if (stack.back().pos == stack.back().end)
{
const bool object = stack.back().object;
stack.pop_back();
newline(stack.size());
m_out.put(object ? '}' : ']');
continue;
}
frame& f = stack.back();
if (!f.first)
{
m_out.put(',');
}
f.first = false;
newline(stack.size());
if (f.object)
{
write_string(*f.pos);
if (m_style.pretty)
{
m_out.put(": ");
}
else
{
m_out.put(':');
}
n = nav::value(f.pos + 1);
f.pos = document_data::after(f.pos + 1);
}
else
{
n = nav::value(f.pos);
f.pos = document_data::after(f.pos);
}
break;
}
dump_text<true, false, false>(root);
}
else
{
dump_text<false, false, false>(root);
}
}
private:
/// a string that is not a plain string of the source (decoded, or written
/// by an edit), without ensure_ascii: runs without characters to escape
/// are copied
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n)
/// are copied. If validate is set, the string must be valid UTF-8: the
/// runs are found by the lexer's scan, which validates the sequences, and
/// invalid UTF-8 throws what basic_json::dump() throws. (The compact output
/// writes the bytes of a damaged image unchanged, and scans without
/// validating them.)
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n, bool validate)
{
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
m_out.put('"');
for (std::size_t i = 0; i < n.len;)
{
const std::size_t run = plain_output_run(s + i, n.len - i);
const std::size_t run = validate ? string_bulk_run(s + i, n.len - i) : plain_output_run(s + i, n.len - i);
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
if (validate && s[i] >= 0x80)
{
invalid_utf8(s, n.len); // (the scan stopped at ill-formed UTF-8)
}
write_codepoint<false>(s[i], s + i, 1); // a quote, a backslash, or a control character
++i;
}
m_out.put('"');
}
/// dump_compact() with ensure_ascii, out of line: the walk without it is
/// faster when it is not merged with this one into dump()
NLOHMANN_VIEW_NOINLINE void dump_compact_ascii(const node* root)
/// dump_text() with indentation, out of line: the compact walk is faster
/// when it is not merged with the indented ones into dump()
NLOHMANN_VIEW_NOINLINE void dump_pretty(const node* root)
{
if (m_style.source_numbers)
{
dump_compact<true, true>(root);
dump_text<true, true, false>(root);
}
else
{
dump_compact<false, true>(root);
dump_text<false, true, false>(root);
}
}
/// whether dump_compact() copies a string or key from the source: one of
/// dump_text() with ensure_ascii, out of line: the walk without it is
/// faster when it is not merged with this one into dump()
NLOHMANN_VIEW_NOINLINE void dump_ascii(const node* root)
{
if (m_style.pretty)
{
if (m_style.source_numbers)
{
dump_text<true, true, true>(root);
}
else
{
dump_text<false, true, true>(root);
}
}
else if (m_style.source_numbers)
{
dump_text<true, false, true>(root);
}
else
{
dump_text<false, false, true>(root);
}
}
/// whether dump_text() copies a string or key from the source: one of
/// the source, and with ensure_ascii one without bytes >= 0x7F (an overload
/// each, so that the walk without ensure_ascii is compiled exactly as
/// before: with GCC, a condition that only folds away later changed its
@@ -458,13 +521,154 @@ class view_serializer
return (n.flags & node_flags::storage) == 0 && is_ascii_without_delete(src + n.off, n.len);
}
/// a string with ensure_ascii (dump_compact()), out of line
NLOHMANN_VIEW_NOINLINE void write_ascii_string(const node& n)
/*!
@brief a string with ensure_ascii: the characters that are not printable ASCII as \\uXXXX
(a character beyond U+FFFF as a surrogate pair); strings of a document
are valid UTF-8, except in a damaged image loaded with image_check::bounds,
for which this throws what basic_json::dump() throws for the string
*/
NLOHMANN_VIEW_NOINLINE void write_ascii(const node& n)
{
write_string(n);
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const std::size_t size = n.len;
char* w = m_out.cursor();
char* lim = m_out.limit();
// room for k bytes
const auto room = [&](std::size_t k)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(lim - w) < k))
{
m_out.set_cursor(w);
m_out.reserve(k);
w = m_out.cursor();
lim = m_out.limit();
}
};
static constexpr const char* hex = "0123456789abcdef";
// \\uXXXX
const auto u_escape = [&](std::uint32_t u)
{
w[0] = '\\';
w[1] = 'u';
w[2] = hex[(u >> 12u) & 0xFu];
w[3] = hex[(u >> 8u) & 0xFu];
w[4] = hex[(u >> 4u) & 0xFu];
w[5] = hex[u & 0xFu];
w += 6;
};
room(1);
*w++ = '"';
std::size_t i = 0;
// (a string of the source starts with a run that has nothing to escape, 16 bytes per step)
const std::size_t prefix = (n.flags & node_flags::storage) == 0 ? ascii_copyable_run(s, size, reinterpret_cast<const unsigned char*>(m_doc.src + m_doc.size)) : 0; // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
if (prefix != 0)
{
room(prefix);
std::memcpy(w, s, prefix);
w += prefix;
i = prefix;
}
while (i < size)
{
// (a character that needs an escape sequence is not worth a call)
const std::size_t run = is_ascii_copyable(s[i]) ? find_ascii_copyable_run(s + i, size - i) : 0;
if (run != 0)
{
room(run);
std::memcpy(w, s + i, run);
w += run;
i += run;
continue;
}
room(12);
if (s[i] >= 0x80)
{
const unsigned char lead = s[i];
// (the two-byte sequences of Latin text and the three-byte sequences of most other
// text, with continuation bytes of any value: the ranges that are restricted (E0,
// ED, and the sequences of four bytes) go through validate_one_utf8())
if (lead >= 0xE1 && lead <= 0xEF && lead != 0xED && size - i >= 3 && (s[i + 1] & 0xC0u) == 0x80 && (s[i + 2] & 0xC0u) == 0x80)
{
u_escape(((lead & 0x0Fu) << 12u) | ((s[i + 1] & 0x3Fu) << 6u) | (s[i + 2] & 0x3Fu));
i += 3;
continue;
}
if (lead >= 0xC2 && lead <= 0xDF && size - i >= 2 && (s[i + 1] & 0xC0u) == 0x80)
{
u_escape(((lead & 0x1Fu) << 6u) | (s[i + 1] & 0x3Fu));
i += 2;
continue;
}
const std::size_t len = validate_one_utf8(s + i, size - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
m_out.set_cursor(w);
invalid_utf8(s, size);
return; // LCOV_EXCL_LINE (invalid_utf8 throws)
}
std::uint32_t codepoint = s[i] & (0xFFu >> (len + 1));
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
i += len;
if (codepoint <= 0xFFFF)
{
u_escape(codepoint);
}
else
{
u_escape(0xD7C0u + (codepoint >> 10u));
u_escape(0xDC00u + (codepoint & 0x3FFu));
}
continue;
}
// a quote, a backslash, or a control character (or DEL)
const unsigned char c = s[i++];
char escape = 0;
switch (c)
{
case 0x08:
escape = 'b';
break;
case 0x09:
escape = 't';
break;
case 0x0A:
escape = 'n';
break;
case 0x0C:
escape = 'f';
break;
case 0x0D:
escape = 'r';
break;
case 0x22:
escape = '"';
break;
case 0x5C:
escape = '\\';
break;
default:
break;
}
if (escape != 0)
{
w[0] = '\\';
w[1] = escape;
w += 2;
}
else
{
u_escape(c);
}
}
room(1);
*w++ = '"';
m_out.set_cursor(w);
}
/// the copies of dump_compact() that are not fixed-size moves (long
/// the copies of dump_text() that are not fixed-size moves (long
/// strings, or near the end of the source); out of line, so that the
/// compiler does not merge the fixed-size moves into this call
NLOHMANN_VIEW_NOINLINE static void copy_long(char* to, const char* from, std::size_t n) noexcept
@@ -481,7 +685,7 @@ class view_serializer
}
/*!
@brief the compact output (the default dump(), or with ensure_ascii)
@brief the output, compact (the default dump()) or indented, with or without ensure_ascii
The same walk as dump(), with the write position in a local variable
(stores through char pointers would otherwise force a reload of the
@@ -489,11 +693,11 @@ class view_serializer
the source copied by fixed-size moves of 32 bytes where the source has
that many bytes left, instead of a library call per token. The buffer
keeps 64 bytes of slack for the overshoot. With ensure_ascii (Ascii), a
string of the source is copied only if it has no byte >= 0x7F (see
is_ascii_without_delete()); any other string is written by write_string().
string of the source is copied only if it has nothing to escape (see
copied()); any other string is written by write_ascii().
*/
template<bool SourceNumbers, bool Ascii>
void dump_compact(const node* root)
template<bool SourceNumbers, bool Pretty, bool Ascii>
void dump_text(const node* root)
{
struct frame
{
@@ -508,6 +712,16 @@ class view_serializer
const char* const src_end = src + m_doc.size;
char* w = m_out.cursor();
char* lim = m_out.limit();
// (indented output) a line break and 63 indentation characters, from
// which the line break of a level of up to 63 characters is a copy
std::array<char, 64> ruler; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled below if used
if (Pretty)
{
ruler.fill(m_style.indent_char);
ruler[0] = '\n';
}
const std::size_t indent = m_style.indent;
depth_counter<Pretty> level; // depth of the innermost open container (indented output)
// room for n bytes and the slack
const auto room = [&](std::size_t n)
{
@@ -539,6 +753,22 @@ class view_serializer
}
w += n;
};
// (indented output) the line break and the indentation of a level
const auto newline = [&](std::size_t depth)
{
const std::size_t count = 1 + (depth * indent);
room(count);
if (count <= ruler.size())
{
std::memcpy(w, ruler.data(), ruler.size());
}
else
{
*w = '\n';
std::memset(w + 1, m_style.indent_char, count - 1);
}
w += count;
};
// a string that is not a plain string of the source (out of line, so
// that the cursor stays in a register here)
const auto escaped = [&](const node & n)
@@ -546,11 +776,11 @@ class view_serializer
m_out.set_cursor(w);
if (enabled(Ascii))
{
write_ascii_string(n);
write_ascii(n);
}
else
{
write_decoded(n);
write_decoded(n, Pretty);
}
w = m_out.cursor();
lim = m_out.limit();
@@ -651,6 +881,7 @@ class view_serializer
{
*w++ = object ? '{' : '[';
stack.push_back(cur);
level.enter();
if (Editable)
{
cur = frame{nav::first(m_doc, n), nav::end(m_doc, n), object};
@@ -693,6 +924,10 @@ class view_serializer
{
break;
}
if (Pretty)
{
newline(level.leave());
}
room(1);
*w++ = cur.object ? '}' : ']';
cur = stack.back();
@@ -701,13 +936,17 @@ class view_serializer
room(1);
*w++ = ',';
}
if (Pretty)
{
newline(level.get());
}
const node* const at = Editable ? cur.pos : n;
if (cur.object)
{
const node& key = *at;
if (copied(key, src, std::integral_constant<bool, Ascii>()))
{
room(key.len + 3);
room(key.len + (Pretty ? 4 : 3));
*w++ = '"';
copy(src + key.off, key.len);
w[0] = '"';
@@ -717,9 +956,13 @@ class view_serializer
else
{
escaped(key);
room(1);
room(Pretty ? 2 : 1);
*w++ = ':';
}
if (Pretty)
{
*w++ = ' ';
}
if (Editable)
{
n = nav::value(at + 1);
@@ -738,69 +981,6 @@ class view_serializer
}
}
void newline(std::size_t level)
{
if (m_style.pretty)
{
m_out.put('\n');
m_out.put_repeated(m_style.indent_char, level * m_style.indent);
}
}
void write_scalar(const node& n)
{
switch (static_cast<value_t>(n.kind))
{
case value_t::null:
m_out.put("null");
break;
case value_t::boolean:
if ((n.flags & node_flags::is_true) != 0)
{
m_out.put("true");
}
else
{
m_out.put("false");
}
break;
case value_t::string:
write_string(n);
break;
case value_t::number_integer:
case value_t::number_unsigned:
{
const char* const token = m_doc.str(n);
const std::uint32_t len = number_length(n);
if (!m_style.source_numbers && len == 2 && token[0] == '-' && token[1] == '0')
{
m_out.put('0'); // parse() reads -0 as the integer 0
}
else
{
m_out.put(token, len);
}
break;
}
case value_t::number_float:
if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
{
m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
}
else
{
write_float_node(n);
}
break;
case value_t::object: // LCOV_EXCL_LINE (containers are written by dump())
case value_t::array: // LCOV_EXCL_LINE
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
}
/// a float node as dump() writes it
void write_float_node(const node& n)
{
@@ -911,72 +1091,6 @@ class view_serializer
m_out.put(s.data(), s.size());
}
void write_string(const node& n)
{
const char* const s = m_doc.str(n);
m_out.put('"');
if ((n.flags & node_flags::storage) == 0 && (!m_style.ensure_ascii || is_ascii_without_delete(s, n.len)))
{
// a string of the source without escape sequences has nothing to
// escape (with ensure_ascii, if it is printable ASCII)
m_out.put(s, n.len);
}
else if (m_style.ensure_ascii)
{
write_escaped<true>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
else
{
write_escaped<false>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
m_out.put('"');
}
/// as serializer::dump_escaped(); strings of a document are valid UTF-8,
/// except in a damaged image loaded with image_check::bounds, for which
/// this throws what basic_json::dump() throws for the string
template<bool EnsureAscii>
void write_escaped(const unsigned char* s, std::size_t n)
{
std::size_t i = 0;
while (i < n)
{
std::size_t run = 0;
if (!EnsureAscii)
{
run = string_bulk_run(s + i, n - i);
}
else if (is_ascii_copyable(s[i]))
{
run = find_ascii_copyable_run(s + i, n - i);
}
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
std::uint32_t codepoint = s[i];
std::size_t len = 1;
if (codepoint >= 0x80)
{
len = validate_one_utf8(s + i, n - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
invalid_utf8(s, n);
return;
}
codepoint &= 0xFFu >> (len + 1);
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
}
write_codepoint<EnsureAscii>(codepoint, s + i, len);
i += len;
}
}
/// throw what basic_json::dump() throws for a string that is not valid UTF-8
NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
{
+32 -157
View File
@@ -6878,72 +6878,6 @@ public:
/// @name JSON Patch functions
/// @{
private:
/// the members of an operation of a JSON patch that are of interest
struct patch_members
{
basic_json* op = nullptr;
basic_json* path = nullptr;
basic_json* value = nullptr;
basic_json* from = nullptr;
};
/// @brief find the members "op", "path", "value", and "from" of the object @a val in one pass
static patch_members find_patch_members(const basic_json& val, std::true_type /*key_type is string_t*/)
{
patch_members members;
for (auto& member : *val.m_data.m_value.object)
{
const string_t& key = member.first;
switch (key.size())
{
case 2:
if (key == "op")
{
members.op = &member.second;
}
break;
case 4:
if (key == "path")
{
members.path = &member.second;
}
else if (key == "from")
{
members.from = &member.second;
}
break;
case 5:
if (key == "value")
{
members.value = &member.second;
}
break;
default:
break;
}
}
return members;
}
/// @brief find the members by lookup for key types other than string_t
static patch_members find_patch_members(const basic_json& val, std::false_type /*key_type is not string_t*/)
{
const auto find_member = [&val](const char* name) -> basic_json *
{
const auto it = val.m_data.m_value.object->find(string_t(name));
return it != val.m_data.m_value.object->end() ? &it->second : nullptr;
};
patch_members members;
members.op = find_member("op");
members.path = find_member("path");
members.value = find_member("value");
members.from = find_member("from");
return members;
}
public:
/// @brief applies a JSON patch in-place without copying the object
/// @sa https://json.nlohmann.me/api/basic_json/patch/
void patch_inplace(const basic_json& json_patch)
@@ -7117,18 +7051,18 @@ public:
// a diff creates them: apply the operation and return true; return
// false, with nothing changed, for all others
string_t token;
typename json_pointer::template plain_steps<basic_json> steps;
const auto try_plain = [&result, &token, &steps](patch_operations operation, const string_t& path, const patch_members & members, const basic_json & val)
const string_t value_key = "value";
const auto try_plain = [&result, &token, &value_key](patch_operations operation, const string_t& path, const basic_json & val)
{
if (path.empty())
if (path.empty() || path.find_first_of('~') != string_t::npos)
{
return false;
}
// a missing "value" is reported below
const basic_json* const value_ptr = members.value;
const basic_json& value = value_ptr != nullptr ? *value_ptr : val;
if (operation != patch_operations::remove && value_ptr == nullptr)
const auto value_it = val.m_data.m_value.object->find(value_key);
const basic_json& value = value_it != val.m_data.m_value.object->end() ? value_it->second : val;
if (operation != patch_operations::remove && value_it == val.m_data.m_value.object->end())
{
return false;
}
@@ -7137,7 +7071,7 @@ public:
{
case patch_operations::replace:
{
basic_json* const target = json_pointer::resolve_plain(&result, path, path.size(), token, steps);
basic_json* const target = json_pointer::resolve_plain(&result, path, path.size(), token);
if (target == nullptr)
{
return false;
@@ -7148,7 +7082,7 @@ public:
case patch_operations::test:
{
const basic_json* const target = json_pointer::resolve_plain(&result, path, path.size(), token, steps);
const basic_json* const target = json_pointer::resolve_plain(&result, path, path.size(), token);
// a failing test is reported below
return target != nullptr && *target == value;
}
@@ -7168,17 +7102,13 @@ public:
}
--slash;
basic_json* const parent = json_pointer::resolve_plain(&result, path, slash, token, steps);
basic_json* const parent = json_pointer::resolve_plain(&result, path, slash, token);
if (parent == nullptr)
{
return false;
}
token.clear();
token.append(path.data() + slash + 1, path.size() - slash - 1);
if (token.find_first_of('~') != string_t::npos)
{
return false;
}
if (operation == patch_operations::add)
{
@@ -7234,32 +7164,31 @@ public:
}
};
// the name of the first member to check
const string_t op_member = "op";
// iterate and apply the operations
for (const auto& val : json_patch)
{
// wrapper to get a value for an operation
const auto get_value = [&val](basic_json * found,
const string_t& op,
const char* member,
const auto get_value = [&val](const string_t& op,
const string_t& member,
bool string_type) -> basic_json &
{
// find value
auto it = val.m_data.m_value.object->find(member);
// check if the desired value is present
if (JSON_HEDLEY_UNLIKELY(found == nullptr))
if (JSON_HEDLEY_UNLIKELY(it == val.m_data.m_value.object->end()))
{
detail::throw_patch_missing_member(op, string_t(member), false, val);
detail::throw_patch_missing_member(op, member, false, val);
}
// check if the result is of type string
if (JSON_HEDLEY_UNLIKELY(string_type && !found->is_string()))
if (JSON_HEDLEY_UNLIKELY(string_type && !it->second.is_string()))
{
detail::throw_patch_missing_member(op, string_t(member), true, val);
detail::throw_patch_missing_member(op, member, true, val);
}
// no error: return value
return *found;
return it->second;
};
// type check: every element of the array must be an object
@@ -7268,30 +7197,26 @@ public:
detail::throw_patch_not_an_array(val);
}
const patch_members members = find_patch_members(val, std::is_same<typename object_t::key_type, string_t> {});
// collect mandatory members
const string_t& op = *get_value(members.op, op_member, "op", true).m_data.m_value.string;
const string_t& path = *get_value(members.path, op, "path", true).m_data.m_value.string;
const string_t& op = *get_value("op", "op", true).m_data.m_value.string;
const string_t& path = *get_value(op, "path", true).m_data.m_value.string;
const patch_operations operation = get_op(op);
// the usual operations on a plain pointer are applied without
// building the pointer's reference tokens; anything else (and
// every error) is handled below
if (try_plain(operation, path, members, val))
if (try_plain(operation, path, val))
{
continue;
}
// the general operations may change any container
steps.reset();
json_pointer ptr(path);
switch (operation)
{
case patch_operations::add:
{
operation_add(ptr, get_value(members.value, "add", "value", false));
operation_add(ptr, get_value("add", "value", false));
break;
}
@@ -7304,13 +7229,13 @@ public:
case patch_operations::replace:
{
// the "path" location must exist - use at()
result.at(ptr) = get_value(members.value, "replace", "value", false);
result.at(ptr) = get_value("replace", "value", false);
break;
}
case patch_operations::move:
{
const string_t& from_path = *get_value(members.from, "move", "from", true).m_data.m_value.string;
const string_t& from_path = *get_value("move", "from", true).m_data.m_value.string;
json_pointer from_ptr(from_path);
if (JSON_HEDLEY_UNLIKELY(is_proper_prefix(from_ptr, ptr)))
@@ -7333,7 +7258,7 @@ public:
case patch_operations::copy:
{
const string_t& from_path = *get_value(members.from, "copy", "from", true).m_data.m_value.string;
const string_t& from_path = *get_value("copy", "from", true).m_data.m_value.string;
const json_pointer from_ptr(from_path);
// The copy is functionally identical to an "add"
@@ -7351,7 +7276,7 @@ public:
{
// check if "value" matches the one at "path"
// the "path" location must exist - use at()
success = (result.at(ptr) == get_value(members.value, "test", "value", false));
success = (result.at(ptr) == get_value("test", "value", false));
}
JSON_INTERNAL_CATCH (out_of_range&)
{
@@ -7443,18 +7368,13 @@ public:
/// @brief append an operation @a op for @a path to @a result, with @a value if it is not null
static void diff_operation(basic_json& result, const char* op, const string_t& path, const basic_json* value)
{
// the keys are distinct and in the order of std::map; the members are
// added without looking them up
basic_json entry(value_t::object);
auto& members = *entry.m_data.m_value.object;
typename object_t::iterator hint = members.end();
json_pointer::add_flattened(members, hint, string_t("op"), string_t(op), detail::priority_tag<2> {});
json_pointer::add_flattened(members, hint, string_t("path"), path, detail::priority_tag<2> {});
entry["op"] = op;
entry["path"] = path;
if (value != nullptr)
{
json_pointer::add_flattened(members, hint, string_t("value"), *value, detail::priority_tag<2> {});
entry["value"] = *value;
}
entry.set_parents();
result.push_back(std::move(entry));
}
@@ -7476,50 +7396,6 @@ public:
diff_operation(result, "add", path, &value);
}
/// @brief the values in the target of the members of a source object, for the
/// objects of up to 16 members without allocating
///
/// reserve() must be called with the number of values that are added.
class diff_target_values
{
public:
diff_target_values() = default;
diff_target_values(const diff_target_values&) = delete;
diff_target_values& operator=(const diff_target_values&) = delete;
~diff_target_values() = default;
void reserve(const std::size_t n)
{
if (n > inline_size)
{
large.resize(n);
items = large.data();
}
}
void push_back(const basic_json* value)
{
items[count++] = value;
}
std::size_t size() const noexcept
{
return count;
}
const basic_json* operator[](const std::size_t i) const noexcept
{
return items[i];
}
private:
static constexpr std::size_t inline_size = 16;
std::size_t count = 0;
const basic_json** items = inline_items;
std::vector<const basic_json*> large{}; // NOLINT(readability-redundant-member-init)
const basic_json* inline_items[inline_size]; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init)
};
/// @brief append "/" and the decimal representation of @a index to @a path
static void diff_append_index(string_t& path, std::size_t index)
{
@@ -7600,9 +7476,8 @@ public:
operations that remove every member of @a source and add every member of
@a target to @a result, and returns false.
*/
template<typename TargetValues>
static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, TargetValues& target_values,
const string_t& path, std::vector<const basic_json*>& target_values,
std::vector<const_iterator>& added)
{
// first pass: one lookup per source key
@@ -7816,7 +7691,7 @@ public:
case value_t::object:
{
diff_target_values target_values;
std::vector<const basic_json*> target_values;
std::vector<const_iterator> added;
if (diff_object_keys(result, source, target, path, target_values, added))
{
+65 -389
View File
@@ -22357,7 +22357,7 @@ NLOHMANN_JSON_NAMESPACE_END
#include <cstddef> // size_t
#include <cstdint> // uint64_t
#include <cstdlib> // strtoull
#include <cstring> // memchr, memcmp, memcpy
#include <cstring> // memcmp, memcpy
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
#endif // JSON_NO_IO
@@ -22653,12 +22653,6 @@ class json_pointer
exceeds_size_type ///< @a s converts to an integer that exceeds size_type
};
/// whether a reference token is "-" (a plain comparison with a string literal is not always inlined)
static bool is_dash(const string_t& s) noexcept
{
return s.size() == 1 && s[0] == '-';
}
/*!
@param[in] s reference token to be converted into an array index
@param[out] idx the integer representation of @a s if @ref array_index_status::ok
@@ -22684,7 +22678,7 @@ class json_pointer
// error condition (cf. RFC 6901, Sect. 4); this also covers single-
// character tokens, so "/x" and "/xy" fail alike; "-" and the empty
// token are left to the conversion below and are reported as unresolved
if (JSON_HEDLEY_UNLIKELY(!s.empty() && !is_dash(s) && !(s[0] >= '0' && s[0] <= '9')))
if (JSON_HEDLEY_UNLIKELY(!s.empty() && s != "-" && !(s[0] >= '0' && s[0] <= '9')))
{
return array_index_status::not_a_number;
}
@@ -22831,40 +22825,9 @@ class json_pointer
return pos == size;
}
/// reserve room for @a n elements of an array type that has reserve()
template<typename ArrayType>
static auto reserve_elements(ArrayType& array, const std::size_t n, detail::priority_tag<1> /*unused*/)
-> decltype(array.reserve(n), void())
{
array.reserve(n);
}
template<typename ArrayType>
static void reserve_elements(ArrayType& /*array*/, const std::size_t /*n*/, detail::priority_tag<0> /*unused*/)
{}
/// @brief object member access with a key that is only copied if it is inserted
///
/// Objects that can find the position of a key and insert there
/// (std::map) search once, others look the key up and add it if it is missing.
template<typename BasicJsonType>
static auto object_member_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace_hint(object.m_data.m_value.object->lower_bound(key), key, nullptr), std::declval<BasicJsonType&>())
{
auto& members = *object.m_data.m_value.object;
auto it = members.lower_bound(key);
if (it != members.end() && !members.key_comp()(key, it->first))
{
return it->second;
}
const auto old_capacity = object.container_capacity();
it = members.emplace_hint(it, key, nullptr);
return object.set_parent_after_insert(it->second, old_capacity);
}
template<typename BasicJsonType>
static BasicJsonType& object_member_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<0> /*unused*/)
static BasicJsonType& object_member(BasicJsonType& object, const string_t& key)
{
auto& members = *object.m_data.m_value.object;
const auto it = members.find(key);
@@ -22875,12 +22838,6 @@ class json_pointer
return object.operator[](key);
}
template<typename BasicJsonType>
static BasicJsonType& object_member(BasicJsonType& object, const string_t& key)
{
return object_member_impl(object, key, detail::priority_tag<1> {});
}
JSON_PRIVATE_UNLESS_TESTED:
json_pointer top() const
{
@@ -23113,103 +23070,48 @@ class json_pointer
}
}
/*!
@brief the values along the pointer last resolved by @ref resolve_plain
A patch from @ref basic_json::diff lists its operations in document order,
so consecutive pointers share most of their reference tokens; the values
along the previous pointer are kept to resume from the common prefix.
`steps[k]` is the value after the first `k` reference tokens together with
the length of the prefix of `*path` that holds them. A value stays valid as
long as no operation changes its container: callers keep only values up to
the parent of the pointer they modify, or call `reset()`.
*/
template<typename BasicJsonType>
struct plain_steps
{
const string_t* path = nullptr;
std::vector<std::pair<std::size_t, BasicJsonType*>> steps{};
void reset() noexcept
{
path = nullptr;
steps.clear();
}
};
/*!
@brief resolve a plain JSON pointer without building its reference tokens
@param[in] root the value the pointer starts at
@param[in] ptr the value the pointer starts at
@param[in] path a JSON pointer string
@param[in] length the number of leading bytes of @a path to resolve; 0 or
the position of a slash
@param[in,out] token a string to hold the reference tokens, to reuse its storage
@param[in,out] cache the values along the previously resolved pointer;
@a path must outlive the next call
@return the pointed to value, or `nullptr` if the pointer does not denote
an existing value, or does so in a way that is not plain: an array
index that is malformed or out of range, or a "-", or a reference
token with an escape. No exception is
thrown for these; callers use @ref json_pointer instead, which
handles them.
an existing value, or does so in a way that is not plain: a
reference token with an escape, an array index that is malformed or
out of range, or a "-". No exception is thrown for these; callers
use @ref json_pointer instead, which handles them.
*/
template<typename BasicJsonType>
static BasicJsonType* resolve_plain(BasicJsonType* root, const string_t& path, const std::size_t length, string_t& token, plain_steps<BasicJsonType>& cache)
static BasicJsonType* resolve_plain(BasicJsonType* ptr, const string_t& path, const std::size_t length, string_t& token)
{
auto& steps = cache.steps;
// resume from the deepest step that is also a prefix of this path
std::size_t depth = 1;
if (cache.path != nullptr && !steps.empty() && steps[0].second == root)
{
const string_t& previous = *cache.path;
for (depth = steps.size(); depth > 1; --depth)
{
// a step ends at a token boundary of the previous path
const std::size_t prefix = steps[depth - 1].first;
if (prefix <= length && (prefix == length || path[prefix] == '/') && std::memcmp(path.data(), previous.data(), prefix) == 0)
{
break;
}
}
steps.resize(depth);
}
else
{
steps.assign(1, std::make_pair(std::size_t{0}, root));
}
cache.path = &path;
BasicJsonType* ptr = steps[depth - 1].second;
for (std::size_t pos = steps[depth - 1].first; pos < length;)
for (std::size_t pos = 0; pos < length;)
{
if (JSON_HEDLEY_UNLIKELY(path[pos] != '/'))
{
return nullptr;
}
// the reference token between the slash at pos and the next one,
// which must not have an escape
const char* const begin = path.data() + pos + 1;
const char* const limit = path.data() + length;
const char* end = begin;
while (end < limit && *end != '/')
// the reference token between the slash at pos and the next one
std::size_t end = path.find_first_of('/', pos + 1);
if (end == string_t::npos || end > length)
{
if (JSON_HEDLEY_UNLIKELY(*end == '~'))
{
return nullptr;
}
++end;
end = length;
}
token.clear();
token.append(path.data() + pos + 1, end - pos - 1);
if (JSON_HEDLEY_UNLIKELY(token.find_first_of('~') != string_t::npos))
{
return nullptr;
}
switch (ptr->type())
{
case detail::value_t::object:
{
token.clear();
token.append(begin, static_cast<std::size_t>(end - begin));
const auto it = ptr->m_data.m_value.object->find(token);
if (JSON_HEDLEY_UNLIKELY(it == ptr->m_data.m_value.object->end()))
{
@@ -23221,24 +23123,8 @@ class json_pointer
case detail::value_t::array:
{
// an index of up to 18 digits without a leading zero fits into size_type
// (9 digits for a 32-bit size_type); everything else is left to json_pointer
constexpr std::size_t max_digits = sizeof(typename BasicJsonType::size_type) >= 8 ? 18 : 9;
const std::size_t digits = static_cast<std::size_t>(end - begin);
if (JSON_HEDLEY_UNLIKELY(digits == 0 || digits > max_digits || (digits > 1 && *begin == '0')))
{
return nullptr;
}
typename BasicJsonType::size_type idx = 0;
for (const char* c = begin; c != end; ++c)
{
if (JSON_HEDLEY_UNLIKELY(*c < '0' || *c > '9'))
{
return nullptr;
}
idx = static_cast<typename BasicJsonType::size_type>((idx * 10) + static_cast<typename BasicJsonType::size_type>(*c - '0'));
}
if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
typename BasicJsonType::size_type idx{};
if (JSON_HEDLEY_UNLIKELY(parse_array_index<BasicJsonType>(token, idx) != array_index_status::ok || idx >= ptr->m_data.m_value.array->size()))
{
return nullptr;
}
@@ -23257,8 +23143,7 @@ class json_pointer
default:
return nullptr;
}
pos = static_cast<std::size_t>(end - path.data());
steps.emplace_back(pos, ptr);
pos = end;
}
return ptr;
@@ -23372,7 +23257,7 @@ class json_pointer
case detail::value_t::array:
{
if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" always fails the range check
JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat(
@@ -23443,7 +23328,7 @@ class json_pointer
case detail::value_t::array:
{
if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" cannot be used for const access
JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
@@ -23505,7 +23390,7 @@ class json_pointer
case detail::value_t::array:
{
if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" always fails the range check
return nullptr;
@@ -23582,7 +23467,7 @@ class json_pointer
case detail::value_t::array:
{
if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" always fails the range check
return false;
@@ -23741,52 +23626,6 @@ class json_pointer
obj[key] = std::forward<ValueType>(v);
}
/// the number of entries that flattening @a value adds: its primitive
/// values and empty containers
template<typename BasicJsonType>
static std::size_t count_flattened(const BasicJsonType& value)
{
std::size_t count = 0;
std::vector<const BasicJsonType*> pending(1, &value);
while (!pending.empty())
{
const BasicJsonType* const v = pending.back();
pending.pop_back();
if (v->is_array() && !v->m_data.m_value.array->empty())
{
for (const auto& element : *v->m_data.m_value.array)
{
pending.push_back(&element);
}
}
else if (v->is_object() && !v->m_data.m_value.object->empty())
{
for (const auto& member : *v->m_data.m_value.object)
{
pending.push_back(&member.second);
}
}
else
{
++count;
}
}
return count;
}
// Reserve the storage for @a value of objects that have a reserve(), so
// that it does not have to be moved while it grows
template<typename ObjectType, typename BasicJsonType>
static auto reserve_flattened(ObjectType& obj, const BasicJsonType& value, detail::priority_tag<1> /*unused*/)
-> decltype(obj.reserve(std::size_t {}), void())
{
obj.reserve(count_flattened(value));
}
template<typename ObjectType, typename BasicJsonType>
static void reserve_flattened(ObjectType& /*obj*/, const BasicJsonType& /*value*/, detail::priority_tag<0> /*unused*/)
{}
/// append the decimal representation of @a i to @a out
static void append_index(string_t& out, std::size_t i)
{
@@ -23896,7 +23735,6 @@ class json_pointer
}
};
reserve_flattened(object, value, detail::priority_tag<1> {});
enter(value);
while (!stack.empty())
{
@@ -23904,9 +23742,8 @@ class json_pointer
// frame, which would invalidate a reference to it
const BasicJsonType* const container = stack.back().container;
// the path of the previous child is replaced by the path of the
// next one; for an array element in one resize
const std::size_t path_length = stack.back().path_length;
// drop the path of the previous child
path.resize(stack.back().path_length);
if (container->is_array())
{
@@ -23920,20 +23757,8 @@ class json_pointer
// iterate array and use index as a reference string
++stack.back().index;
std::array<char, 24> digits{};
std::size_t pos = digits.size();
std::size_t rest = i;
do
{
digits[--pos] = static_cast<char>('0' + rest % 10);
rest /= 10;
}
while (rest != 0);
const std::size_t count = digits.size() - pos;
path.resize(path_length + 1 + count);
char* const out = &path[path_length];
out[0] = '/';
std::memcpy(out + 1, digits.data() + pos, count);
path.push_back('/');
append_index(path, i);
enter(array[i]);
}
else
@@ -23947,7 +23772,6 @@ class json_pointer
// iterate object and use keys as reference string
++stack.back().member;
path.resize(path_length);
path.push_back('/');
detail::escape_into(path, it->first);
enter(it->second);
@@ -24043,28 +23867,8 @@ class json_pointer
{
end = key.size();
}
const std::size_t token_size = end - pos - 1;
// a token that is a plain index of up to 9 digits, which is
// the case for almost all array elements, is not copied
std::size_t index = 0;
bool plain_index = current->type() == detail::value_t::array && token_size != 0 && token_size <= 9 && (token_size == 1 || data[pos + 1] != '0');
for (std::size_t k = pos + 1; plain_index && k < end; ++k)
{
plain_index = data[k] >= '0' && data[k] <= '9';
index = (index * 10) + static_cast<std::size_t>(data[k] - '0');
}
if (plain_index)
{
current = &current->operator[](static_cast<typename BasicJsonType::size_type>(index));
values.push_back(current);
ends.push_back(end);
pos = end;
continue;
}
token.clear();
token.append(data + pos + 1, token_size);
token.append(data + pos + 1, end - pos - 1);
if (token.find_first_of('~') != string_t::npos)
{
detail::unescape(token);
@@ -24074,13 +23878,10 @@ class json_pointer
{
case detail::value_t::null:
{
// a first child 0 makes the value an array without a look-up
if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, pos))
if (arrays.contains(data, pos))
{
// some reference token below this position is 0, so the
// value is an array
*current = BasicJsonType(detail::value_t::array);
reserve_elements(*current->m_data.m_value.array, 2, detail::priority_tag<1> {});
current = &current->operator[](array_index<BasicJsonType>(token));
}
else
@@ -39387,72 +39188,6 @@ public:
/// @name JSON Patch functions
/// @{
private:
/// the members of an operation of a JSON patch that are of interest
struct patch_members
{
basic_json* op = nullptr;
basic_json* path = nullptr;
basic_json* value = nullptr;
basic_json* from = nullptr;
};
/// @brief find the members "op", "path", "value", and "from" of the object @a val in one pass
static patch_members find_patch_members(const basic_json& val, std::true_type /*key_type is string_t*/)
{
patch_members members;
for (auto& member : *val.m_data.m_value.object)
{
const string_t& key = member.first;
switch (key.size())
{
case 2:
if (key == "op")
{
members.op = &member.second;
}
break;
case 4:
if (key == "path")
{
members.path = &member.second;
}
else if (key == "from")
{
members.from = &member.second;
}
break;
case 5:
if (key == "value")
{
members.value = &member.second;
}
break;
default:
break;
}
}
return members;
}
/// @brief find the members by lookup for key types other than string_t
static patch_members find_patch_members(const basic_json& val, std::false_type /*key_type is not string_t*/)
{
const auto find_member = [&val](const char* name) -> basic_json *
{
const auto it = val.m_data.m_value.object->find(string_t(name));
return it != val.m_data.m_value.object->end() ? &it->second : nullptr;
};
patch_members members;
members.op = find_member("op");
members.path = find_member("path");
members.value = find_member("value");
members.from = find_member("from");
return members;
}
public:
/// @brief applies a JSON patch in-place without copying the object
/// @sa https://json.nlohmann.me/api/basic_json/patch/
void patch_inplace(const basic_json& json_patch)
@@ -39626,18 +39361,18 @@ public:
// a diff creates them: apply the operation and return true; return
// false, with nothing changed, for all others
string_t token;
typename json_pointer::template plain_steps<basic_json> steps;
const auto try_plain = [&result, &token, &steps](patch_operations operation, const string_t& path, const patch_members & members, const basic_json & val)
const string_t value_key = "value";
const auto try_plain = [&result, &token, &value_key](patch_operations operation, const string_t& path, const basic_json & val)
{
if (path.empty())
if (path.empty() || path.find_first_of('~') != string_t::npos)
{
return false;
}
// a missing "value" is reported below
const basic_json* const value_ptr = members.value;
const basic_json& value = value_ptr != nullptr ? *value_ptr : val;
if (operation != patch_operations::remove && value_ptr == nullptr)
const auto value_it = val.m_data.m_value.object->find(value_key);
const basic_json& value = value_it != val.m_data.m_value.object->end() ? value_it->second : val;
if (operation != patch_operations::remove && value_it == val.m_data.m_value.object->end())
{
return false;
}
@@ -39646,7 +39381,7 @@ public:
{
case patch_operations::replace:
{
basic_json* const target = json_pointer::resolve_plain(&result, path, path.size(), token, steps);
basic_json* const target = json_pointer::resolve_plain(&result, path, path.size(), token);
if (target == nullptr)
{
return false;
@@ -39657,7 +39392,7 @@ public:
case patch_operations::test:
{
const basic_json* const target = json_pointer::resolve_plain(&result, path, path.size(), token, steps);
const basic_json* const target = json_pointer::resolve_plain(&result, path, path.size(), token);
// a failing test is reported below
return target != nullptr && *target == value;
}
@@ -39677,17 +39412,13 @@ public:
}
--slash;
basic_json* const parent = json_pointer::resolve_plain(&result, path, slash, token, steps);
basic_json* const parent = json_pointer::resolve_plain(&result, path, slash, token);
if (parent == nullptr)
{
return false;
}
token.clear();
token.append(path.data() + slash + 1, path.size() - slash - 1);
if (token.find_first_of('~') != string_t::npos)
{
return false;
}
if (operation == patch_operations::add)
{
@@ -39743,32 +39474,31 @@ public:
}
};
// the name of the first member to check
const string_t op_member = "op";
// iterate and apply the operations
for (const auto& val : json_patch)
{
// wrapper to get a value for an operation
const auto get_value = [&val](basic_json * found,
const string_t& op,
const char* member,
const auto get_value = [&val](const string_t& op,
const string_t& member,
bool string_type) -> basic_json &
{
// find value
auto it = val.m_data.m_value.object->find(member);
// check if the desired value is present
if (JSON_HEDLEY_UNLIKELY(found == nullptr))
if (JSON_HEDLEY_UNLIKELY(it == val.m_data.m_value.object->end()))
{
detail::throw_patch_missing_member(op, string_t(member), false, val);
detail::throw_patch_missing_member(op, member, false, val);
}
// check if the result is of type string
if (JSON_HEDLEY_UNLIKELY(string_type && !found->is_string()))
if (JSON_HEDLEY_UNLIKELY(string_type && !it->second.is_string()))
{
detail::throw_patch_missing_member(op, string_t(member), true, val);
detail::throw_patch_missing_member(op, member, true, val);
}
// no error: return value
return *found;
return it->second;
};
// type check: every element of the array must be an object
@@ -39777,30 +39507,26 @@ public:
detail::throw_patch_not_an_array(val);
}
const patch_members members = find_patch_members(val, std::is_same<typename object_t::key_type, string_t> {});
// collect mandatory members
const string_t& op = *get_value(members.op, op_member, "op", true).m_data.m_value.string;
const string_t& path = *get_value(members.path, op, "path", true).m_data.m_value.string;
const string_t& op = *get_value("op", "op", true).m_data.m_value.string;
const string_t& path = *get_value(op, "path", true).m_data.m_value.string;
const patch_operations operation = get_op(op);
// the usual operations on a plain pointer are applied without
// building the pointer's reference tokens; anything else (and
// every error) is handled below
if (try_plain(operation, path, members, val))
if (try_plain(operation, path, val))
{
continue;
}
// the general operations may change any container
steps.reset();
json_pointer ptr(path);
switch (operation)
{
case patch_operations::add:
{
operation_add(ptr, get_value(members.value, "add", "value", false));
operation_add(ptr, get_value("add", "value", false));
break;
}
@@ -39813,13 +39539,13 @@ public:
case patch_operations::replace:
{
// the "path" location must exist - use at()
result.at(ptr) = get_value(members.value, "replace", "value", false);
result.at(ptr) = get_value("replace", "value", false);
break;
}
case patch_operations::move:
{
const string_t& from_path = *get_value(members.from, "move", "from", true).m_data.m_value.string;
const string_t& from_path = *get_value("move", "from", true).m_data.m_value.string;
json_pointer from_ptr(from_path);
if (JSON_HEDLEY_UNLIKELY(is_proper_prefix(from_ptr, ptr)))
@@ -39842,7 +39568,7 @@ public:
case patch_operations::copy:
{
const string_t& from_path = *get_value(members.from, "copy", "from", true).m_data.m_value.string;
const string_t& from_path = *get_value("copy", "from", true).m_data.m_value.string;
const json_pointer from_ptr(from_path);
// The copy is functionally identical to an "add"
@@ -39860,7 +39586,7 @@ public:
{
// check if "value" matches the one at "path"
// the "path" location must exist - use at()
success = (result.at(ptr) == get_value(members.value, "test", "value", false));
success = (result.at(ptr) == get_value("test", "value", false));
}
JSON_INTERNAL_CATCH (out_of_range&)
{
@@ -39952,18 +39678,13 @@ public:
/// @brief append an operation @a op for @a path to @a result, with @a value if it is not null
static void diff_operation(basic_json& result, const char* op, const string_t& path, const basic_json* value)
{
// the keys are distinct and in the order of std::map; the members are
// added without looking them up
basic_json entry(value_t::object);
auto& members = *entry.m_data.m_value.object;
typename object_t::iterator hint = members.end();
json_pointer::add_flattened(members, hint, string_t("op"), string_t(op), detail::priority_tag<2> {});
json_pointer::add_flattened(members, hint, string_t("path"), path, detail::priority_tag<2> {});
entry["op"] = op;
entry["path"] = path;
if (value != nullptr)
{
json_pointer::add_flattened(members, hint, string_t("value"), *value, detail::priority_tag<2> {});
entry["value"] = *value;
}
entry.set_parents();
result.push_back(std::move(entry));
}
@@ -39985,50 +39706,6 @@ public:
diff_operation(result, "add", path, &value);
}
/// @brief the values in the target of the members of a source object, for the
/// objects of up to 16 members without allocating
///
/// reserve() must be called with the number of values that are added.
class diff_target_values
{
public:
diff_target_values() = default;
diff_target_values(const diff_target_values&) = delete;
diff_target_values& operator=(const diff_target_values&) = delete;
~diff_target_values() = default;
void reserve(const std::size_t n)
{
if (n > inline_size)
{
large.resize(n);
items = large.data();
}
}
void push_back(const basic_json* value)
{
items[count++] = value;
}
std::size_t size() const noexcept
{
return count;
}
const basic_json* operator[](const std::size_t i) const noexcept
{
return items[i];
}
private:
static constexpr std::size_t inline_size = 16;
std::size_t count = 0;
const basic_json** items = inline_items;
std::vector<const basic_json*> large{}; // NOLINT(readability-redundant-member-init)
const basic_json* inline_items[inline_size]; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init)
};
/// @brief append "/" and the decimal representation of @a index to @a path
static void diff_append_index(string_t& path, std::size_t index)
{
@@ -40109,9 +39786,8 @@ public:
operations that remove every member of @a source and add every member of
@a target to @a result, and returns false.
*/
template<typename TargetValues>
static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, TargetValues& target_values,
const string_t& path, std::vector<const basic_json*>& target_values,
std::vector<const_iterator>& added)
{
// first pass: one lookup per source key
@@ -40325,7 +40001,7 @@ public:
case value_t::object:
{
diff_target_values target_values;
std::vector<const basic_json*> target_values;
std::vector<const_iterator> added;
if (diff_object_keys(result, source, target, path, target_values, added))
{
+362 -247
View File
@@ -6089,6 +6089,8 @@ NLOHMANN_JSON_NAMESPACE_END
// #include <nlohmann/detail/view/number.hpp>
// #include <nlohmann/detail/view/simd.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -6146,13 +6148,6 @@ class output_buffer
put(s, N - 1);
}
void put_repeated(char c, std::size_t n)
{
reserve(n);
std::memset(m_pos, c, n);
m_pos += n;
}
/// the write position and the end of the writable space, for a writer
/// that keeps the position in a local variable (set_cursor() hands it back)
char* cursor() const noexcept
@@ -6209,9 +6204,25 @@ class output_buffer
/// image_check::bounds can have others, which are then written unchanged).
inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexcept
{
std::size_t i = 0;
#if NLOHMANN_VIEW_SSE2
// (16 bytes per step; with NEON, the mask of a step costs more than it saves on the
// short runs that are typical here)
for (; i + 16 <= n; i += 16)
{
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(s + i))); // NOLINT(bugprone-casting-through-void)
// (a byte is below 0x20 if it is its own minimum with 0x1F)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmpeq_epi8(_mm_min_epu8(in, _mm_set1_epi8(0x1F)), in));
const auto mask = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
if (mask != 0)
{
return i + static_cast<std::size_t>(count_trailing_zeros(mask));
}
}
#endif
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
const std::uint64_t v = read_eight_bytes(s + i);
@@ -6262,6 +6273,70 @@ inline bool is_ascii_without_delete(const char* s, std::size_t n) noexcept
return true;
}
#if NLOHMANN_VIEW_VECTOR
/// the bytes of the 16 at p that ensure_ascii cannot copy (a quote, a
/// backslash, a control character, DEL, or not ASCII; a signed compare with
/// 0x20 finds control characters and non-ASCII bytes at once), one bit per
/// byte (x86) or one nibble per byte (NEON)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t ascii_stop_mask(const unsigned char* p) noexcept
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vorrq_u8(vceqq_u8(in, vdupq_n_u8(0x7F)), vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20))));
return vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F)), _mm_cmplt_epi8(in, _mm_set1_epi8(0x20))));
return static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
#endif
}
/// the width in bits of a byte in the mask of ascii_stop_mask()
constexpr unsigned ascii_stop_bits() noexcept
{
return NLOHMANN_VIEW_NEON ? 4u : 1u;
}
#endif
/// As find_ascii_copyable_run(), 16 bytes per step. end is the end of the
/// buffer that holds the n bytes at p (a short run is read as 16 bytes if
/// they are in the buffer).
inline std::size_t ascii_copyable_run(const unsigned char* p, std::size_t n, const unsigned char* end) noexcept
{
#if NLOHMANN_VIEW_VECTOR
if (n >= 16)
{
std::size_t i = 0;
for (; i + 16 <= n; i += 16)
{
const std::uint64_t mask = ascii_stop_mask(p + i);
if (mask != 0)
{
return i + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits());
}
}
if (i == n)
{
return n;
}
// the last 16 bytes: those before i are known to be copyable
const std::uint64_t mask = ascii_stop_mask(p + n - 16);
return mask != 0 ? n - 16 + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits()) : n;
}
if (end - p >= 16)
{
// (the bytes after the run are masked out)
const std::uint64_t mask = ascii_stop_mask(p) & ((std::uint64_t{1} << (n * ascii_stop_bits())) - 1u);
return mask != 0 ? static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits() : n;
}
#else
static_cast<void>(end);
#endif
return find_ascii_copyable_run(p, n);
}
/// A stack that starts in a buffer of the caller (a local array) and moves to
/// the heap (a vector of the caller) only when that is full, so that dumps of
/// shallow documents need no allocation. The top is a pointer, as in
@@ -6341,6 +6416,39 @@ NLOHMANN_VIEW_ALWAYS_INLINE char* put_literal(char* w, const char (&s)[N]) noexc
return w + N - 1;
}
/// the depth of the innermost open container, counted only for indented output
template<bool Counted>
struct depth_counter
{
std::size_t value = 0;
void enter() noexcept
{
++value;
}
std::size_t leave() noexcept
{
return --value;
}
std::size_t get() const noexcept
{
return value;
}
};
template<>
struct depth_counter<false>
{
void enter() noexcept {}
std::size_t leave() noexcept
{
return 0;
}
std::size_t get() const noexcept
{
return 0;
}
};
/// how the view's dump() writes a value
struct dump_style
{
@@ -6377,139 +6485,95 @@ class view_serializer
void dump(const node* root)
{
if (!m_style.pretty)
if (m_style.ensure_ascii)
{
if (m_style.ensure_ascii)
{
dump_compact_ascii(root);
}
else if (m_style.source_numbers)
{
dump_compact<true, false>(root);
}
else
{
dump_compact<false, false>(root);
}
return;
dump_ascii(root);
}
struct frame
else if (m_style.pretty)
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
bool first; ///< nothing written yet
};
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
const node* n = root;
for (;;)
dump_pretty(root);
}
else if (m_style.source_numbers)
{
// write the value at n
if (is_container(*n))
{
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
if (n->len == 0)
{
m_out.put(object ? "{}" : "[]", 2);
}
else
{
m_out.put(object ? '{' : '[');
stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
}
}
else
{
write_scalar(*n);
}
// go to the next value: close finished containers, then separate
for (;;)
{
if (stack.empty())
{
m_out.finish();
return;
}
if (stack.back().pos == stack.back().end)
{
const bool object = stack.back().object;
stack.pop_back();
newline(stack.size());
m_out.put(object ? '}' : ']');
continue;
}
frame& f = stack.back();
if (!f.first)
{
m_out.put(',');
}
f.first = false;
newline(stack.size());
if (f.object)
{
write_string(*f.pos);
if (m_style.pretty)
{
m_out.put(": ");
}
else
{
m_out.put(':');
}
n = nav::value(f.pos + 1);
f.pos = document_data::after(f.pos + 1);
}
else
{
n = nav::value(f.pos);
f.pos = document_data::after(f.pos);
}
break;
}
dump_text<true, false, false>(root);
}
else
{
dump_text<false, false, false>(root);
}
}
private:
/// a string that is not a plain string of the source (decoded, or written
/// by an edit), without ensure_ascii: runs without characters to escape
/// are copied
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n)
/// are copied. If validate is set, the string must be valid UTF-8: the
/// runs are found by the lexer's scan, which validates the sequences, and
/// invalid UTF-8 throws what basic_json::dump() throws. (The compact output
/// writes the bytes of a damaged image unchanged, and scans without
/// validating them.)
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n, bool validate)
{
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
m_out.put('"');
for (std::size_t i = 0; i < n.len;)
{
const std::size_t run = plain_output_run(s + i, n.len - i);
const std::size_t run = validate ? string_bulk_run(s + i, n.len - i) : plain_output_run(s + i, n.len - i);
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
if (validate && s[i] >= 0x80)
{
invalid_utf8(s, n.len); // (the scan stopped at ill-formed UTF-8)
}
write_codepoint<false>(s[i], s + i, 1); // a quote, a backslash, or a control character
++i;
}
m_out.put('"');
}
/// dump_compact() with ensure_ascii, out of line: the walk without it is
/// faster when it is not merged with this one into dump()
NLOHMANN_VIEW_NOINLINE void dump_compact_ascii(const node* root)
/// dump_text() with indentation, out of line: the compact walk is faster
/// when it is not merged with the indented ones into dump()
NLOHMANN_VIEW_NOINLINE void dump_pretty(const node* root)
{
if (m_style.source_numbers)
{
dump_compact<true, true>(root);
dump_text<true, true, false>(root);
}
else
{
dump_compact<false, true>(root);
dump_text<false, true, false>(root);
}
}
/// whether dump_compact() copies a string or key from the source: one of
/// dump_text() with ensure_ascii, out of line: the walk without it is
/// faster when it is not merged with this one into dump()
NLOHMANN_VIEW_NOINLINE void dump_ascii(const node* root)
{
if (m_style.pretty)
{
if (m_style.source_numbers)
{
dump_text<true, true, true>(root);
}
else
{
dump_text<false, true, true>(root);
}
}
else if (m_style.source_numbers)
{
dump_text<true, false, true>(root);
}
else
{
dump_text<false, false, true>(root);
}
}
/// whether dump_text() copies a string or key from the source: one of
/// the source, and with ensure_ascii one without bytes >= 0x7F (an overload
/// each, so that the walk without ensure_ascii is compiled exactly as
/// before: with GCC, a condition that only folds away later changed its
@@ -6524,13 +6588,154 @@ class view_serializer
return (n.flags & node_flags::storage) == 0 && is_ascii_without_delete(src + n.off, n.len);
}
/// a string with ensure_ascii (dump_compact()), out of line
NLOHMANN_VIEW_NOINLINE void write_ascii_string(const node& n)
/*!
@brief a string with ensure_ascii: the characters that are not printable ASCII as \\uXXXX
(a character beyond U+FFFF as a surrogate pair); strings of a document
are valid UTF-8, except in a damaged image loaded with image_check::bounds,
for which this throws what basic_json::dump() throws for the string
*/
NLOHMANN_VIEW_NOINLINE void write_ascii(const node& n)
{
write_string(n);
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const std::size_t size = n.len;
char* w = m_out.cursor();
char* lim = m_out.limit();
// room for k bytes
const auto room = [&](std::size_t k)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(lim - w) < k))
{
m_out.set_cursor(w);
m_out.reserve(k);
w = m_out.cursor();
lim = m_out.limit();
}
};
static constexpr const char* hex = "0123456789abcdef";
// \\uXXXX
const auto u_escape = [&](std::uint32_t u)
{
w[0] = '\\';
w[1] = 'u';
w[2] = hex[(u >> 12u) & 0xFu];
w[3] = hex[(u >> 8u) & 0xFu];
w[4] = hex[(u >> 4u) & 0xFu];
w[5] = hex[u & 0xFu];
w += 6;
};
room(1);
*w++ = '"';
std::size_t i = 0;
// (a string of the source starts with a run that has nothing to escape, 16 bytes per step)
const std::size_t prefix = (n.flags & node_flags::storage) == 0 ? ascii_copyable_run(s, size, reinterpret_cast<const unsigned char*>(m_doc.src + m_doc.size)) : 0; // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
if (prefix != 0)
{
room(prefix);
std::memcpy(w, s, prefix);
w += prefix;
i = prefix;
}
while (i < size)
{
// (a character that needs an escape sequence is not worth a call)
const std::size_t run = is_ascii_copyable(s[i]) ? find_ascii_copyable_run(s + i, size - i) : 0;
if (run != 0)
{
room(run);
std::memcpy(w, s + i, run);
w += run;
i += run;
continue;
}
room(12);
if (s[i] >= 0x80)
{
const unsigned char lead = s[i];
// (the two-byte sequences of Latin text and the three-byte sequences of most other
// text, with continuation bytes of any value: the ranges that are restricted (E0,
// ED, and the sequences of four bytes) go through validate_one_utf8())
if (lead >= 0xE1 && lead <= 0xEF && lead != 0xED && size - i >= 3 && (s[i + 1] & 0xC0u) == 0x80 && (s[i + 2] & 0xC0u) == 0x80)
{
u_escape(((lead & 0x0Fu) << 12u) | ((s[i + 1] & 0x3Fu) << 6u) | (s[i + 2] & 0x3Fu));
i += 3;
continue;
}
if (lead >= 0xC2 && lead <= 0xDF && size - i >= 2 && (s[i + 1] & 0xC0u) == 0x80)
{
u_escape(((lead & 0x1Fu) << 6u) | (s[i + 1] & 0x3Fu));
i += 2;
continue;
}
const std::size_t len = validate_one_utf8(s + i, size - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
m_out.set_cursor(w);
invalid_utf8(s, size);
return; // LCOV_EXCL_LINE (invalid_utf8 throws)
}
std::uint32_t codepoint = s[i] & (0xFFu >> (len + 1));
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
i += len;
if (codepoint <= 0xFFFF)
{
u_escape(codepoint);
}
else
{
u_escape(0xD7C0u + (codepoint >> 10u));
u_escape(0xDC00u + (codepoint & 0x3FFu));
}
continue;
}
// a quote, a backslash, or a control character (or DEL)
const unsigned char c = s[i++];
char escape = 0;
switch (c)
{
case 0x08:
escape = 'b';
break;
case 0x09:
escape = 't';
break;
case 0x0A:
escape = 'n';
break;
case 0x0C:
escape = 'f';
break;
case 0x0D:
escape = 'r';
break;
case 0x22:
escape = '"';
break;
case 0x5C:
escape = '\\';
break;
default:
break;
}
if (escape != 0)
{
w[0] = '\\';
w[1] = escape;
w += 2;
}
else
{
u_escape(c);
}
}
room(1);
*w++ = '"';
m_out.set_cursor(w);
}
/// the copies of dump_compact() that are not fixed-size moves (long
/// the copies of dump_text() that are not fixed-size moves (long
/// strings, or near the end of the source); out of line, so that the
/// compiler does not merge the fixed-size moves into this call
NLOHMANN_VIEW_NOINLINE static void copy_long(char* to, const char* from, std::size_t n) noexcept
@@ -6547,7 +6752,7 @@ class view_serializer
}
/*!
@brief the compact output (the default dump(), or with ensure_ascii)
@brief the output, compact (the default dump()) or indented, with or without ensure_ascii
The same walk as dump(), with the write position in a local variable
(stores through char pointers would otherwise force a reload of the
@@ -6555,11 +6760,11 @@ class view_serializer
the source copied by fixed-size moves of 32 bytes where the source has
that many bytes left, instead of a library call per token. The buffer
keeps 64 bytes of slack for the overshoot. With ensure_ascii (Ascii), a
string of the source is copied only if it has no byte >= 0x7F (see
is_ascii_without_delete()); any other string is written by write_string().
string of the source is copied only if it has nothing to escape (see
copied()); any other string is written by write_ascii().
*/
template<bool SourceNumbers, bool Ascii>
void dump_compact(const node* root)
template<bool SourceNumbers, bool Pretty, bool Ascii>
void dump_text(const node* root)
{
struct frame
{
@@ -6574,6 +6779,16 @@ class view_serializer
const char* const src_end = src + m_doc.size;
char* w = m_out.cursor();
char* lim = m_out.limit();
// (indented output) a line break and 63 indentation characters, from
// which the line break of a level of up to 63 characters is a copy
std::array<char, 64> ruler; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled below if used
if (Pretty)
{
ruler.fill(m_style.indent_char);
ruler[0] = '\n';
}
const std::size_t indent = m_style.indent;
depth_counter<Pretty> level; // depth of the innermost open container (indented output)
// room for n bytes and the slack
const auto room = [&](std::size_t n)
{
@@ -6605,6 +6820,22 @@ class view_serializer
}
w += n;
};
// (indented output) the line break and the indentation of a level
const auto newline = [&](std::size_t depth)
{
const std::size_t count = 1 + (depth * indent);
room(count);
if (count <= ruler.size())
{
std::memcpy(w, ruler.data(), ruler.size());
}
else
{
*w = '\n';
std::memset(w + 1, m_style.indent_char, count - 1);
}
w += count;
};
// a string that is not a plain string of the source (out of line, so
// that the cursor stays in a register here)
const auto escaped = [&](const node & n)
@@ -6612,11 +6843,11 @@ class view_serializer
m_out.set_cursor(w);
if (enabled(Ascii))
{
write_ascii_string(n);
write_ascii(n);
}
else
{
write_decoded(n);
write_decoded(n, Pretty);
}
w = m_out.cursor();
lim = m_out.limit();
@@ -6717,6 +6948,7 @@ class view_serializer
{
*w++ = object ? '{' : '[';
stack.push_back(cur);
level.enter();
if (Editable)
{
cur = frame{nav::first(m_doc, n), nav::end(m_doc, n), object};
@@ -6759,6 +6991,10 @@ class view_serializer
{
break;
}
if (Pretty)
{
newline(level.leave());
}
room(1);
*w++ = cur.object ? '}' : ']';
cur = stack.back();
@@ -6767,13 +7003,17 @@ class view_serializer
room(1);
*w++ = ',';
}
if (Pretty)
{
newline(level.get());
}
const node* const at = Editable ? cur.pos : n;
if (cur.object)
{
const node& key = *at;
if (copied(key, src, std::integral_constant<bool, Ascii>()))
{
room(key.len + 3);
room(key.len + (Pretty ? 4 : 3));
*w++ = '"';
copy(src + key.off, key.len);
w[0] = '"';
@@ -6783,9 +7023,13 @@ class view_serializer
else
{
escaped(key);
room(1);
room(Pretty ? 2 : 1);
*w++ = ':';
}
if (Pretty)
{
*w++ = ' ';
}
if (Editable)
{
n = nav::value(at + 1);
@@ -6804,69 +7048,6 @@ class view_serializer
}
}
void newline(std::size_t level)
{
if (m_style.pretty)
{
m_out.put('\n');
m_out.put_repeated(m_style.indent_char, level * m_style.indent);
}
}
void write_scalar(const node& n)
{
switch (static_cast<value_t>(n.kind))
{
case value_t::null:
m_out.put("null");
break;
case value_t::boolean:
if ((n.flags & node_flags::is_true) != 0)
{
m_out.put("true");
}
else
{
m_out.put("false");
}
break;
case value_t::string:
write_string(n);
break;
case value_t::number_integer:
case value_t::number_unsigned:
{
const char* const token = m_doc.str(n);
const std::uint32_t len = number_length(n);
if (!m_style.source_numbers && len == 2 && token[0] == '-' && token[1] == '0')
{
m_out.put('0'); // parse() reads -0 as the integer 0
}
else
{
m_out.put(token, len);
}
break;
}
case value_t::number_float:
if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
{
m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
}
else
{
write_float_node(n);
}
break;
case value_t::object: // LCOV_EXCL_LINE (containers are written by dump())
case value_t::array: // LCOV_EXCL_LINE
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
}
/// a float node as dump() writes it
void write_float_node(const node& n)
{
@@ -6977,72 +7158,6 @@ class view_serializer
m_out.put(s.data(), s.size());
}
void write_string(const node& n)
{
const char* const s = m_doc.str(n);
m_out.put('"');
if ((n.flags & node_flags::storage) == 0 && (!m_style.ensure_ascii || is_ascii_without_delete(s, n.len)))
{
// a string of the source without escape sequences has nothing to
// escape (with ensure_ascii, if it is printable ASCII)
m_out.put(s, n.len);
}
else if (m_style.ensure_ascii)
{
write_escaped<true>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
else
{
write_escaped<false>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
m_out.put('"');
}
/// as serializer::dump_escaped(); strings of a document are valid UTF-8,
/// except in a damaged image loaded with image_check::bounds, for which
/// this throws what basic_json::dump() throws for the string
template<bool EnsureAscii>
void write_escaped(const unsigned char* s, std::size_t n)
{
std::size_t i = 0;
while (i < n)
{
std::size_t run = 0;
if (!EnsureAscii)
{
run = string_bulk_run(s + i, n - i);
}
else if (is_ascii_copyable(s[i]))
{
run = find_ascii_copyable_run(s + i, n - i);
}
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
std::uint32_t codepoint = s[i];
std::size_t len = 1;
if (codepoint >= 0x80)
{
len = validate_one_utf8(s + i, n - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
invalid_utf8(s, n);
return;
}
codepoint &= 0xFFu >> (len + 1);
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
}
write_codepoint<EnsureAscii>(codepoint, s + i, len);
i += len;
}
}
/// throw what basic_json::dump() throws for a string that is not valid UTF-8
NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
{
+125
View File
@@ -186,6 +186,131 @@ TEST_CASE("json_view dump")
CHECK(deep_document.root().dump() == deep);
}
SECTION("ensure_ascii: a character that needs an escape at each position of a string")
{
// string lengths around the width of the vector steps, in a string value, a key, and a value near the end of the text
for (const char* const special :
{
"\xC3\xA9", "\xE4\xB8\xAD", "\xF0\x9F\x98\x80", "\x7F", "\\n", "\\u0001", "\\\"", "\\\\"
})
{
for (std::size_t length = 0; length <= 40; ++length)
{
for (std::size_t position = 0; position <= length; ++position)
{
std::string content(length, 'a');
content.insert(position, special);
const std::string texts[] =
{
"\"" + content + "\"",
"[\"" + content + "\"]",
"{\"" + content + "\":1}",
};
for (const std::string& text : texts)
{
CAPTURE(text)
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
}
}
}
}
}
SECTION("ensure_ascii: every character of the Basic Multilingual Plane and some beyond")
{
// (the two- and three-byte sequences, including the ranges that need a check: E0, ED, EF BF BF)
std::string text = "[";
std::size_t in_chunk = 0;
const auto append = [&](std::uint32_t cp)
{
if (cp < 0x80)
{
if (cp < 0x20 || cp == '"' || cp == '\\')
{
return;
}
text += static_cast<char>(cp);
}
else if (cp < 0x800)
{
text += static_cast<char>(0xC0 | (cp >> 6));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else if (cp < 0x10000)
{
text += static_cast<char>(0xE0 | (cp >> 12));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else
{
text += static_cast<char>(0xF0 | (cp >> 18));
text += static_cast<char>(0x80 | ((cp >> 12) & 0x3F));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
};
for (std::uint32_t cp = 0x20; cp < 0x10400; ++cp)
{
if (cp >= 0xD800 && cp < 0xE000)
{
continue;
}
if (in_chunk == 0)
{
text += text.size() > 1 ? ",\"" : "\"";
}
append(cp);
if (++in_chunk == 37)
{
text += "\"";
in_chunk = 0;
}
}
text += in_chunk != 0 ? "\"]" : "]";
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
CHECK(d.root().dump(2) == j.dump(2));
}
SECTION("deep nesting, indented")
{
// indentations of up to and beyond the width of the writer's fixed-size copy
for (const int depth :
{
1, 5, 20, 70, 300
})
{
std::string text;
for (int i = 0; i < depth; ++i)
{
text += i % 2 == 0 ? "[1," : "{\"k\":2,\"\\u00e9\":";
}
text += "[]";
for (int i = depth - 1; i >= 0; --i)
{
text += i % 2 == 0 ? "]" : "}";
}
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
for (const int indent :
{
0, 1, 2, 3, 8
})
{
CHECK(d.root().dump(indent) == j.dump(indent));
CHECK(d.root().dump(indent, '\t') == j.dump(indent, '\t'));
}
}
}
SECTION("the output buffer of a small value is small")
{
// an escaped key after the value: its node lies in the arena, so the
+20
View File
@@ -682,6 +682,26 @@ TEST_CASE("json_view images: ownership")
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("json_view images: errors")
{
SECTION("invalid UTF-8 of a decoded string: the compact dump writes it, the others throw")
{
const json_document d = json_document::parse(R"({"k\n":"\u00e9x"})");
std::vector<std::uint8_t> image = d.save();
const std::size_t arena = text_at(image) + header_size_field(image, 16) + 1;
image[arena + 3] = 0x28; // the key is "k\n"; the value starts at 3: 0xC3 0x28 is invalid
const json_document loaded = json_document::load(image, image_check::bounds);
CHECK(loaded.root().dump().size() == 13);
for (const int indent :
{
0, 2
})
{
CHECK_THROWS_AS(loaded.root().dump(indent), json::type_error);
CHECK_THROWS_AS(loaded.root().dump(indent, ' ', true), json::type_error);
}
CHECK_THROWS_AS(loaded.root().dump(-1, ' ', true), json::type_error);
CHECK_THROWS_WITH_AS(loaded.root().dump(2), "[json.exception.type_error.316] invalid UTF-8 byte at index 1: 0x28", json::type_error&);
}
SECTION("a literal as the root: dump() after loading")
{
for (const char* text :