Compare commits

..
8 Commits
Author SHA1 Message Date
Niels Lohmann 8380792ff3 Regenerate the amalgamation
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 22:46:36 +02:00
Niels Lohmann 395f2f83a9 Merge branch 'develop' into cbor-integer-writer
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 22:46:33 +02:00
Niels Lohmann d60ac0aff1 Merge branch 'develop' into cbor-integer-writer
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 20:26:44 +02:00
Niels Lohmann 35d1ec5ece Fix clang-tidy and GCC warnings in the integer array test
Use std::array for the boundary table (modernize-avoid-c-arrays) and drop
the casts to number_integer_t, which is std::int64_t already
(-Wuseless-cast).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 13:10:20 +02:00
Niels Lohmann 3aa3d87983 Use std::array for the integer size class table
clang-tidy (cppcoreguidelines-avoid-c-arrays) rejects the C-style array.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 19:34:12 +02:00
Niels Lohmann 5010011656 Write MessagePack integers without branching on their sign or size
Like CBOR before #5824, MessagePack integer arrays were no faster than in
3.12.0: the type switch (number_integer vs. number_unsigned), the sign
test, and the fixint/8/16/32/64-bit ladder mispredict on mixed data.

They now take the CBOR approach:

- number_integer and number_unsigned share one case of write_msgpack.
- The format comes from the size class of x, where x is the value itself
  if it is not negative and 2 * (-1 - n) + 1 for a negative n: n fits
  int 8/16/32 exactly if that fits uint 8/16/32, so one table lookup
  serves both signs. The fixint bound (127, or -32) is selected through a
  mask, and the marker is 0xCB + class for uint and 0xCF + class for int.
- Arrays are written by an out-of-line write_msgpack_array() with runs of
  integers encoded straight into space reserved from the sink.

The CBOR integer code is factored into helpers both formats share
(integer_bits, integer_size_class, encode_first_byte_and_tail,
write_array_elements). The new path picks the format from the low 64
bits, so number types wider than that keep the previous range checks.

The output is unchanged; the integer-array test now also checks
MessagePack against an independent encoder.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 14:26:24 +02:00
Niels Lohmann bbb33374d4 Write CBOR integers without branching on their sign or size
The sizes of a run of integers are hard to predict, and so is the type of
each element in an array that mixes negative (number_integer) and
non-negative (number_unsigned) values. Writing an integer branched on the
type, the sign, and the 1/2/4/8-byte size class, and the mispredictions
were the larger part of the time left on integer arrays.

- number_integer and number_unsigned share one case of write_cbor; the
  major type and the argument are selected with a sign mask.
- An integer's head is assembled without branches: the size class comes
  from a table indexed by count_leading_zeros(), and the first byte and
  the argument in network order are stored as one byte and one 8-byte
  word, of which only the head's length is kept. Heads of strings and
  containers keep the branching write_cbor_head(), as their lengths are
  mostly predictable.
- Arrays are written by write_cbor_array(), kept out of line so that the
  registers its loop needs are not saved on every call of write_cbor.
  Runs of integers are encoded straight into space reserved from the sink
  (the new reserve()/commit() of both sinks), so the write pointer stays
  in a register instead of being reloaded for every value.

The output is unchanged; the new test compares integer arrays at every
head-size boundary, in runs shorter and longer than one reservation and
broken up by other values, with an independent encoder through each sink.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 13:48:16 +02:00
Niels Lohmann e26e1ad312 Append to the output vector through a position-tracking sink
The vector-returning to_cbor(), to_msgpack(), ... wrote through
output_vector_sink, which appended each write with push_back() or
insert(). A number's 1 to 8 payload bytes went through vector::insert(),
which libc++ and libstdc++ both run as an out-of-line insert-anywhere
routine; on integer arrays that call cost more than encoding the number,
so the devirtualization of #5286 brought nothing there, and with GCC the
writer was slower than in 3.12.0.

output_vector_sink now keeps its own write position: it grows the vector
geometrically ahead of the position (resize(), using any capacity the
vector already has), copies each write there, and shrinks the vector to
the written length in its destructor, also when a write throws. Writes of
256 characters or more that do not fit are appended with insert(), so a
large string or binary value is copied once instead of zero-filled and
copied. The sink is move-only, as it owns that trim.

output_vector_adapter (the to_*(j, vector) overloads) appends with
push_back()/insert() again, so a caller holding the adapter never sees
the space kept ahead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 13:47:36 +02:00
25 changed files with 637 additions and 867 deletions

No files matched your search

+8 -10
View File
@@ -166,21 +166,19 @@ jobs:
# link, but the binaries clang 11.0.1 and clang 18.1.8 then produce crash
# before doctest prints its first line - 39 of 102 tests on clang 18.
# Keep the objects small by splitting the test files instead.
# The tests link with LLD: GNU ld fails on objects with more than 32767
# sections with "relocation truncated to fit: IMAGE_REL_AMD64_REL32
# against `.rdata'", as it stores the section an associative COMDAT
# section belongs to in 16 bits (x_associated in binutils'
# include/coff/internal.h) and so discards the jump tables of inline
# functions together with the wrong function. Each basic_json
# specialization and input type a test file instantiates adds many
# sections, so unit-msgpack.cpp and unit-regression2.cpp kept crossing
# that limit as the library grew.
# Objects with more than 65535 sections fail to link with "relocation
# truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'": the MinGW
# linker stores the section an associative COMDAT section belongs to in
# 16 bits (x_associated in binutils' include/coff/internal.h), so it
# discards the jump tables of inline functions together with the wrong
# function. Each basic_json specialization a test file instantiates adds
# many sections, so test a second one in a file of its own.
- name: Run CMake
run: cmake -S . -B build ^
-DCMAKE_CXX_COMPILER="C:/Program Files/LLVM/bin/clang++.exe" ^
-DCMAKE_CXX_FLAGS="--target=x86_64-w64-mingw32 -stdlib=libstdc++ -pthread" ^
-DCMAKE_CXX_FLAGS_DEBUG="-g0" ^
-DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld -lwinpthread" ^
-DCMAKE_EXE_LINKER_FLAGS="-lwinpthread" ^
-G"MinGW Makefiles" ^
-DCMAKE_BUILD_TYPE=Debug ^
-DJSON_BuildTests=On
@@ -13,7 +13,7 @@
#include <optional> // optional
#endif
#include <iterator> // begin, end, make_move_iterator, move_iterator
#include <iterator> // begin, end
#include <memory> // allocator_traits
#include <string> // basic_string, char_traits
#include <tuple> // tuple, get
@@ -48,32 +48,6 @@ namespace detail
* leave j with the type of the new value, but the pointer to the destroyed old one.
*/
// a move iterator over the elements of an rvalue container, unless they are
// const (e.g., std::set): libstdc++ before GCC 5 defines the reference of a
// std::move_iterator as value_type&&, which cannot bind to a const element
template<typename Iterator>
std::move_iterator<Iterator> make_move_iterator_if_mutable_impl(const Iterator& it, std::true_type /*mutable*/)
{
return std::make_move_iterator(it);
}
template<typename Iterator>
Iterator make_move_iterator_if_mutable_impl(const Iterator& it, std::false_type /*mutable*/)
{
return it;
}
template<typename Iterator>
using is_mutable_iterator = std::integral_constant < bool,
!std::is_const<typename std::remove_reference<decltype(*std::declval<Iterator&>())>::type>::value >;
template<typename Iterator>
auto make_move_iterator_if_mutable(const Iterator& it)
-> decltype(make_move_iterator_if_mutable_impl(it, is_mutable_iterator<Iterator> {}))
{
return make_move_iterator_if_mutable_impl(it, is_mutable_iterator<Iterator> {});
}
template<value_t> struct external_constructor;
template<>
@@ -200,7 +174,7 @@ struct external_constructor<value_t::array>
using std::begin;
using std::end;
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::array_t>(make_move_iterator_if_mutable(begin(arr)), make_move_iterator_if_mutable(end(arr))));
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::array_t>(std::make_move_iterator(begin(arr)), std::make_move_iterator(end(arr))));
j.replace_value(value_t::array, value);
}
@@ -261,7 +235,7 @@ struct external_constructor<value_t::object>
using std::begin;
using std::end;
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::object_t>(make_move_iterator_if_mutable(begin(obj)), make_move_iterator_if_mutable(end(obj))));
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::object_t>(std::make_move_iterator(begin(obj)), std::make_move_iterator(end(obj))));
j.replace_value(value_t::object, value);
}
@@ -439,27 +413,25 @@ using is_movable_container_rvalue = negation<disjunction<
std::is_const<T>,
has_dedicated_to_json<T>,
is_basic_json<T>,
is_range_view_type<T>,
is_enum_keyed_map_type<T>,
std::is_same<T, std::vector<bool>>>>;
// rvalue array-like containers; object types are ruled out before views are
// checked, as std::ranges::view<T> for the object_t of an ordered_json would
// need that ordered_json to be complete (GCC 11, see #3312)
// rvalue array-like containers
template<typename BasicJsonType, typename T>
using is_rvalue_array_to_json = conjunction<is_movable_container_rvalue<T>,
negation<is_compatible_object_type<BasicJsonType, T>>,
negation<is_range_view_type<T>>,
is_compatible_array_type<BasicJsonType, T>,
negation<disjunction<
std::is_same<T, typename BasicJsonType::array_t>,
std::is_same<T, typename BasicJsonType::binary_t>,
is_compatible_object_type<BasicJsonType, T>,
is_compatible_string_type<BasicJsonType, T>,
is_compatible_binary_type<BasicJsonType, T>>>>;
// rvalue containers: their elements are moved instead of copied
template<typename BasicJsonType, typename CompatibleArrayType,
enable_if_t<is_rvalue_array_to_json<BasicJsonType, CompatibleArrayType>::value, int> = 0>
inline void to_json(BasicJsonType& j, CompatibleArrayType && arr) // NOLINT(cppcoreguidelines-missing-std-forward)
inline void to_json(BasicJsonType& j, CompatibleArrayType && arr)
{
external_constructor<value_t::array>::construct_moving(j, arr);
}
@@ -537,8 +509,7 @@ inline void to_json(BasicJsonType& j, const EnumKeyedMap& map)
}
#endif
// rvalue object-like containers (views have no key_type and mapped_type, so
// they are no compatible object type)
// rvalue object-like containers
template<typename BasicJsonType, typename T>
using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>,
negation<std::is_same<T, typename BasicJsonType::object_t>>,
@@ -546,7 +517,7 @@ using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>,
template<typename BasicJsonType, typename CompatibleObjectType,
enable_if_t<is_rvalue_object_to_json<BasicJsonType, CompatibleObjectType>::value, int> = 0>
inline void to_json(BasicJsonType& j, CompatibleObjectType && obj) // NOLINT(cppcoreguidelines-missing-std-forward)
inline void to_json(BasicJsonType& j, CompatibleObjectType && obj)
{
external_constructor<value_t::object>::construct_moving(j, obj);
}
+1 -1
View File
@@ -185,7 +185,7 @@ JSON_HEDLEY_ALWAYS_INLINE shortest_decimal to_shortest(std::uint64_t bits) noexc
const uint128_parts p = umul192_hi128(p10.high, p10.low, bin_sig << static_cast<unsigned>(shift));
std::uint64_t integral = p.high >> static_cast<unsigned>(extra_shift);
// (no casts in the shifts: MSVC 2015 warns C4554 on them despite the parentheses)
constexpr auto fractional_low_shift = static_cast<unsigned>(extra_shift);
constexpr unsigned fractional_low_shift = static_cast<unsigned>(extra_shift);
constexpr unsigned fractional_high_shift = 64u - fractional_low_shift;
const std::uint64_t fractional = (p.high << fractional_high_shift) | (p.low >> fractional_low_shift);
std::uint64_t digit = 0;
+2 -1
View File
@@ -494,7 +494,8 @@ JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE void throw_patch_missing_member(c
static_cast<void>(string_type);
static_cast<void>(val);
// context-sensitive error message
JSON_THROW(parse_error::create(exception_id::patch_invalid_operation, 0, concat((op == "op") ? std::string("operation") : concat("operation '", op, '\''), string_type ? " must have string member '" : " must have member '", member, "'"), &val));
const std::string error_msg = (op == "op") ? std::string("operation") : concat("operation '", op, '\'');
JSON_THROW(parse_error::create(exception_id::patch_invalid_operation, 0, concat(error_msg, string_type ? " must have string member '" : " must have member '", member, "'"), &val));
}
template<typename StringType, typename BasicJsonType>
@@ -680,12 +680,12 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
{
JSON_ASSERT(m_object != nullptr);
if (JSON_HEDLEY_UNLIKELY(!m_object->is_object()))
if (JSON_HEDLEY_LIKELY(m_object->is_object()))
{
throw_literal<invalid_iterator>(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators", m_object);
return m_it.object_iterator->first;
}
return m_it.object_iterator->first;
throw_literal<invalid_iterator>(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators", m_object);
}
/*!
+3 -4
View File
@@ -11,7 +11,6 @@
#include <algorithm> // all_of
#include <cctype> // isdigit
#include <cerrno> // errno, ERANGE
#include <cstddef> // size_t
#include <cstdlib> // strtoull
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
@@ -335,7 +334,7 @@ class json_pointer
{
break; // not a plain number: let strtoull decide
}
acc = static_cast<size_type>((acc * 10) + static_cast<size_type>(c - '0'));
acc = static_cast<size_type>(acc * 10 + static_cast<size_type>(c - '0'));
if (JSON_HEDLEY_UNLIKELY(i == s.size() - 1))
{
idx = acc;
@@ -530,10 +529,10 @@ class json_pointer
static std::size_t hash_of(std::size_t parent, const string_t& token) noexcept
{
std::size_t h = parent * static_cast<std::size_t>(0x9E3779B97F4A7C15ULL & (std::numeric_limits<std::size_t>::max)());
std::size_t h = parent * static_cast<std::size_t>(0x9E3779B97F4A7C15ULL);
for (std::size_t i = 0; i < token.size(); ++i)
{
h = (h ^ static_cast<unsigned char>(token[i])) * static_cast<std::size_t>(0x100000001B3ULL & (std::numeric_limits<std::size_t>::max)());
h = (h ^ static_cast<unsigned char>(token[i])) * static_cast<std::size_t>(0x100000001B3ULL);
}
return h ^ (h >> 29u);
}
+49 -51
View File
@@ -29,7 +29,6 @@
#endif // JSON_NO_IO
#include <iterator> // make_move_iterator, random_access_iterator_tag
#include <limits> // numeric_limits
#include <map> // map
#include <memory> // unique_ptr
#include <set> // swap, operator!=
#include <stdexcept> // length_error
@@ -755,20 +754,26 @@ private:
// one level deep); stops at the first structured child it meets
static bool all_children_leaves(const basic_json& v) noexcept
{
// array_t only needs forward iterators (see custom_array_type)
if (v.m_data.m_type == value_t::array)
{
const auto& a = *v.m_data.m_value.array;
return std::all_of(a.begin(), a.end(), [](const basic_json & el)
for (auto it = a.rbegin(); it != a.rend(); ++it)
{
return has_no_children(el);
});
if (!has_no_children(*it))
{
return false;
}
}
return true;
}
const auto& o = *v.m_data.m_value.object;
return std::all_of(o.begin(), o.end(), [](const typename object_t::value_type & el)
for (const auto& el : *v.m_data.m_value.object)
{
return has_no_children(el.second);
});
if (!has_no_children(el.second))
{
return false;
}
}
return true;
}
// The walk in destroy_container() may take the children of a
@@ -1225,7 +1230,7 @@ public:
}
template < typename O = object_t, detail::enable_if_t < detail::has_stable_elements<O>::value, int > = 0 >
std::size_t object_capacity() const noexcept
static constexpr std::size_t object_capacity() noexcept
{
return 0;
}
@@ -1237,7 +1242,7 @@ public:
}
template < typename O = object_t, detail::enable_if_t < !detail::has_stable_elements<O>::value && !detail::is_nlohmann_ordered_map<O>::value, int > = 0 >
std::size_t object_capacity() const noexcept
static constexpr std::size_t object_capacity() noexcept
{
return detail::unknown_size();
}
@@ -3038,21 +3043,18 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_iterator_range_of_null(const basic_json* obj)
{
static_cast<void>(obj); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_of_null, detail::concat("cannot construct with iterators from ", obj->type_name()), obj));
}
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_incompatible_reference_type(const basic_json& obj)
{
static_cast<void>(obj); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(type_error::create(detail::exception_id::incompatible_reference_type, detail::concat("incompatible ReferenceType for get_ref, actual type is ", obj.type_name()), &obj));
}
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_array_index_exceeds_size_type(size_type idx)
{
static_cast<void>(idx); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(std::length_error(detail::concat("array index ", std::to_string(idx), " exceeds size_type")));
}
@@ -3060,8 +3062,6 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_array_index_out_of_range(size_type idx, const basic_json& j)
{
static_cast<void>(idx); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(j);
JSON_THROW(out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat("array index ", std::to_string(idx), " is out of range"), &j));
}
@@ -3070,8 +3070,6 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_key_not_found(const KeyType& key, const basic_json& j)
{
static_cast<void>(key); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(j);
JSON_THROW(out_of_range::create(detail::exception_id::key_not_found, detail::concat("key '", key, "' not found"), &j));
}
@@ -4895,19 +4893,19 @@ public:
iterator insert(const_iterator pos, const basic_json& val) // NOLINT(performance-unnecessary-value-param)
{
// insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array()))
if (JSON_HEDLEY_LIKELY(is_array()))
{
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// insert to array and return iterator
return insert_iterator(pos, val);
}
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// insert to array and return iterator
return insert_iterator(pos, val);
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts element into array
@@ -4915,21 +4913,21 @@ public:
iterator insert(const_iterator pos, basic_json&& val) // NOLINT(performance-unnecessary-value-param)
{
// insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array()))
if (JSON_HEDLEY_LIKELY(is_array()))
{
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// moving into a local first keeps this safe even if val aliases
// an element of this array
basic_json tmp(std::move(val));
return insert_iterator(pos, std::move(tmp));
}
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// moving into a local first keeps this safe even if val aliases
// an element of this array
basic_json tmp(std::move(val));
return insert_iterator(pos, std::move(tmp));
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts copies of element into array
@@ -4937,19 +4935,19 @@ public:
iterator insert(const_iterator pos, size_type cnt, const basic_json& val) // NOLINT(performance-unnecessary-value-param)
{
// insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array()))
if (JSON_HEDLEY_LIKELY(is_array()))
{
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// insert to array and return iterator
return insert_iterator(pos, cnt, val);
}
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// insert to array and return iterator
return insert_iterator(pos, cnt, val);
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts range of elements into array
@@ -5455,7 +5453,7 @@ public:
// inline fast path for two values of the same non-owning scalar type; anything
// else falls through to the out-of-line general comparison
#define JSON_IMPLEMENT_SCALAR_FAST(op, null_result) \
if (lhs.m_data.m_type == rhs.m_data.m_type) /* NOLINT(readability/braces) */ \
if (lhs.m_data.m_type == rhs.m_data.m_type) \
{ \
switch (lhs.m_data.m_type) \
{ \
-1
View File
@@ -23,7 +23,6 @@
#include <nlohmann/detail/abi_macros.hpp>
#include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/meta/type_traits.hpp>
#include <nlohmann/thirdparty/hedley/hedley.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
+67 -100
View File
@@ -29,7 +29,6 @@
#endif // JSON_NO_IO
#include <iterator> // make_move_iterator, random_access_iterator_tag
#include <limits> // numeric_limits
#include <map> // map
#include <memory> // unique_ptr
#include <set> // swap, operator!=
#include <stdexcept> // length_error
@@ -5795,7 +5794,8 @@ JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE void throw_patch_missing_member(c
static_cast<void>(string_type);
static_cast<void>(val);
// context-sensitive error message
JSON_THROW(parse_error::create(exception_id::patch_invalid_operation, 0, concat((op == "op") ? std::string("operation") : concat("operation '", op, '\''), string_type ? " must have string member '" : " must have member '", member, "'"), &val));
const std::string error_msg = (op == "op") ? std::string("operation") : concat("operation '", op, '\'');
JSON_THROW(parse_error::create(exception_id::patch_invalid_operation, 0, concat(error_msg, string_type ? " must have string member '" : " must have member '", member, "'"), &val));
}
template<typename StringType, typename BasicJsonType>
@@ -6555,7 +6555,7 @@ NLOHMANN_JSON_NAMESPACE_END
#include <optional> // optional
#endif
#include <iterator> // begin, end, make_move_iterator, move_iterator
#include <iterator> // begin, end
#include <memory> // allocator_traits
#include <string> // basic_string, char_traits
#include <tuple> // tuple, get
@@ -7204,32 +7204,6 @@ namespace detail
* leave j with the type of the new value, but the pointer to the destroyed old one.
*/
// a move iterator over the elements of an rvalue container, unless they are
// const (e.g., std::set): libstdc++ before GCC 5 defines the reference of a
// std::move_iterator as value_type&&, which cannot bind to a const element
template<typename Iterator>
std::move_iterator<Iterator> make_move_iterator_if_mutable_impl(const Iterator& it, std::true_type /*mutable*/)
{
return std::make_move_iterator(it);
}
template<typename Iterator>
Iterator make_move_iterator_if_mutable_impl(const Iterator& it, std::false_type /*mutable*/)
{
return it;
}
template<typename Iterator>
using is_mutable_iterator = std::integral_constant < bool,
!std::is_const<typename std::remove_reference<decltype(*std::declval<Iterator&>())>::type>::value >;
template<typename Iterator>
auto make_move_iterator_if_mutable(const Iterator& it)
-> decltype(make_move_iterator_if_mutable_impl(it, is_mutable_iterator<Iterator> {}))
{
return make_move_iterator_if_mutable_impl(it, is_mutable_iterator<Iterator> {});
}
template<value_t> struct external_constructor;
template<>
@@ -7356,7 +7330,7 @@ struct external_constructor<value_t::array>
using std::begin;
using std::end;
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::array_t>(make_move_iterator_if_mutable(begin(arr)), make_move_iterator_if_mutable(end(arr))));
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::array_t>(std::make_move_iterator(begin(arr)), std::make_move_iterator(end(arr))));
j.replace_value(value_t::array, value);
}
@@ -7417,7 +7391,7 @@ struct external_constructor<value_t::object>
using std::begin;
using std::end;
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::object_t>(make_move_iterator_if_mutable(begin(obj)), make_move_iterator_if_mutable(end(obj))));
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::object_t>(std::make_move_iterator(begin(obj)), std::make_move_iterator(end(obj))));
j.replace_value(value_t::object, value);
}
@@ -7595,27 +7569,25 @@ using is_movable_container_rvalue = negation<disjunction<
std::is_const<T>,
has_dedicated_to_json<T>,
is_basic_json<T>,
is_range_view_type<T>,
is_enum_keyed_map_type<T>,
std::is_same<T, std::vector<bool>>>>;
// rvalue array-like containers; object types are ruled out before views are
// checked, as std::ranges::view<T> for the object_t of an ordered_json would
// need that ordered_json to be complete (GCC 11, see #3312)
// rvalue array-like containers
template<typename BasicJsonType, typename T>
using is_rvalue_array_to_json = conjunction<is_movable_container_rvalue<T>,
negation<is_compatible_object_type<BasicJsonType, T>>,
negation<is_range_view_type<T>>,
is_compatible_array_type<BasicJsonType, T>,
negation<disjunction<
std::is_same<T, typename BasicJsonType::array_t>,
std::is_same<T, typename BasicJsonType::binary_t>,
is_compatible_object_type<BasicJsonType, T>,
is_compatible_string_type<BasicJsonType, T>,
is_compatible_binary_type<BasicJsonType, T>>>>;
// rvalue containers: their elements are moved instead of copied
template<typename BasicJsonType, typename CompatibleArrayType,
enable_if_t<is_rvalue_array_to_json<BasicJsonType, CompatibleArrayType>::value, int> = 0>
inline void to_json(BasicJsonType& j, CompatibleArrayType && arr) // NOLINT(cppcoreguidelines-missing-std-forward)
inline void to_json(BasicJsonType& j, CompatibleArrayType && arr)
{
external_constructor<value_t::array>::construct_moving(j, arr);
}
@@ -7693,8 +7665,7 @@ inline void to_json(BasicJsonType& j, const EnumKeyedMap& map)
}
#endif
// rvalue object-like containers (views have no key_type and mapped_type, so
// they are no compatible object type)
// rvalue object-like containers
template<typename BasicJsonType, typename T>
using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>,
negation<std::is_same<T, typename BasicJsonType::object_t>>,
@@ -7702,7 +7673,7 @@ using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>,
template<typename BasicJsonType, typename CompatibleObjectType,
enable_if_t<is_rvalue_object_to_json<BasicJsonType, CompatibleObjectType>::value, int> = 0>
inline void to_json(BasicJsonType& j, CompatibleObjectType && obj) // NOLINT(cppcoreguidelines-missing-std-forward)
inline void to_json(BasicJsonType& j, CompatibleObjectType && obj)
{
external_constructor<value_t::object>::construct_moving(j, obj);
}
@@ -20744,12 +20715,12 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
{
JSON_ASSERT(m_object != nullptr);
if (JSON_HEDLEY_UNLIKELY(!m_object->is_object()))
if (JSON_HEDLEY_LIKELY(m_object->is_object()))
{
throw_literal<invalid_iterator>(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators", m_object);
return m_it.object_iterator->first;
}
return m_it.object_iterator->first;
throw_literal<invalid_iterator>(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators", m_object);
}
/*!
@@ -20976,7 +20947,6 @@ NLOHMANN_JSON_NAMESPACE_END
#include <algorithm> // all_of
#include <cctype> // isdigit
#include <cerrno> // errno, ERANGE
#include <cstddef> // size_t
#include <cstdlib> // strtoull
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
@@ -21307,7 +21277,7 @@ class json_pointer
{
break; // not a plain number: let strtoull decide
}
acc = static_cast<size_type>((acc * 10) + static_cast<size_type>(c - '0'));
acc = static_cast<size_type>(acc * 10 + static_cast<size_type>(c - '0'));
if (JSON_HEDLEY_UNLIKELY(i == s.size() - 1))
{
idx = acc;
@@ -21502,10 +21472,10 @@ class json_pointer
static std::size_t hash_of(std::size_t parent, const string_t& token) noexcept
{
std::size_t h = parent * static_cast<std::size_t>(0x9E3779B97F4A7C15ULL & (std::numeric_limits<std::size_t>::max)());
std::size_t h = parent * static_cast<std::size_t>(0x9E3779B97F4A7C15ULL);
for (std::size_t i = 0; i < token.size(); ++i)
{
h = (h ^ static_cast<unsigned char>(token[i])) * static_cast<std::size_t>(0x100000001B3ULL & (std::numeric_limits<std::size_t>::max)());
h = (h ^ static_cast<unsigned char>(token[i])) * static_cast<std::size_t>(0x100000001B3ULL);
}
return h ^ (h >> 29u);
}
@@ -26860,7 +26830,7 @@ JSON_HEDLEY_ALWAYS_INLINE shortest_decimal to_shortest(std::uint64_t bits) noexc
const uint128_parts p = umul192_hi128(p10.high, p10.low, bin_sig << static_cast<unsigned>(shift));
std::uint64_t integral = p.high >> static_cast<unsigned>(extra_shift);
// (no casts in the shifts: MSVC 2015 warns C4554 on them despite the parentheses)
constexpr auto fractional_low_shift = static_cast<unsigned>(extra_shift);
constexpr unsigned fractional_low_shift = static_cast<unsigned>(extra_shift);
constexpr unsigned fractional_high_shift = 64u - fractional_low_shift;
const std::uint64_t fractional = (p.high << fractional_high_shift) | (p.low >> fractional_low_shift);
std::uint64_t digit = 0;
@@ -30110,8 +30080,6 @@ NLOHMANN_JSON_NAMESPACE_END
// #include <nlohmann/detail/meta/type_traits.hpp>
// #include <nlohmann/thirdparty/hedley/hedley.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -31196,20 +31164,26 @@ private:
// one level deep); stops at the first structured child it meets
static bool all_children_leaves(const basic_json& v) noexcept
{
// array_t only needs forward iterators (see custom_array_type)
if (v.m_data.m_type == value_t::array)
{
const auto& a = *v.m_data.m_value.array;
return std::all_of(a.begin(), a.end(), [](const basic_json & el)
for (auto it = a.rbegin(); it != a.rend(); ++it)
{
return has_no_children(el);
});
if (!has_no_children(*it))
{
return false;
}
}
return true;
}
const auto& o = *v.m_data.m_value.object;
return std::all_of(o.begin(), o.end(), [](const typename object_t::value_type & el)
for (const auto& el : *v.m_data.m_value.object)
{
return has_no_children(el.second);
});
if (!has_no_children(el.second))
{
return false;
}
}
return true;
}
// The walk in destroy_container() may take the children of a
@@ -31666,7 +31640,7 @@ public:
}
template < typename O = object_t, detail::enable_if_t < detail::has_stable_elements<O>::value, int > = 0 >
std::size_t object_capacity() const noexcept
static constexpr std::size_t object_capacity() noexcept
{
return 0;
}
@@ -31678,7 +31652,7 @@ public:
}
template < typename O = object_t, detail::enable_if_t < !detail::has_stable_elements<O>::value && !detail::is_nlohmann_ordered_map<O>::value, int > = 0 >
std::size_t object_capacity() const noexcept
static constexpr std::size_t object_capacity() noexcept
{
return detail::unknown_size();
}
@@ -33479,21 +33453,18 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_iterator_range_of_null(const basic_json* obj)
{
static_cast<void>(obj); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_of_null, detail::concat("cannot construct with iterators from ", obj->type_name()), obj));
}
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_incompatible_reference_type(const basic_json& obj)
{
static_cast<void>(obj); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(type_error::create(detail::exception_id::incompatible_reference_type, detail::concat("incompatible ReferenceType for get_ref, actual type is ", obj.type_name()), &obj));
}
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_array_index_exceeds_size_type(size_type idx)
{
static_cast<void>(idx); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(std::length_error(detail::concat("array index ", std::to_string(idx), " exceeds size_type")));
}
@@ -33501,8 +33472,6 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_array_index_out_of_range(size_type idx, const basic_json& j)
{
static_cast<void>(idx); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(j);
JSON_THROW(out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat("array index ", std::to_string(idx), " is out of range"), &j));
}
@@ -33511,8 +33480,6 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_key_not_found(const KeyType& key, const basic_json& j)
{
static_cast<void>(key); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(j);
JSON_THROW(out_of_range::create(detail::exception_id::key_not_found, detail::concat("key '", key, "' not found"), &j));
}
@@ -35336,19 +35303,19 @@ public:
iterator insert(const_iterator pos, const basic_json& val) // NOLINT(performance-unnecessary-value-param)
{
// insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array()))
if (JSON_HEDLEY_LIKELY(is_array()))
{
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// insert to array and return iterator
return insert_iterator(pos, val);
}
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// insert to array and return iterator
return insert_iterator(pos, val);
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts element into array
@@ -35356,21 +35323,21 @@ public:
iterator insert(const_iterator pos, basic_json&& val) // NOLINT(performance-unnecessary-value-param)
{
// insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array()))
if (JSON_HEDLEY_LIKELY(is_array()))
{
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// moving into a local first keeps this safe even if val aliases
// an element of this array
basic_json tmp(std::move(val));
return insert_iterator(pos, std::move(tmp));
}
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// moving into a local first keeps this safe even if val aliases
// an element of this array
basic_json tmp(std::move(val));
return insert_iterator(pos, std::move(tmp));
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts copies of element into array
@@ -35378,19 +35345,19 @@ public:
iterator insert(const_iterator pos, size_type cnt, const basic_json& val) // NOLINT(performance-unnecessary-value-param)
{
// insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array()))
if (JSON_HEDLEY_LIKELY(is_array()))
{
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// insert to array and return iterator
return insert_iterator(pos, cnt, val);
}
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
}
// insert to array and return iterator
return insert_iterator(pos, cnt, val);
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts range of elements into array
@@ -35896,7 +35863,7 @@ public:
// inline fast path for two values of the same non-owning scalar type; anything
// else falls through to the out-of-line general comparison
#define JSON_IMPLEMENT_SCALAR_FAST(op, null_result) \
if (lhs.m_data.m_type == rhs.m_data.m_type) /* NOLINT(readability/braces) */ \
if (lhs.m_data.m_type == rhs.m_data.m_type) \
{ \
switch (lhs.m_data.m_type) \
{ \
+1 -1
View File
@@ -118,7 +118,7 @@ endif()
if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# avoid stack overflow, see https://github.com/nlohmann/json/issues/2955
json_test_set_test_options("test-bon8;test-cbor;test-msgpack;test-ubjson;test-bjdata;test-bjdata_round_trip_draft2;test-bjdata_round_trip_draft3;test-binary_formats" LINK_OPTIONS /STACK:4000000)
json_test_set_test_options("test-bon8;test-cbor;test-msgpack;test-ubjson;test-bjdata;test-binary_formats" LINK_OPTIONS /STACK:4000000)
endif()
# disable exceptions for test-disabled_exceptions
-103
View File
@@ -1,103 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <cstdint> // uint8_t
#include <vector> // vector
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
#include "round_trip_corpus.hpp"
// The BJData round-trip invariants, split by the BJData version that turns a
// corpus value into j1 (unit-bjdata_round_trip_draft2.cpp and
// unit-bjdata_round_trip_draft3.cpp). They were part of unit-bjdata.cpp, but
// took 80 % of its time; as separate tests, both halves run in parallel and
// stay below the ctest timeout under Valgrind.
namespace utils
{
// This checks what the parse_bjdata_fuzzer driver checks (see
// tests/src/fuzzer-parse_bjdata.cpp), so that a regression shows up in CI
// rather than as an OSS-Fuzz report: every value from_bjdata() returns
// (j1) can be serialized with any combination of options, the result can
// be parsed back (j2), and serializing j2 again with the same options
// yields a value-equal result.
//
// Beyond the driver, this also checks that j2 equals j1 and that
// serializing j2 reproduces the exact bytes, both except for values that
// contain a binary value: a binary value is only written as a binary
// value with Draft 3's optimized binary array, and otherwise read back as
// an array of integers, for which the writer may choose different (but
// equally valid) type markers when it is serialized again (see #5494).
//
// Values are compared with dump() rather than operator==, because a NaN
// never compares equal to itself.
//
// Only the corpus values written with @a initial_version become j1; each j1
// is then serialized with every combination of options, both versions
// included.
inline void check_bjdata_round_trip(const nlohmann::json::bjdata_version_t initial_version)
{
using nlohmann::json;
struct options
{
bool use_size;
bool use_type;
json::bjdata_version_t version;
};
const std::vector<options> all_options =
{
{false, false, json::bjdata_version_t::draft2},
{true, false, json::bjdata_version_t::draft2},
{true, true, json::bjdata_version_t::draft2},
{false, false, json::bjdata_version_t::draft3},
{true, false, json::bjdata_version_t::draft3},
{true, true, json::bjdata_version_t::draft3},
};
for (const auto& j0 : utils::round_trip_corpus::values())
{
// turn the corpus value into a value as from_bjdata() returns it
for (const auto& initial : all_options)
{
if (initial.version != initial_version)
{
continue;
}
const json j1 = json::from_bjdata(json::to_bjdata(j0, initial.use_size, initial.use_type, initial.version));
const bool has_binary = utils::round_trip_corpus::contains_binary(j1);
for (const auto& o : all_options)
{
INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type
<< ", draft3 = " << (o.version == json::bjdata_version_t::draft3));
const std::vector<std::uint8_t> vec = json::to_bjdata(j1, o.use_size, o.use_type, o.version);
json j2;
// anything the library writes must be parsable by the library
REQUIRE_NOTHROW(j2 = json::from_bjdata(vec));
const std::vector<std::uint8_t> vec2 = json::to_bjdata(j2, o.use_size, o.use_type, o.version);
CHECK(json::from_bjdata(vec2).dump() == j2.dump());
if (!has_binary)
{
CHECK(j2.dump() == j1.dump());
CHECK(vec2 == vec);
}
}
}
}
}
} // namespace utils
+2 -14
View File
@@ -64,18 +64,6 @@ std::vector<json> test_values()
};
}
// explicit char -> uint8_t conversion (the iterator-range constructor converts implicitly, which -fsanitize=integer flags)
std::vector<std::uint8_t> to_bytes(const std::string& str)
{
std::vector<std::uint8_t> result;
result.reserve(str.size());
for (const char c : str)
{
result.push_back(static_cast<std::uint8_t>(c));
}
return result;
}
// BON8 has no integers above the int64 range, so to_bon8() rejects them
bool bon8_representable(const json& j)
{
@@ -419,11 +407,11 @@ TEST_CASE("CBOR and MessagePack integer arrays")
std::ostringstream stream;
json::to_cbor(array, stream);
std::string streamed = stream.str();
CHECK(to_bytes(streamed) == expected_cbor);
CHECK(std::vector<std::uint8_t>(streamed.begin(), streamed.end()) == expected_cbor);
stream.str("");
json::to_msgpack(array, stream);
streamed = stream.str();
CHECK(to_bytes(streamed) == expected_msgpack);
CHECK(std::vector<std::uint8_t>(streamed.begin(), streamed.end()) == expected_msgpack);
CHECK(json::from_cbor(expected_cbor) == array);
CHECK(json::from_msgpack(expected_msgpack) == array);
+65
View File
@@ -28,6 +28,7 @@ using ordered_json = nlohmann::ordered_json;
#include <fstream>
#include <set>
#include "make_test_data_available.hpp"
#include "round_trip_corpus.hpp"
#include "test_utils.hpp"
#include "sax_countdown.hpp"
using utils::SaxCountdown;
@@ -4440,6 +4441,70 @@ TEST_CASE("BJData use_type requires use_size")
}
}
TEST_CASE("BJData round-trip invariants")
{
// This checks what the parse_bjdata_fuzzer driver checks (see
// tests/src/fuzzer-parse_bjdata.cpp), so that a regression shows up in CI
// rather than as an OSS-Fuzz report: every value from_bjdata() returns
// (j1) can be serialized with any combination of options, the result can
// be parsed back (j2), and serializing j2 again with the same options
// yields a value-equal result.
//
// Beyond the driver, this also checks that j2 equals j1 and that
// serializing j2 reproduces the exact bytes, both except for values that
// contain a binary value: a binary value is only written as a binary
// value with Draft 3's optimized binary array, and otherwise read back as
// an array of integers, for which the writer may choose different (but
// equally valid) type markers when it is serialized again (see #5494).
//
// Values are compared with dump() rather than operator==, because a NaN
// never compares equal to itself.
struct options
{
bool use_size;
bool use_type;
json::bjdata_version_t version;
};
const std::vector<options> all_options =
{
{false, false, json::bjdata_version_t::draft2},
{true, false, json::bjdata_version_t::draft2},
{true, true, json::bjdata_version_t::draft2},
{false, false, json::bjdata_version_t::draft3},
{true, false, json::bjdata_version_t::draft3},
{true, true, json::bjdata_version_t::draft3},
};
for (const auto& j0 : utils::round_trip_corpus::values())
{
// turn the corpus value into a value as from_bjdata() returns it
for (const auto& initial : all_options)
{
const json j1 = json::from_bjdata(json::to_bjdata(j0, initial.use_size, initial.use_type, initial.version));
const bool has_binary = utils::round_trip_corpus::contains_binary(j1);
for (const auto& o : all_options)
{
INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type
<< ", draft3 = " << (o.version == json::bjdata_version_t::draft3));
const std::vector<std::uint8_t> vec = json::to_bjdata(j1, o.use_size, o.use_type, o.version);
json j2;
// anything the library writes must be parsable by the library
REQUIRE_NOTHROW(j2 = json::from_bjdata(vec));
const std::vector<std::uint8_t> vec2 = json::to_bjdata(j2, o.use_size, o.use_type, o.version);
CHECK(json::from_bjdata(vec2).dump() == j2.dump());
if (!has_binary)
{
CHECK(j2.dump() == j1.dump());
CHECK(vec2 == vec);
}
}
}
}
}
TEST_CASE("BJData round trip of a binary value is value-stable, not byte-stable")
{
// OSS-Fuzz issue 474480402: a Draft 3 optimized binary array is read as a
@@ -1,22 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using nlohmann::json;
#include "bjdata_round_trip.hpp"
// split from unit-bjdata.cpp, see bjdata_round_trip.hpp; the other half is in
// unit-bjdata_round_trip_draft3.cpp
TEST_CASE("BJData round-trip invariants (j1 written with Draft 2)")
{
utils::check_bjdata_round_trip(json::bjdata_version_t::draft2);
}
@@ -1,22 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using nlohmann::json;
#include "bjdata_round_trip.hpp"
// split from unit-bjdata.cpp, see bjdata_round_trip.hpp; the other half is in
// unit-bjdata_round_trip_draft2.cpp
TEST_CASE("BJData round-trip invariants (j1 written with Draft 3)")
{
utils::check_bjdata_round_trip(json::bjdata_version_t::draft3);
}
+7 -7
View File
@@ -198,11 +198,11 @@ TEST_CASE("copy constructor and destructor across value types")
const json src_uint = 7u;
const json src_float = 2.5;
json c_null(src_null); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor
json c_bool(src_bool); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor
json c_int(src_int); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor
json c_uint(src_uint); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor
json c_float(src_float); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor
json c_null(src_null);
json c_bool(src_bool);
json c_int(src_int);
json c_uint(src_uint);
json c_float(src_float);
CHECK(c_null.is_null());
CHECK(c_bool == true);
@@ -300,12 +300,12 @@ TEST_CASE("copy constructor and destructor across value types")
REQUIRE(v.size() == 70);
std::vector<json> copy(v);
CHECK((copy == v));
CHECK(copy == v);
// mutating the copy leaves the original alone
copy[5] = "changed";
copy[6].push_back(1);
CHECK((copy != v));
CHECK(copy != v);
CHECK(v[5] == std::string(5, 'x'));
CHECK(v[6].size() == 3);
+2 -2
View File
@@ -817,8 +817,8 @@ template<class Key, class T, class IgnoredLess = std::less<Key>,
struct relocating_map : nlohmann::ordered_map<Key, T, IgnoredLess, Allocator>
{
using base_type = nlohmann::ordered_map<Key, T, IgnoredLess, Allocator>;
using iterator = typename base_type::iterator;
using Container = typename base_type::Container;
using typename base_type::iterator;
using typename base_type::Container;
using base_type::base_type;
template<typename... Args>
+4 -5
View File
@@ -1637,7 +1637,6 @@ TEST_CASE_TEMPLATE("reverse iteration and member access", Json, nlohmann::json,
{
Json j = {1, 2, 3, 4};
const std::vector<int> reversed = {4, 3, 2, 1};
auto it = j.end();
std::vector<int> seen;
while (it != j.begin())
@@ -1645,7 +1644,7 @@ TEST_CASE_TEMPLATE("reverse iteration and member access", Json, nlohmann::json,
--it;
seen.push_back(it->template get<int>());
}
CHECK(seen == reversed);
CHECK(seen == std::vector<int> {4, 3, 2, 1});
auto it2 = j.end();
auto old = it2--;
@@ -1657,14 +1656,14 @@ TEST_CASE_TEMPLATE("reverse iteration and member access", Json, nlohmann::json,
{
seen.push_back(r->template get<int>());
}
CHECK(seen == reversed);
CHECK(seen == std::vector<int> {4, 3, 2, 1});
seen.clear();
for (auto r = j.crbegin(); r != j.crend(); ++r)
{
seen.push_back(r->template get<int>());
seen.push_back(*r);
}
CHECK(seen == reversed);
CHECK(seen == std::vector<int> {4, 3, 2, 1});
}
SECTION("object: -- and rbegin/rend")
+2 -2
View File
@@ -1230,11 +1230,11 @@ TEST_CASE("JSON pointer array index tokens match strtoull semantics")
SECTION("random tokens")
{
std::mt19937 gen(42); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
std::mt19937 gen(42);
const std::string alphabet = "0123456789012345678901234567890123456789-+ ex";
for (int i = 0; i < 5000; ++i)
{
std::string token(gen() % 23, '0');
std::string token(static_cast<std::size_t>(gen() % 23), '0');
for (auto& c : token)
{
c = alphabet[gen() % alphabet.size()];
+401
View File
@@ -13,6 +13,8 @@ using nlohmann::json;
#include <algorithm>
#include <string>
#include <utility>
#include <vector>
TEST_CASE("tests on very large JSONs")
{
@@ -53,6 +55,24 @@ const json* innermost_value(const json& j, std::size_t& depth)
return current;
}
// The text of a value nested depth levels deep around the number 0. Level i is
// an array if pattern[i % pattern.size()] is '[', and otherwise an object with
// the single member "a", which every object type enumerates in the same order.
std::string nested_text(std::size_t depth, const std::string& pattern)
{
std::string text;
std::string closing;
for (std::size_t i = 0; i < depth; ++i)
{
const bool array = pattern[i % pattern.size()] == '[';
text += array ? "[" : "{\"a\":";
closing += array ? ']' : '}';
}
text += '0';
text.append(closing.rbegin(), closing.rend());
return text;
}
} // namespace
TEST_CASE("tests on deeply nested JSONs")
@@ -224,4 +244,385 @@ TEST_CASE("tests on deeply nested JSONs")
CHECK(*innermost_value(j, unused) == 0);
}
}
SECTION("issue #5650 - stack overflow converting between specializations")
{
const std::vector<std::string> patterns = {"[", "{", "[{"};
SECTION("json to ordered_json")
{
for (const auto& pattern : patterns)
{
CAPTURE(pattern)
const std::string text = nested_text(depth, pattern);
const json j = json::parse(text);
const nlohmann::ordered_json converted = j;
CHECK(converted.dump() == text);
}
}
SECTION("ordered_json to json")
{
for (const auto& pattern : patterns)
{
CAPTURE(pattern)
const std::string text = nested_text(depth, pattern);
const nlohmann::ordered_json o = nlohmann::ordered_json::parse(text);
const json converted = o;
CHECK(converted.dump() == text);
}
}
SECTION("get<ordered_json>()")
{
for (const auto& pattern : patterns)
{
CAPTURE(pattern)
const std::string text = nested_text(depth, pattern);
const json j = json::parse(text);
CHECK(j.get<nlohmann::ordered_json>().dump() == text);
}
}
SECTION("depths around the bound of the recursive descent")
{
for (std::size_t d = 1; d <= 300; ++d)
{
CAPTURE(d)
for (const auto& pattern : patterns)
{
CAPTURE(pattern)
const std::string text = nested_text(d, pattern);
const json j = json::parse(text);
const nlohmann::ordered_json converted = j;
CHECK(converted.dump() == text);
const json back = converted;
CHECK(back.dump() == text);
}
}
}
SECTION("values below the bound are converted as values above it")
{
// Bury a value below the bound, where it is converted without the
// call stack, and compare it with the same value converted on its
// own by the containers' range constructors. Its objects have
// members that the two object types enumerate in different orders.
const auto bury = [](nlohmann::ordered_json value)
{
for (std::size_t i = 0; i < 200; ++i)
{
value = nlohmann::ordered_json::array({std::move(value)});
}
return value;
};
const auto dig = [](const json & value)
{
const json* current = &value;
for (std::size_t i = 0; i < 200; ++i)
{
current = &current->at(0);
}
return current;
};
nlohmann::ordered_json value = nlohmann::ordered_json::object();
value["z"] = {1, -2, 3U, 4.5, true, nullptr, "six", nlohmann::ordered_json::binary({7, 8}, 9),
nlohmann::ordered_json::binary({10}), nlohmann::ordered_json::array(), nlohmann::ordered_json::object()
};
value["y"] = {{"x", {{"w", 1}, {"v", 2}}}, {"u", {3, {{"t", 4}, {"s", 5}}}}};
value["r"] = nlohmann::ordered_json::array({nlohmann::ordered_json(nlohmann::ordered_json::value_t::discarded)});
const json converted_above = value;
const json buried = bury(value);
const json& converted_below = *dig(buried);
CHECK(converted_below.dump() == converted_above.dump());
CHECK(converted_below.at("z").at(7).get_binary().subtype() == 9);
CHECK_FALSE(converted_below.at("z").at(8).get_binary().has_subtype());
CHECK(converted_below.at("r").at(0).is_discarded());
// a discarded value is never equal to anything, so compare the rest
value.erase("r");
const json without_discarded_above = value;
const json without_discarded_buried = bury(value);
CHECK(*dig(without_discarded_buried) == without_discarded_above);
}
}
}
namespace
{
json nested_array(const std::size_t depth, json leaf)
{
json j = std::move(leaf);
for (std::size_t i = 0; i < depth; ++i)
{
json a = json::array();
a.push_back(std::move(j));
j = std::move(a);
}
return j;
}
json nested_object(const std::size_t depth, json leaf)
{
json j = std::move(leaf);
for (std::size_t i = 0; i < depth; ++i)
{
json o = json::object();
o["k"] = std::move(j);
j = std::move(o);
}
return j;
}
} // namespace
TEST_CASE("issue #5392 - binary writers on deeply nested values")
{
// 200 is past the point where the writers stop recursing, and still
// shallow enough that from_* and operator== (which still recurse) are fine.
const json deep_array = nested_array(200, json(0));
const json deep_object = nested_object(200, json("x"));
const json empty_array = nested_array(200, json::array());
const json empty_object = nested_object(200, json::object());
const json mixed = nested_object(80, nested_array(80, json(true)));
SECTION("roundtrip past the recursion bound")
{
CHECK(json::from_cbor(json::to_cbor(deep_array)) == deep_array);
CHECK(json::from_msgpack(json::to_msgpack(deep_array)) == deep_array);
CHECK(json::from_ubjson(json::to_ubjson(deep_array)) == deep_array);
CHECK(json::from_ubjson(json::to_ubjson(deep_array, true, false)) == deep_array);
CHECK(json::from_ubjson(json::to_ubjson(deep_array, true, true)) == deep_array);
CHECK(json::from_bjdata(json::to_bjdata(deep_array)) == deep_array);
CHECK(json::from_bon8(json::to_bon8(deep_array)) == deep_array);
CHECK(json::from_cbor(json::to_cbor(deep_object)) == deep_object);
CHECK(json::from_msgpack(json::to_msgpack(deep_object)) == deep_object);
CHECK(json::from_ubjson(json::to_ubjson(deep_object)) == deep_object);
CHECK(json::from_ubjson(json::to_ubjson(deep_object, true, true)) == deep_object);
CHECK(json::from_bjdata(json::to_bjdata(deep_object)) == deep_object);
CHECK(json::from_bon8(json::to_bon8(deep_object)) == deep_object);
CHECK(json::from_cbor(json::to_cbor(empty_array)) == empty_array);
CHECK(json::from_msgpack(json::to_msgpack(empty_array)) == empty_array);
CHECK(json::from_ubjson(json::to_ubjson(empty_array)) == empty_array);
CHECK(json::from_ubjson(json::to_ubjson(empty_array, true, true)) == empty_array);
CHECK(json::from_bon8(json::to_bon8(empty_array)) == empty_array);
CHECK(json::from_cbor(json::to_cbor(empty_object)) == empty_object);
CHECK(json::from_msgpack(json::to_msgpack(empty_object)) == empty_object);
CHECK(json::from_ubjson(json::to_ubjson(empty_object)) == empty_object);
CHECK(json::from_bon8(json::to_bon8(empty_object)) == empty_object);
CHECK(json::from_cbor(json::to_cbor(mixed)) == mixed);
CHECK(json::from_msgpack(json::to_msgpack(mixed)) == mixed);
CHECK(json::from_ubjson(json::to_ubjson(mixed)) == mixed);
CHECK(json::from_bjdata(json::to_bjdata(mixed)) == mixed);
CHECK(json::from_bon8(json::to_bon8(mixed)) == mixed);
}
SECTION("the two ways of writing a value meet at the bound")
{
for (std::size_t depth = 120; depth <= 140; ++depth)
{
CAPTURE(depth)
const json array = nested_array(depth, json(7));
CHECK(json::from_cbor(json::to_cbor(array)) == array);
CHECK(json::from_msgpack(json::to_msgpack(array)) == array);
CHECK(json::from_ubjson(json::to_ubjson(array, true, true)) == array);
CHECK(json::from_bon8(json::to_bon8(array)) == array);
const json object = nested_object(depth, json(7));
CHECK(json::from_cbor(json::to_cbor(object)) == object);
CHECK(json::from_msgpack(json::to_msgpack(object)) == object);
CHECK(json::from_bjdata(json::to_bjdata(object)) == object);
CHECK(json::from_bon8(json::to_bon8(object)) == object);
}
}
SECTION("a BJData ndarray below the bound is still an ndarray")
{
const json ndarray = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
const json invalid = json({{"_ArrayType_", "nope"}, {"_ArraySize_", {1}}, {"_ArrayData_", {1}}});
const json deep_ndarray = nested_array(140, ndarray);
const json deep_invalid = nested_array(140, invalid);
CHECK(json::from_bjdata(json::to_bjdata(deep_ndarray)) == deep_ndarray);
CHECK(json::from_bjdata(json::to_bjdata(deep_invalid)) == deep_invalid);
CHECK(json::from_bjdata(json::to_bjdata(ndarray)) == ndarray);
}
SECTION("byte-exact across the switch-over")
{
// nested one-element arrays around the recursion bound: the exact
// bytes a writer produces do not depend on whether it stayed on the
// call stack or moved to the heap one partway through
for (const std::size_t depth :
{
nlohmann::detail::recursion_depth_limit() - 1, nlohmann::detail::recursion_depth_limit(),
nlohmann::detail::recursion_depth_limit() + 1, nlohmann::detail::recursion_depth_limit() + 2
})
{
CAPTURE(depth)
const json array = nested_array(depth, json(0));
std::vector<std::uint8_t> expected_cbor(depth, 0x81);
expected_cbor.push_back(0x00);
CHECK(json::to_cbor(array) == expected_cbor);
std::vector<std::uint8_t> expected_msgpack(depth, 0x91);
expected_msgpack.push_back(0x00);
CHECK(json::to_msgpack(array) == expected_msgpack);
std::string expected_ubjson(depth, '[');
expected_ubjson += "i";
expected_ubjson += '\0';
expected_ubjson.append(depth, ']');
const auto packed_ubjson = json::to_ubjson(array);
CHECK(std::string(packed_ubjson.begin(), packed_ubjson.end()) == expected_ubjson);
std::vector<std::uint8_t> expected_bon8(depth, 0x81);
expected_bon8.push_back(0x90);
CHECK(json::to_bon8(array) == expected_bon8);
}
}
SECTION("a deep object, and a BJData ndarray, past the recursion bound")
{
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
const json object = nested_object(depth, json(42));
CHECK(json::from_cbor(json::to_cbor(object)) == object);
CHECK(json::from_msgpack(json::to_msgpack(object)) == object);
CHECK(json::from_ubjson(json::to_ubjson(object, true, true)) == object);
CHECK(json::from_bjdata(json::to_bjdata(object)) == object);
CHECK(json::from_bon8(json::to_bon8(object)) == object);
const json ndarray = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
const json deep_ndarray = nested_array(depth, ndarray);
CHECK(json::from_bjdata(json::to_bjdata(deep_ndarray)) == deep_ndarray);
}
SECTION("a discarded value past the recursion bound still throws type_error.321")
{
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
const json discarded_leaf(json::value_t::discarded);
const json deep_discarded = nested_array(depth, discarded_leaf);
// with diagnostics, the message names the path to the discarded leaf
#if JSON_DIAGNOSTICS
std::string path;
for (std::size_t i = 0; i < depth; ++i)
{
path += "/0";
}
const std::string prefix = "[json.exception.type_error.321] (" + path + ") ";
#else
const std::string prefix = "[json.exception.type_error.321] ";
#endif
CHECK_THROWS_WITH_AS(json::to_cbor(deep_discarded), (prefix + "cannot serialize discarded value to CBOR").c_str(), json::type_error);
CHECK_THROWS_WITH_AS(json::to_msgpack(deep_discarded), (prefix + "cannot serialize discarded value to MessagePack").c_str(), json::type_error);
CHECK_THROWS_WITH_AS(json::to_ubjson(deep_discarded), (prefix + "cannot serialize discarded value to UBJSON").c_str(), json::type_error);
CHECK_THROWS_WITH_AS(json::to_bjdata(deep_discarded), (prefix + "cannot serialize discarded value to BJData").c_str(), json::type_error);
}
SECTION("BON8 past the recursion bound: discarded values and errors")
{
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
// BON8 rejects a discarded value (type_error.321), deep or not
const json deep_discarded = nested_array(depth, json(json::value_t::discarded));
CHECK_THROWS_AS(json::to_bon8(deep_discarded), json::type_error);
// errors are thrown from below the bound as from above it
const json deep_invalid_string = nested_array(depth, json(std::string("\x80")));
CHECK_THROWS_AS(json::to_bon8(deep_invalid_string), json::type_error);
json deep_invalid_key = json::object();
deep_invalid_key[std::string("\x80")] = 1;
deep_invalid_key = nested_object(depth, deep_invalid_key);
CHECK_THROWS_AS(json::to_bon8(deep_invalid_key), json::type_error);
const json deep_too_large = nested_array(depth, json(9223372036854775808u));
CHECK_THROWS_AS(json::to_bon8(deep_too_large), json::out_of_range);
}
SECTION("does not overflow the C++ stack")
{
const std::size_t depth = 100000;
const json j = json::parse(std::string(depth, '[') + "0" + std::string(depth, ']'));
std::vector<std::uint8_t> packed;
CHECK_NOTHROW(packed = json::to_cbor(j));
CHECK(json::from_cbor(packed) == j);
CHECK_NOTHROW(packed = json::to_msgpack(j));
CHECK(json::from_msgpack(packed) == j);
CHECK_NOTHROW(packed = json::to_ubjson(j));
CHECK(json::from_ubjson(packed) == j);
CHECK_NOTHROW(packed = json::to_ubjson(j, true, false));
CHECK(json::from_ubjson(packed) == j);
CHECK_NOTHROW(packed = json::to_bjdata(j));
CHECK(json::from_bjdata(packed) == j);
CHECK_NOTHROW(packed = json::to_bon8(j));
CHECK(json::from_bon8(packed) == j);
}
SECTION("BON8 containers with more than four elements past the recursion bound")
{
// such containers are closed with 0xFE, which the iterative writer
// must emit when it leaves them; strings next to each other must
// still be separated with 0xFF
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
json wide_array = json::array({1, "a", "b", json::array(), 5});
json wide_object = json({{"k1", "v"}, {"k2", 2}, {"k3", ""}, {"k4", json::object()}, {"k5", nullptr}});
for (std::size_t i = 0; i < depth; ++i)
{
wide_array = json::array({"s", wide_array, "t", wide_object, true});
wide_object = json({{"a", wide_object}, {"b", "x"}, {"c", "y"}, {"d", wide_array.size()}, {"e", "z"}});
}
CHECK(json::from_bon8(json::to_bon8(wide_array)) == wide_array);
CHECK(json::from_bon8(json::to_bon8(wide_object)) == wide_object);
}
SECTION("regression test for https://issues.oss-fuzz.com/issues/566583014")
{
// 200000 nested one-element CBOR arrays, the innermost holding null;
// round-tripping this used to recurse once per level on the way back
// out through to_cbor(), deep enough to overflow the stack
std::vector<std::uint8_t> v(200000, 0x81);
v.push_back(0xf6);
const json j = json::from_cbor(v);
CHECK(json::to_cbor(j) == v);
// the MessagePack analogue: fixarray of 1 nesting down to nil
std::vector<std::uint8_t> v_msgpack(200000, 0x91);
v_msgpack.push_back(0xc0);
const json j_msgpack = json::from_msgpack(v_msgpack);
CHECK(json::to_msgpack(j_msgpack) == v_msgpack);
}
SECTION("regression test for https://issues.oss-fuzz.com/issues/572238015")
{
// the BON8 analogue: 200000 nested one-element arrays down to null
std::vector<std::uint8_t> v(200000, 0x81);
v.push_back(0xFA);
const json j = json::from_bon8(v);
CHECK(json::to_bon8(j) == v);
}
}
@@ -1,294 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <cstddef>
#include <cstdint>
#include <string>
#include <utility>
#include <vector>
// split from unit-large_json.cpp so that both run in parallel and stay below
// the ctest timeout under Valgrind
namespace
{
json nested_array(const std::size_t depth, json leaf)
{
json j = std::move(leaf);
for (std::size_t i = 0; i < depth; ++i)
{
json a = json::array();
a.push_back(std::move(j));
j = std::move(a);
}
return j;
}
json nested_object(const std::size_t depth, json leaf)
{
json j = std::move(leaf);
for (std::size_t i = 0; i < depth; ++i)
{
json o = json::object();
o["k"] = std::move(j);
j = std::move(o);
}
return j;
}
} // namespace
TEST_CASE("issue #5392 - binary writers on deeply nested values")
{
// 200 is past the point where the writers stop recursing, and still
// shallow enough that from_* and operator== (which still recurse) are fine.
const json deep_array = nested_array(200, json(0));
const json deep_object = nested_object(200, json("x"));
const json empty_array = nested_array(200, json::array());
const json empty_object = nested_object(200, json::object());
const json mixed = nested_object(80, nested_array(80, json(true)));
SECTION("roundtrip past the recursion bound")
{
CHECK(json::from_cbor(json::to_cbor(deep_array)) == deep_array);
CHECK(json::from_msgpack(json::to_msgpack(deep_array)) == deep_array);
CHECK(json::from_ubjson(json::to_ubjson(deep_array)) == deep_array);
CHECK(json::from_ubjson(json::to_ubjson(deep_array, true, false)) == deep_array);
CHECK(json::from_ubjson(json::to_ubjson(deep_array, true, true)) == deep_array);
CHECK(json::from_bjdata(json::to_bjdata(deep_array)) == deep_array);
CHECK(json::from_bon8(json::to_bon8(deep_array)) == deep_array);
CHECK(json::from_cbor(json::to_cbor(deep_object)) == deep_object);
CHECK(json::from_msgpack(json::to_msgpack(deep_object)) == deep_object);
CHECK(json::from_ubjson(json::to_ubjson(deep_object)) == deep_object);
CHECK(json::from_ubjson(json::to_ubjson(deep_object, true, true)) == deep_object);
CHECK(json::from_bjdata(json::to_bjdata(deep_object)) == deep_object);
CHECK(json::from_bon8(json::to_bon8(deep_object)) == deep_object);
CHECK(json::from_cbor(json::to_cbor(empty_array)) == empty_array);
CHECK(json::from_msgpack(json::to_msgpack(empty_array)) == empty_array);
CHECK(json::from_ubjson(json::to_ubjson(empty_array)) == empty_array);
CHECK(json::from_ubjson(json::to_ubjson(empty_array, true, true)) == empty_array);
CHECK(json::from_bon8(json::to_bon8(empty_array)) == empty_array);
CHECK(json::from_cbor(json::to_cbor(empty_object)) == empty_object);
CHECK(json::from_msgpack(json::to_msgpack(empty_object)) == empty_object);
CHECK(json::from_ubjson(json::to_ubjson(empty_object)) == empty_object);
CHECK(json::from_bon8(json::to_bon8(empty_object)) == empty_object);
CHECK(json::from_cbor(json::to_cbor(mixed)) == mixed);
CHECK(json::from_msgpack(json::to_msgpack(mixed)) == mixed);
CHECK(json::from_ubjson(json::to_ubjson(mixed)) == mixed);
CHECK(json::from_bjdata(json::to_bjdata(mixed)) == mixed);
CHECK(json::from_bon8(json::to_bon8(mixed)) == mixed);
}
SECTION("the two ways of writing a value meet at the bound")
{
for (std::size_t depth = 120; depth <= 140; ++depth)
{
CAPTURE(depth)
const json array = nested_array(depth, json(7));
CHECK(json::from_cbor(json::to_cbor(array)) == array);
CHECK(json::from_msgpack(json::to_msgpack(array)) == array);
CHECK(json::from_ubjson(json::to_ubjson(array, true, true)) == array);
CHECK(json::from_bon8(json::to_bon8(array)) == array);
const json object = nested_object(depth, json(7));
CHECK(json::from_cbor(json::to_cbor(object)) == object);
CHECK(json::from_msgpack(json::to_msgpack(object)) == object);
CHECK(json::from_bjdata(json::to_bjdata(object)) == object);
CHECK(json::from_bon8(json::to_bon8(object)) == object);
}
}
SECTION("a BJData ndarray below the bound is still an ndarray")
{
const json ndarray = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
const json invalid = json({{"_ArrayType_", "nope"}, {"_ArraySize_", {1}}, {"_ArrayData_", {1}}});
const json deep_ndarray = nested_array(140, ndarray);
const json deep_invalid = nested_array(140, invalid);
CHECK(json::from_bjdata(json::to_bjdata(deep_ndarray)) == deep_ndarray);
CHECK(json::from_bjdata(json::to_bjdata(deep_invalid)) == deep_invalid);
CHECK(json::from_bjdata(json::to_bjdata(ndarray)) == ndarray);
}
SECTION("byte-exact across the switch-over")
{
// nested one-element arrays around the recursion bound: the exact
// bytes a writer produces do not depend on whether it stayed on the
// call stack or moved to the heap one partway through
for (const std::size_t depth :
{
nlohmann::detail::recursion_depth_limit() - 1, nlohmann::detail::recursion_depth_limit(),
nlohmann::detail::recursion_depth_limit() + 1, nlohmann::detail::recursion_depth_limit() + 2
})
{
CAPTURE(depth)
const json array = nested_array(depth, json(0));
std::vector<std::uint8_t> expected_cbor(depth, 0x81);
expected_cbor.push_back(0x00);
CHECK(json::to_cbor(array) == expected_cbor);
std::vector<std::uint8_t> expected_msgpack(depth, 0x91);
expected_msgpack.push_back(0x00);
CHECK(json::to_msgpack(array) == expected_msgpack);
std::string expected_ubjson(depth, '[');
expected_ubjson += "i";
expected_ubjson += '\0';
expected_ubjson.append(depth, ']');
const auto packed_ubjson = json::to_ubjson(array);
CHECK(std::string(packed_ubjson.begin(), packed_ubjson.end()) == expected_ubjson);
std::vector<std::uint8_t> expected_bon8(depth, 0x81);
expected_bon8.push_back(0x90);
CHECK(json::to_bon8(array) == expected_bon8);
}
}
SECTION("a deep object, and a BJData ndarray, past the recursion bound")
{
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
const json object = nested_object(depth, json(42));
CHECK(json::from_cbor(json::to_cbor(object)) == object);
CHECK(json::from_msgpack(json::to_msgpack(object)) == object);
CHECK(json::from_ubjson(json::to_ubjson(object, true, true)) == object);
CHECK(json::from_bjdata(json::to_bjdata(object)) == object);
CHECK(json::from_bon8(json::to_bon8(object)) == object);
const json ndarray = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
const json deep_ndarray = nested_array(depth, ndarray);
CHECK(json::from_bjdata(json::to_bjdata(deep_ndarray)) == deep_ndarray);
}
SECTION("a discarded value past the recursion bound still throws type_error.321")
{
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
const json discarded_leaf(json::value_t::discarded);
const json deep_discarded = nested_array(depth, discarded_leaf);
// with diagnostics, the message names the path to the discarded leaf
#if JSON_DIAGNOSTICS
std::string path;
for (std::size_t i = 0; i < depth; ++i)
{
path += "/0";
}
const std::string prefix = "[json.exception.type_error.321] (" + path + ") ";
#else
const std::string prefix = "[json.exception.type_error.321] ";
#endif
CHECK_THROWS_WITH_AS(json::to_cbor(deep_discarded), (prefix + "cannot serialize discarded value to CBOR").c_str(), json::type_error);
CHECK_THROWS_WITH_AS(json::to_msgpack(deep_discarded), (prefix + "cannot serialize discarded value to MessagePack").c_str(), json::type_error);
CHECK_THROWS_WITH_AS(json::to_ubjson(deep_discarded), (prefix + "cannot serialize discarded value to UBJSON").c_str(), json::type_error);
CHECK_THROWS_WITH_AS(json::to_bjdata(deep_discarded), (prefix + "cannot serialize discarded value to BJData").c_str(), json::type_error);
}
SECTION("BON8 past the recursion bound: discarded values and errors")
{
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
// BON8 rejects a discarded value (type_error.321), deep or not
const json deep_discarded = nested_array(depth, json(json::value_t::discarded));
CHECK_THROWS_AS(json::to_bon8(deep_discarded), json::type_error);
// errors are thrown from below the bound as from above it
const json deep_invalid_string = nested_array(depth, json(std::string("\x80")));
CHECK_THROWS_AS(json::to_bon8(deep_invalid_string), json::type_error);
json deep_invalid_key = json::object();
deep_invalid_key[std::string("\x80")] = 1;
deep_invalid_key = nested_object(depth, deep_invalid_key);
CHECK_THROWS_AS(json::to_bon8(deep_invalid_key), json::type_error);
const json deep_too_large = nested_array(depth, json(9223372036854775808u));
CHECK_THROWS_AS(json::to_bon8(deep_too_large), json::out_of_range);
}
SECTION("does not overflow the C++ stack")
{
const std::size_t depth = 100000;
const json j = json::parse(std::string(depth, '[') + "0" + std::string(depth, ']'));
std::vector<std::uint8_t> packed;
CHECK_NOTHROW(packed = json::to_cbor(j));
CHECK(json::from_cbor(packed) == j);
CHECK_NOTHROW(packed = json::to_msgpack(j));
CHECK(json::from_msgpack(packed) == j);
CHECK_NOTHROW(packed = json::to_ubjson(j));
CHECK(json::from_ubjson(packed) == j);
CHECK_NOTHROW(packed = json::to_ubjson(j, true, false));
CHECK(json::from_ubjson(packed) == j);
CHECK_NOTHROW(packed = json::to_bjdata(j));
CHECK(json::from_bjdata(packed) == j);
CHECK_NOTHROW(packed = json::to_bon8(j));
CHECK(json::from_bon8(packed) == j);
}
SECTION("BON8 containers with more than four elements past the recursion bound")
{
// such containers are closed with 0xFE, which the iterative writer
// must emit when it leaves them; strings next to each other must
// still be separated with 0xFF
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
json wide_array = json::array({1, "a", "b", json::array(), 5});
json wide_object = json({{"k1", "v"}, {"k2", 2}, {"k3", ""}, {"k4", json::object()}, {"k5", nullptr}});
for (std::size_t i = 0; i < depth; ++i)
{
wide_array = json::array({"s", wide_array, "t", wide_object, true});
wide_object = json({{"a", wide_object}, {"b", "x"}, {"c", "y"}, {"d", wide_array.size()}, {"e", "z"}});
}
CHECK(json::from_bon8(json::to_bon8(wide_array)) == wide_array);
CHECK(json::from_bon8(json::to_bon8(wide_object)) == wide_object);
}
SECTION("regression test for https://issues.oss-fuzz.com/issues/566583014")
{
// 200000 nested one-element CBOR arrays, the innermost holding null;
// round-tripping this used to recurse once per level on the way back
// out through to_cbor(), deep enough to overflow the stack
std::vector<std::uint8_t> v(200000, 0x81);
v.push_back(0xf6);
const json j = json::from_cbor(v);
CHECK(json::to_cbor(j) == v);
// the MessagePack analogue: fixarray of 1 nesting down to nil
std::vector<std::uint8_t> v_msgpack(200000, 0x91);
v_msgpack.push_back(0xc0);
const json j_msgpack = json::from_msgpack(v_msgpack);
CHECK(json::to_msgpack(j_msgpack) == v_msgpack);
}
SECTION("regression test for https://issues.oss-fuzz.com/issues/572238015")
{
// the BON8 analogue: 200000 nested one-element arrays down to null
std::vector<std::uint8_t> v(200000, 0x81);
v.push_back(0xFA);
const json j = json::from_bon8(v);
CHECK(json::to_bon8(j) == v);
}
}
-156
View File
@@ -1,156 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <cstddef>
#include <string>
#include <utility>
#include <vector>
// split from unit-large_json.cpp so that both run in parallel and stay below
// the ctest timeout under Valgrind
namespace
{
// The text of a value nested depth levels deep around the number 0. Level i is
// an array if pattern[i % pattern.size()] is '[', and otherwise an object with
// the single member "a", which every object type enumerates in the same order.
std::string nested_text(std::size_t depth, const std::string& pattern)
{
std::string text;
std::string closing;
for (std::size_t i = 0; i < depth; ++i)
{
const bool array = pattern[i % pattern.size()] == '[';
text += array ? "[" : "{\"a\":";
closing += array ? ']' : '}';
}
text += '0';
text.append(closing.rbegin(), closing.rend());
return text;
}
} // namespace
TEST_CASE("issue #5650 - stack overflow converting between specializations")
{
// deep enough to exhaust the call stack, but small enough to stay cheap:
// parsing is iterative, so building the values below costs little
const std::size_t depth = 100000;
const std::vector<std::string> patterns = {"[", "{", "[{"};
SECTION("json to ordered_json")
{
for (const auto& pattern : patterns)
{
CAPTURE(pattern)
const std::string text = nested_text(depth, pattern);
const json j = json::parse(text);
const nlohmann::ordered_json converted = j;
CHECK(converted.dump() == text);
}
}
SECTION("ordered_json to json")
{
for (const auto& pattern : patterns)
{
CAPTURE(pattern)
const std::string text = nested_text(depth, pattern);
const nlohmann::ordered_json o = nlohmann::ordered_json::parse(text);
const json converted = o;
CHECK(converted.dump() == text);
}
}
SECTION("get<ordered_json>()")
{
for (const auto& pattern : patterns)
{
CAPTURE(pattern)
const std::string text = nested_text(depth, pattern);
const json j = json::parse(text);
CHECK(j.get<nlohmann::ordered_json>().dump() == text);
}
}
SECTION("depths around the bound of the recursive descent")
{
for (std::size_t d = 1; d <= 300; ++d)
{
CAPTURE(d)
for (const auto& pattern : patterns)
{
CAPTURE(pattern)
const std::string text = nested_text(d, pattern);
const json j = json::parse(text);
const nlohmann::ordered_json converted = j;
CHECK(converted.dump() == text);
const json back = converted;
CHECK(back.dump() == text);
}
}
}
SECTION("values below the bound are converted as values above it")
{
// Bury a value below the bound, where it is converted without the
// call stack, and compare it with the same value converted on its
// own by the containers' range constructors. Its objects have
// members that the two object types enumerate in different orders.
const auto bury = [](nlohmann::ordered_json value)
{
for (std::size_t i = 0; i < 200; ++i)
{
value = nlohmann::ordered_json::array({std::move(value)});
}
return value;
};
const auto dig = [](const json & value)
{
const json* current = &value;
for (std::size_t i = 0; i < 200; ++i)
{
current = &current->at(0);
}
return current;
};
nlohmann::ordered_json value = nlohmann::ordered_json::object();
value["z"] = {1, -2, 3U, 4.5, true, nullptr, "six", nlohmann::ordered_json::binary({7, 8}, 9),
nlohmann::ordered_json::binary({10}), nlohmann::ordered_json::array(), nlohmann::ordered_json::object()
};
value["y"] = {{"x", {{"w", 1}, {"v", 2}}}, {"u", {3, {{"t", 4}, {"s", 5}}}}};
value["r"] = nlohmann::ordered_json::array({nlohmann::ordered_json(nlohmann::ordered_json::value_t::discarded)});
const json converted_above = value;
const json buried = bury(value);
const json& converted_below = *dig(buried);
CHECK(converted_below.dump() == converted_above.dump());
CHECK(converted_below.at("z").at(7).get_binary().subtype() == 9);
CHECK_FALSE(converted_below.at("z").at(8).get_binary().has_subtype());
CHECK(converted_below.at("r").at(0).is_discarded());
// a discarded value is never equal to anything, so compare the rest
value.erase("r");
const json without_discarded_above = value;
const json without_discarded_buried = bury(value);
CHECK(*dig(without_discarded_buried) == without_discarded_above);
}
}
+2 -8
View File
@@ -1374,13 +1374,7 @@ void check_dump_against_reference(const std::string& s, const bool with_strict =
};
for (const auto error_handler : error_handlers)
{
#if defined(JSON_NOEXCEPTION)
// a strict dump of invalid UTF-8 calls std::abort() instead of throwing
static_cast<void>(with_strict);
if (error_handler == json::error_handler_t::strict)
#else
if (!with_strict && error_handler == json::error_handler_t::strict)
#endif
{
continue;
}
@@ -1435,7 +1429,7 @@ TEST_CASE("invalid UTF-8 at every position is handled like the reference")
"\xF4\x90\x80\x80",
// sequences cut short by ASCII, a control character, a quote, a
// backslash, DEL, or the lead byte of a well-formed character
"\xC3" "a", "\xE2\x82" "\n", "\xE2\x82\"", "\xF0\x9F\x98\\", "\xC3\x7F", "\xC3\xC3\xA4", "\xE2\xF0\x9F\x98\x80", // NOLINT(bugprone-suspicious-missing-comma): "\xC3" "a" ends the hex escape
"\xC3" "a", "\xE2\x82" "\n", "\xE2\x82\"", "\xF0\x9F\x98\\", "\xC3\x7F", "\xC3\xC3\xA4", "\xE2\xF0\x9F\x98\x80",
// truncated sequences
"\xC3", "\xE2\x82", "\xF0\x9F\x98"
};
@@ -1495,7 +1489,7 @@ TEST_CASE("invalid UTF-8 at every position is handled like the reference")
{
for (const auto& pattern : patterns)
{
check_dump_against_reference(make_string(filler, pos, pattern, length - pos), bytes <= (filler.size() == 1 ? 17u : 8u));
check_dump_against_reference(make_string(filler, pos, pattern, length - pos), bytes <= (filler.size() == 1 ? 17 : 8));
}
}
}
-4
View File
@@ -218,11 +218,7 @@ TEST_CASE("compliance tests from nativejson-benchmark")
n1e308[0] = '[';
n1e308[1] = '1';
n1e308[310] = ']';
#if JSON_STRICT_NUL_HANDLING
n1e308.pop_back(); // a NUL byte after the value is a syntax error in strict mode
#else
n1e308[311] = '\0';
#endif
TEST_DOUBLE(n1e308, 1E308);
}
+9 -18
View File
@@ -54,8 +54,8 @@ TEST_CASE("exception messages of out-of-line throw sites")
SECTION("at")
{
const json& carr = arr;
const json& cobj = obj;
const json carr = arr;
const json cobj = obj;
CHECK_MSG((void)arr.at(static_cast<json::size_type>(10)), "[json.exception.out_of_range.401] array index 10 is out of range");
CHECK_MSG((void)carr.at(static_cast<json::size_type>(3)), "[json.exception.out_of_range.401] array index 3 is out of range");
CHECK_MSG((void)num.at(static_cast<json::size_type>(0)), "[json.exception.type_error.304] cannot use at() with number");
@@ -65,7 +65,7 @@ TEST_CASE("exception messages of out-of-line throw sites")
CHECK_MSG((void)num.at("a"), "[json.exception.type_error.304] cannot use at() with number");
nlohmann::ordered_json oobj = {{"a", 1}};
const nlohmann::ordered_json& coobj = oobj;
const nlohmann::ordered_json coobj = oobj;
CHECK_MSG((void)oobj.at("z"), "[json.exception.out_of_range.403] key 'z' not found");
CHECK_MSG((void)coobj.at("z"), "[json.exception.out_of_range.403] key 'z' not found");
CHECK_MSG((void)coobj.at(std::string("zz")), "[json.exception.out_of_range.403] key 'zz' not found");
@@ -94,10 +94,9 @@ TEST_CASE("exception messages of out-of-line throw sites")
CHECK_MSG(obj.update(arr.begin(), arr2.end()), "[json.exception.invalid_iterator.210] iterators do not fit");
}
#if !defined(JSON_NOEXCEPTION) // patch_msg() catches the exceptions, which JSON_NOEXCEPTION turns into std::abort()
SECTION("patch")
{
const auto patch_msg = [](const char* doc, const char* patch) -> std::string
const auto patch_msg = [](const char* doc, const char* patch)
{
try
{
@@ -106,23 +105,16 @@ TEST_CASE("exception messages of out-of-line throw sites")
}
catch (const json::exception& e)
{
// with JSON_DIAGNOSTICS, what() names the offending value like "(/0) ",
// and with JSON_DIAGNOSTIC_POSITIONS its position like "(bytes 0-2) "; drop both
// with JSON_DIAGNOSTICS, what() names the offending value like "(/0) "; drop it
std::string msg = e.what();
for (const char* marker :
{
" (/", " (bytes "
})
const auto pos = msg.find(" (/");
if (pos != std::string::npos)
{
const auto pos = msg.find(marker);
if (pos != std::string::npos)
{
msg.erase(pos + 1, msg.find(") ", pos) - pos + 1);
}
msg.erase(pos + 1, msg.find(") ", pos) - pos + 1);
}
return msg;
}
return {};
return std::string();
};
CHECK(patch_msg("{}", "{}") == "[json.exception.parse_error.104] parse error: JSON patch must be an array of objects");
@@ -140,5 +132,4 @@ TEST_CASE("exception messages of out-of-line throw sites")
CHECK(patch_msg("{\"a\":1}", "[{\"op\":\"test\",\"path\":\"/a\",\"value\":2}]") == "[json.exception.other_error.501] unsuccessful: {\"op\":\"test\",\"path\":\"/a\",\"value\":2}");
CHECK(patch_msg("{\"a\":1}", "[{\"op\":\"move\",\"path\":\"/a\"}]") == "[json.exception.parse_error.105] parse error: operation 'move' must have member 'from'");
}
#endif
}