Compare commits

..
Author SHA1 Message Date
Niels Lohmann 2241161307 insert(): strong guarantee for single elements, basic for the rest
Rework of the previous two commits, following the discussion in #5996:
the standard containers give the strong guarantee only for a single
element (std::vector: at the end, or if the exception does not come from
copying T; std::map: single-element insert), and copying a basic_json
allocates. Rebuilding a rollback for the count, range and object-range
overloads on top of array_t/object_t made the code verbose and slower.

- insert(pos, const basic_json&): at end() forward as before (the
  standard covers it); elsewhere copy first and insert by move, as
  libstdc++ does internally (libc++ shifts the tail before copying).
- insert_iterator() and object insert(first, last) set the parents in a
  scope guard, so a failed insertion leaves a valid value with
  JSON_DIAGNOSTICS too (the next assert_invariant() aborted before).
- Docs: strong guarantee for (1), basic guarantee for (2)-(5); the
  argument checks throw before anything is changed.
- Test: allocation-failure sweep checks the strong guarantee for (1) and
  validity, type and parents for the other overloads.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 08:54:34 +01:00
Claude 727a15c710 Fix clang-tidy and -Wnrvo findings in insert rollback helpers
Declare the rollback helpers non-movable, replace the nested conditional
and C array in object_insert_rollback, silence the intentional copy in the
allocator test, and avoid the missed NRVO in the test factory lambda.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>
2026-11-02 05:44:29 +00:00
Niels Lohmann c84e22cc62 Make basic_json::insert() provide its documented strong exception guarantee (#5986)
A failed allocation while the new elements were copied left the value
changed, and with JSON_DIAGNOSTICS stale parent pointers behind. The
extra work now happens almost only on the failure path:

- Array insert(pos, cnt, val) and insert(pos, first, last) append the
  copies at the end and rotate them into place. If a copy throws, the
  appended elements are erased again (noexcept) and the parents are set.
  Moving a basic_json cannot throw, so reallocation is safe. This is no
  slower than vector::insert (faster for numbers), needs no temporary
  array, and keeps the elements that libstdc++ lost on reallocation.
- Array insert(pos, val) copies a non-scalar value first and inserts it
  by move, except at the end, where vector::insert is strong. Scalars
  keep the direct path.
- The object range insert(first, last) into an empty object uses the
  container's own range insertion and clears the object on an exception.
  Into a non-empty object it inserts member by member and logs one bit
  per source member (a new member is detected by the size change; no
  allocation for the first 256 members); on an exception the members
  added are erased by walking the source range. No assumption that
  key_type is string_t.
- The parents are set when a failed insertion is undone.

The regression test fails every allocation of every insert() overload in
turn, for json and ordered_json, and checks the value and its parents.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 22:27:43 +01:00
20 changed files with 340 additions and 250 deletions

No files matched your search

+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
permissions:
actions: write
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
steps:
- name: Harden Runner
+1 -1
View File
@@ -19,7 +19,7 @@ permissions:
jobs:
save:
if: github.event_name == 'pull_request'
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -16,7 +16,7 @@ concurrency:
jobs:
check_docs_links:
if: github.repository == 'nlohmann/json'
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Harden Runner
@@ -12,7 +12,7 @@ jobs:
comment:
# push runs on develop have no PR to comment on (and no "pr" artifact)
if: ${{ github.event.workflow_run.conclusion == 'failure' && github.event.workflow_run.event == 'pull_request' }}
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
permissions:
contents: read
actions: read
+1 -1
View File
@@ -18,7 +18,7 @@ permissions:
jobs:
dependency-review:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -24,7 +24,7 @@ on:
jobs:
flawfinder:
name: Flawfinder
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
contents: read
pull-requests: write
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
steps:
- name: Harden Runner
+1 -1
View File
@@ -34,7 +34,7 @@ jobs:
contents: write
if: github.repository == 'nlohmann/json'
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -24,7 +24,7 @@ permissions:
jobs:
analysis:
name: Scorecard analysis
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
permissions:
# Needed to upload the results to code-scanning dashboard.
security-events: write
+1 -1
View File
@@ -8,7 +8,7 @@ permissions:
jobs:
stale:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
permissions:
issues: write
+13 -13
View File
@@ -18,7 +18,7 @@ concurrency:
jobs:
ci_test_gcc:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
container: gcc:latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -130,7 +130,7 @@ jobs:
run: cmake --build build --target ${{ matrix.target }}
ci_static_analysis_clang:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
container: silkeh/clang:22
strategy:
matrix:
@@ -149,7 +149,7 @@ jobs:
run: cmake --build build --target ${{ matrix.target }}
ci_cmake_options:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
container: ubuntu:24.04
strategy:
matrix:
@@ -209,7 +209,7 @@ jobs:
fail-on-error: false
ci_test_compilers_gcc_old:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['4.8', '4.9', '5', '6']
@@ -251,7 +251,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }}
ci_test_compilers_gcc:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
strategy:
matrix:
# older GCC docker images (4, 5, 6) fail to check out code
@@ -269,7 +269,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_default
ci_test_compilers_clang:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', '21', '22', 'latest']
@@ -289,7 +289,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_default
ci_test_standards_gcc:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
container: gcc:latest
strategy:
matrix:
@@ -306,7 +306,7 @@ jobs:
run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }}
ci_test_standards_clang:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
container: silkeh/clang:latest
strategy:
matrix:
@@ -330,7 +330,7 @@ jobs:
if: ${{ matrix.stdlib == 'libstdcxx' }}
ci_cuda_example:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
@@ -356,7 +356,7 @@ jobs:
strategy:
matrix:
container: ['gcc:latest', 'silkeh/clang:latest']
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
container: ${{ matrix.container }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -383,7 +383,7 @@ jobs:
run: cmake --build build --target ci_module_cpp20
ci_icpc:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
# Intel discontinued the classic icc/icpc compiler in oneAPI 2024.0; this is
# Intel's own last officially published image that still includes it.
container: intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04
@@ -402,7 +402,7 @@ jobs:
run: cmake --build build --target ci_icpc
ci_icpx:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
container: intel/oneapi-hpckit:latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -416,7 +416,7 @@ jobs:
run: cmake --build build --target ci_icpx
ci_nvhpc:
runs-on: ubuntu-26.04
runs-on: ubuntu-latest
container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+11 -1
View File
@@ -65,7 +65,17 @@ the same index, which is now a different value.
## Exception safety
Strong exception safety: if an exception occurs, the original value stays intact.
1. Strong guarantee: if an exception is thrown, there are no changes in the JSON value.
2. Basic guarantee: if an exception is thrown while the elements are copied (for example, `std::bad_alloc`), the JSON
value may be partially modified, but it stays a valid array.
3. Basic guarantee: if an exception is thrown while the elements are copied (for example, `std::bad_alloc`), the JSON
value may be partially modified, but it stays a valid array.
4. Basic guarantee: if an exception is thrown while the elements are copied (for example, `std::bad_alloc`), the JSON
value may be partially modified, but it stays a valid array.
5. Basic guarantee: if an exception is thrown while the members are copied (for example, `std::bad_alloc`), the JSON
value may be partially modified, but it stays a valid object.
The exceptions listed below are thrown before the JSON value is changed.
## Exceptions
@@ -467,13 +467,11 @@ class binary_reader
const std::size_t element_type_parse_position = chars_read;
// an array's elements are named "0", "1", ... in the wire format,
// and those names are not passed on: they are skipped, and only
// checked if error_handler_t::strict must reject ill-formed UTF-8
// and those names are not passed on: they are skipped unless
// error_handler_t::strict must reject ill-formed UTF-8 in them
const bool is_object = container_stack.back().is_object;
key.clear();
if (JSON_HEDLEY_UNLIKELY(!(is_object || error_handler != error_handler_t::strict
? get_bson_cstr(key, is_object)
: check_bson_cstr(key, std::integral_constant<bool, bulk_scan> {}))))
if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key, is_object || error_handler == error_handler_t::strict)))
{
return false;
}
@@ -497,7 +495,7 @@ class binary_reader
@param[in] store whether the characters are needed; if not (the names of
an array's elements), they are skipped and @a result may
be left unchanged, and they are not checked against
@ref error_handler (@ref check_bson_cstr checks them)
@ref error_handler
@return `true` if the \\x00-byte indicating the end of the string was
encountered before the EOF; false` indicates an unexpected EOF.
*/
@@ -556,42 +554,6 @@ class binary_reader
return true;
}
/*!
@brief skip a C-style string that error_handler_t::strict must check
The names of an array's elements are not passed on. On contiguous input,
a well-formed name is checked in place and skipped without being copied.
Otherwise (no \\x00-byte in the input that can be scanned, or ill-formed
UTF-8), it is read by @ref get_bson_cstr, which reports the error.
@param[in,out] result an empty string, which may receive the name
@return whether the name was read and is well-formed
*/
bool check_bson_cstr(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining != 0)
{
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = find_zero_byte(data, remaining);
if (length != remaining && JSON_HEDLEY_LIKELY(valid_utf8_string(data, length)))
{
// consume the string and its \x00-byte, as get_bson_cstr does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
}
return get_bson_cstr(result);
}
/// input that is not contiguous: the name is read and checked byte by byte
bool check_bson_cstr(string_t& result, std::false_type /*bulk*/)
{
return get_bson_cstr(result);
}
/// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept
{
@@ -1856,18 +1818,14 @@ class binary_reader
Only the start of an array is tried, which covers arrays of numbers
(e.g. coordinates) but costs arrays of other values only one call, not one
per element. An array of one element is not tried at all: that element is
mostly an object or an array (`[[...]]`, the single ring of a GeoJSON
polygon), for which the call would find no scalar, while a single scalar,
which the run would read faster, is rare. The benchmarks
`FromBinaryShape/nested` and `FromBinaryShape/singles` show both sides.
per element.
@param[in] len the number of elements
@return whether the SAX parser accepted the array and the values read
*/
bool enter_cbor_array(const std::size_t len)
{
return enter_array(len) && (len <= 1 || parse_cbor_scalar_run(std::integral_constant<bool, bulk_scan> {}));
return enter_array(len) && (len == 0 || parse_cbor_scalar_run(std::integral_constant<bool, bulk_scan> {}));
}
/// input that is not contiguous: every value is read by parse_cbor_value
@@ -2768,16 +2726,12 @@ class binary_reader
/*!
@brief open a MessagePack array and read the scalars it begins with (see
@ref parse_msgpack_scalar_run)
An array of one element is not tried, for the reason given at
@ref enter_cbor_array.
@param[in] len the number of elements
@return whether the SAX parser accepted the array and the values read
*/
bool enter_msgpack_array(const std::size_t len)
{
return enter_array(len) && (len <= 1 || parse_msgpack_scalar_run(std::integral_constant<bool, bulk_scan> {}));
return enter_array(len) && (len == 0 || parse_msgpack_scalar_run(std::integral_constant<bool, bulk_scan> {}));
}
/// input that is not contiguous: every value is read by parse_msgpack_value
+36 -4
View File
@@ -4914,6 +4914,30 @@ public:
return {it, res.second};
}
private:
/// Sets the parents of a value when it goes out of scope, also when an
/// exception is thrown: a container insertion that throws may already
/// have added elements, whose parents must be set for the value to stay
/// valid (basic exception guarantee). Does nothing without JSON_DIAGNOSTICS.
class set_parents_on_exit
{
public:
explicit set_parents_on_exit(basic_json* v) noexcept : m_value(v) {}
set_parents_on_exit(const set_parents_on_exit&) = delete;
set_parents_on_exit& operator=(const set_parents_on_exit&) = delete;
set_parents_on_exit(set_parents_on_exit&&) = delete;
set_parents_on_exit& operator=(set_parents_on_exit&&) = delete;
~set_parents_on_exit()
{
m_value->set_parents();
}
private:
basic_json* m_value;
};
public:
/// Helper for insertion of an iterator
/// @note: This uses std::distance to support GCC 4.8,
/// see https://github.com/nlohmann/json/pull/1257
@@ -4923,6 +4947,7 @@ public:
iterator result(this);
JSON_ASSERT(m_data.m_value.array != nullptr);
const set_parents_on_exit parents(this);
auto insert_pos = std::distance(m_data.m_value.array->begin(), pos.m_it.array_iterator);
m_data.m_value.array->insert(pos.m_it.array_iterator, std::forward<Args>(args)...);
result.m_it.array_iterator = m_data.m_value.array->begin() + insert_pos;
@@ -4931,7 +4956,6 @@ public:
// result.m_it.array_iterator = m_data.m_value.array->insert(pos.m_it.array_iterator, cnt, val);
// but the return value of insert is missing in GCC 4.8, so it is written this way instead.
set_parents();
return result;
}
@@ -4951,8 +4975,16 @@ public:
throw_iterator_from_other_value(this);
}
// insert to array and return iterator
return insert_iterator(pos, val);
// vector::insert has no effects if copying a single element to the
// end throws; elsewhere, it may already have shifted the elements, so
// copy first and insert by move, which cannot throw but by allocating
if (pos.m_it.array_iterator == m_data.m_value.array->end())
{
return insert_iterator(pos, val);
}
basic_json tmp(val);
return insert_iterator(pos, std::move(tmp));
}
/// @brief inserts element into array
@@ -5084,8 +5116,8 @@ public:
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterators first and last must point to objects", this);
}
const set_parents_on_exit parents(this);
m_data.m_value.object->insert(first.m_it.object_iterator, last.m_it.object_iterator);
set_parents();
}
/// @brief updates a JSON object from another object, overwriting existing keys
+43 -57
View File
@@ -18474,13 +18474,11 @@ class binary_reader
const std::size_t element_type_parse_position = chars_read;
// an array's elements are named "0", "1", ... in the wire format,
// and those names are not passed on: they are skipped, and only
// checked if error_handler_t::strict must reject ill-formed UTF-8
// and those names are not passed on: they are skipped unless
// error_handler_t::strict must reject ill-formed UTF-8 in them
const bool is_object = container_stack.back().is_object;
key.clear();
if (JSON_HEDLEY_UNLIKELY(!(is_object || error_handler != error_handler_t::strict
? get_bson_cstr(key, is_object)
: check_bson_cstr(key, std::integral_constant<bool, bulk_scan> {}))))
if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key, is_object || error_handler == error_handler_t::strict)))
{
return false;
}
@@ -18504,7 +18502,7 @@ class binary_reader
@param[in] store whether the characters are needed; if not (the names of
an array's elements), they are skipped and @a result may
be left unchanged, and they are not checked against
@ref error_handler (@ref check_bson_cstr checks them)
@ref error_handler
@return `true` if the \\x00-byte indicating the end of the string was
encountered before the EOF; false` indicates an unexpected EOF.
*/
@@ -18563,42 +18561,6 @@ class binary_reader
return true;
}
/*!
@brief skip a C-style string that error_handler_t::strict must check
The names of an array's elements are not passed on. On contiguous input,
a well-formed name is checked in place and skipped without being copied.
Otherwise (no \\x00-byte in the input that can be scanned, or ill-formed
UTF-8), it is read by @ref get_bson_cstr, which reports the error.
@param[in,out] result an empty string, which may receive the name
@return whether the name was read and is well-formed
*/
bool check_bson_cstr(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining != 0)
{
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = find_zero_byte(data, remaining);
if (length != remaining && JSON_HEDLEY_LIKELY(valid_utf8_string(data, length)))
{
// consume the string and its \x00-byte, as get_bson_cstr does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
}
return get_bson_cstr(result);
}
/// input that is not contiguous: the name is read and checked byte by byte
bool check_bson_cstr(string_t& result, std::false_type /*bulk*/)
{
return get_bson_cstr(result);
}
/// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept
{
@@ -19863,18 +19825,14 @@ class binary_reader
Only the start of an array is tried, which covers arrays of numbers
(e.g. coordinates) but costs arrays of other values only one call, not one
per element. An array of one element is not tried at all: that element is
mostly an object or an array (`[[...]]`, the single ring of a GeoJSON
polygon), for which the call would find no scalar, while a single scalar,
which the run would read faster, is rare. The benchmarks
`FromBinaryShape/nested` and `FromBinaryShape/singles` show both sides.
per element.
@param[in] len the number of elements
@return whether the SAX parser accepted the array and the values read
*/
bool enter_cbor_array(const std::size_t len)
{
return enter_array(len) && (len <= 1 || parse_cbor_scalar_run(std::integral_constant<bool, bulk_scan> {}));
return enter_array(len) && (len == 0 || parse_cbor_scalar_run(std::integral_constant<bool, bulk_scan> {}));
}
/// input that is not contiguous: every value is read by parse_cbor_value
@@ -20775,16 +20733,12 @@ class binary_reader
/*!
@brief open a MessagePack array and read the scalars it begins with (see
@ref parse_msgpack_scalar_run)
An array of one element is not tried, for the reason given at
@ref enter_cbor_array.
@param[in] len the number of elements
@return whether the SAX parser accepted the array and the values read
*/
bool enter_msgpack_array(const std::size_t len)
{
return enter_array(len) && (len <= 1 || parse_msgpack_scalar_run(std::integral_constant<bool, bulk_scan> {}));
return enter_array(len) && (len == 0 || parse_msgpack_scalar_run(std::integral_constant<bool, bulk_scan> {}));
}
/// input that is not contiguous: every value is read by parse_msgpack_value
@@ -42752,6 +42706,30 @@ public:
return {it, res.second};
}
private:
/// Sets the parents of a value when it goes out of scope, also when an
/// exception is thrown: a container insertion that throws may already
/// have added elements, whose parents must be set for the value to stay
/// valid (basic exception guarantee). Does nothing without JSON_DIAGNOSTICS.
class set_parents_on_exit
{
public:
explicit set_parents_on_exit(basic_json* v) noexcept : m_value(v) {}
set_parents_on_exit(const set_parents_on_exit&) = delete;
set_parents_on_exit& operator=(const set_parents_on_exit&) = delete;
set_parents_on_exit(set_parents_on_exit&&) = delete;
set_parents_on_exit& operator=(set_parents_on_exit&&) = delete;
~set_parents_on_exit()
{
m_value->set_parents();
}
private:
basic_json* m_value;
};
public:
/// Helper for insertion of an iterator
/// @note: This uses std::distance to support GCC 4.8,
/// see https://github.com/nlohmann/json/pull/1257
@@ -42761,6 +42739,7 @@ public:
iterator result(this);
JSON_ASSERT(m_data.m_value.array != nullptr);
const set_parents_on_exit parents(this);
auto insert_pos = std::distance(m_data.m_value.array->begin(), pos.m_it.array_iterator);
m_data.m_value.array->insert(pos.m_it.array_iterator, std::forward<Args>(args)...);
result.m_it.array_iterator = m_data.m_value.array->begin() + insert_pos;
@@ -42769,7 +42748,6 @@ public:
// result.m_it.array_iterator = m_data.m_value.array->insert(pos.m_it.array_iterator, cnt, val);
// but the return value of insert is missing in GCC 4.8, so it is written this way instead.
set_parents();
return result;
}
@@ -42789,8 +42767,16 @@ public:
throw_iterator_from_other_value(this);
}
// insert to array and return iterator
return insert_iterator(pos, val);
// vector::insert has no effects if copying a single element to the
// end throws; elsewhere, it may already have shifted the elements, so
// copy first and insert by move, which cannot throw but by allocating
if (pos.m_it.array_iterator == m_data.m_value.array->end())
{
return insert_iterator(pos, val);
}
basic_json tmp(val);
return insert_iterator(pos, std::move(tmp));
}
/// @brief inserts element into array
@@ -42922,8 +42908,8 @@ public:
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterators first and last must point to objects", this);
}
const set_parents_on_exit parents(this);
m_data.m_value.object->insert(first.m_it.object_iterator, last.m_it.object_iterator);
set_parents();
}
/// @brief updates a JSON object from another object, overwriting existing keys
-18
View File
@@ -629,22 +629,6 @@ static json make_containers()
return j;
}
/// many arrays of one number each: one-element arrays are left to the general
/// path (CBOR and MessagePack), which reads a lone number slower than the loop
/// for arrays of scalars would
static json make_singles()
{
json data = json::array();
for (std::size_t i = 0; i < 100000; ++i)
{
data.push_back(json::array({i}));
}
json j = json::object();
j["data"] = std::move(data);
return j;
}
/// one flat array of numbers: the scalar decoding path, which must not move
static json make_scalars()
{
@@ -692,8 +676,6 @@ BENCHMARK_CAPTURE(FromBinaryShape, containers / ubjson_optimized, make_container
BENCHMARK_CAPTURE(FromBinaryShape, containers / bjdata, make_containers, binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryShape, containers / bson, make_containers, binary_format::bson);
BENCHMARK_CAPTURE(FromBinaryShape, containers / bon8, make_containers, binary_format::bon8);
BENCHMARK_CAPTURE(FromBinaryShape, singles / cbor, make_singles, binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryShape, singles / msgpack, make_singles, binary_format::msgpack);
// BSON names every array element, so a large array measures key generation
// rather than scalar decoding and is left out here
BENCHMARK_CAPTURE(FromBinaryShape, scalars / cbor, make_scalars, binary_format::cbor);
+220
View File
@@ -12,6 +12,7 @@
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <functional>
#include <valarray>
#if JSON_HAS_RANGES
#include <ranges>
@@ -804,3 +805,222 @@ TEST_CASE("a failed allocation leaves the value unchanged")
#endif
}
#endif
// the no-exceptions CI job has no std::bad_alloc to catch; with iterator
// debugging (MSVC STL debug builds) containers allocate inside noexcept
// functions, so failing an allocation there terminates the program
#if !defined(JSON_NOEXCEPTION) && !(defined(_ITERATOR_DEBUG_LEVEL) && _ITERATOR_DEBUG_LEVEL > 0)
namespace
{
// whether every child of j names j as its parent (always true without
// JSON_DIAGNOSTICS, where no parents are stored)
template<class BasicJsonType>
bool children_name_their_parent(const BasicJsonType& j)
{
#if JSON_DIAGNOSTICS
if (j.is_array())
{
for (const auto& element : *j.m_data.m_value.array)
{
if (element.m_parent != &j)
{
return false;
}
}
}
else if (j.is_object())
{
for (const auto& element : *j.m_data.m_value.object)
{
if (element.second.m_parent != &j)
{
return false;
}
}
}
#else
static_cast<void>(j);
#endif
return true;
}
enum class guarantee
{
basic, // the value stays valid and keeps its type
strong // the value is unchanged
};
// Runs op on a fresh value make() with the 0th, 1st, 2nd, ... allocation made
// by the library failing in turn, until op completes. Every failed attempt
// must have thrown std::bad_alloc and left the value valid, with all parent
// pointers intact (copying a value asserts the invariant), and for the strong
// guarantee exactly as it was.
template<class BasicJsonType>
void check_exception_guarantee(guarantee g,
const std::function<BasicJsonType()>& make,
const std::function<void(BasicJsonType&)>& op)
{
fail_at_alloc_call = -1;
bool completed = false;
std::size_t failures = 0;
for (std::size_t n = 0; !completed; ++n)
{
CAPTURE(n)
REQUIRE(n < 1000);
BasicJsonType j = make();
const std::string before = j.dump();
const auto type_before = j.type();
alloc_call_count = 0;
fail_at_alloc_call = static_cast<long>(n);
try
{
op(j);
completed = true;
}
catch (const std::bad_alloc&)
{
++failures;
}
fail_at_alloc_call = -1;
if (!completed)
{
CHECK((j.type() == type_before));
CHECK(children_name_their_parent(j));
const BasicJsonType copy(j); // NOLINT(performance-unnecessary-copy-initialization): the copy is the point, it allocates
CHECK(copy.dump() == j.dump());
if (g == guarantee::strong)
{
CHECK(j.dump() == before);
}
}
}
// the operation allocates, so at least one attempt must have failed
CHECK(failures > 0);
}
template<class BasicJsonType>
void check_insert_exception_guarantees()
{
using json_t = BasicJsonType;
using make_t = std::function<json_t()>;
// strings of this length do not fit into the small string buffer, so
// copying them allocates
const json_t val = std::string(64, 'x');
const json_t source = json_t::array({std::string(64, 'p'), std::string(64, 'q'), std::string(64, 'r')});
for (const bool spare_capacity :
{
false, true
})
{
CAPTURE(spare_capacity)
// [ "aaa...", "bbb...", "ccc..." ], with a spare slot if requested
const make_t make = [spare_capacity]()
{
json_t j = json_t::array({std::string(64, 'a'), std::string(64, 'b'), std::string(64, 'c')});
if (spare_capacity)
{
j.push_back(std::string(64, 'd'));
j.erase(3);
}
return json_t(std::move(j));
};
for (const bool at_end :
{
false, true
})
{
CAPTURE(at_end)
SECTION("insert(pos, const basic_json&)")
{
check_exception_guarantee<json_t>(guarantee::strong, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, val);
});
}
SECTION("insert(pos, basic_json&&)")
{
check_exception_guarantee<json_t>(guarantee::strong, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, json_t(val));
});
}
SECTION("insert(pos, size_type, const basic_json&)")
{
check_exception_guarantee<json_t>(guarantee::basic, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, 3, val);
});
}
SECTION("insert(pos, const_iterator, const_iterator)")
{
check_exception_guarantee<json_t>(guarantee::basic, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, source.begin(), source.end());
});
}
SECTION("insert(pos, initializer_list_t)")
{
check_exception_guarantee<json_t>(guarantee::basic, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, {val, val});
});
}
}
}
SECTION("insert(const_iterator, const_iterator) for objects")
{
const json_t members = {{"m", std::string(64, 'm')}, {"n", std::string(64, 'n')}, {"o", std::string(64, 'o')}};
SECTION("no member exists yet")
{
check_exception_guarantee<json_t>(guarantee::basic, []()
{
return json_t({{"a", 1}});
}, [&](json_t& j)
{
j.insert(members.begin(), members.end());
});
}
SECTION("some members exist already")
{
check_exception_guarantee<json_t>(guarantee::basic, []()
{
return json_t({{"a", 1}, {"n", std::string(64, 'x')}});
}, [&](json_t& j)
{
j.insert(members.begin(), members.end());
});
}
}
}
} // namespace
TEST_CASE("insert() provides its documented exception guarantees (#5986)")
{
SECTION("std::map-backed object_t")
{
check_insert_exception_guarantees<nlohmann::json::with_allocator_t<nth_alloc_fails_allocator>>();
}
SECTION("ordered_map-backed object_t")
{
check_insert_exception_guarantees<nlohmann::ordered_json::with_allocator_t<nth_alloc_fails_allocator>>();
}
}
#endif
-38
View File
@@ -23,7 +23,6 @@ using nlohmann::json;
#include <cstdint>
#include <fstream>
#include <limits>
#include <list>
#include <sstream>
#include <vector>
#include "make_test_data_available.hpp"
@@ -1200,31 +1199,6 @@ TEST_CASE("BSON keys from contiguous and stream input")
from_stream = e.what();
}
CHECK(from_vector == from_stream);
// error_handler_t::strict checks the array element names in place on
// contiguous input and byte by byte otherwise; the names are
// well-formed, so both give the same value or error as above
std::string strict_from_vector;
std::string strict_from_list;
try
{
strict_from_vector = json::from_bson(input, true, true, json::error_handler_t::strict).dump();
}
catch (const json::parse_error& e)
{
strict_from_vector = e.what();
}
try
{
const std::list<std::uint8_t> list(input.begin(), input.end());
strict_from_list = json::from_bson(list.begin(), list.end(), true, true, json::error_handler_t::strict).dump();
}
catch (const json::parse_error& e)
{
strict_from_list = e.what();
}
CHECK(strict_from_vector == from_vector);
CHECK(strict_from_list == from_vector);
}
}
@@ -1253,18 +1227,6 @@ TEST_CASE("BSON array element names")
std::istringstream stream(std::string(reinterpret_cast<const char*>(bson.data()), bson.size()));
CHECK(json::from_bson(stream) == expected);
// input that is not contiguous reads the names byte by byte and must reject
// the same name
const std::list<std::uint8_t> list(bson.begin(), bson.end());
CHECK_THROWS_WITH_AS(_ = json::from_bson(list.begin(), list.end(), true, true, json::error_handler_t::strict), "[json.exception.parse_error.113] parse error at byte 14: syntax error while parsing BSON key: invalid string: ill-formed UTF-8 byte", json::parse_error&);
// without exceptions, the rejected name ends the parse
CHECK(json::from_bson(bson, true, false, json::error_handler_t::strict).is_discarded());
// well-formed names pass the check without being stored
const json valid = {{"a", {true, false, {{"k", json::array({1, 2})}}}}};
CHECK(json::from_bson(json::to_bson(valid), true, true, json::error_handler_t::strict) == valid);
}
#endif
-28
View File
@@ -2520,34 +2520,6 @@ TEST_CASE("CBOR arrays of scalars from contiguous and non-contiguous input")
}
}
TEST_CASE("CBOR arrays of one element from contiguous and non-contiguous input")
{
// contiguous input leaves an array of one element to the general path
// rather than to the loop that reads arrays of scalars; both inputs must
// still give the same events, value, or error for the complete document
// and for every truncation of it, with enough bytes after each array that
// the loop could have been entered
const json j = json::parse(R"({"a": [[[[1.5]]]], "b": [7], "c": [{"x": 2.5}], "d": ["s"], "e": [[1, 2]], "f": [null], "g": 1.25})");
const std::vector<std::uint8_t> bytes = json::to_cbor(j);
CHECK(json::from_cbor(bytes) == j);
for (std::size_t length = 0; length <= bytes.size(); ++length)
{
CAPTURE(length)
const std::vector<std::uint8_t> contiguous(bytes.begin(), bytes.begin() + static_cast<std::ptrdiff_t>(length));
const std::list<std::uint8_t> bytewise(contiguous.begin(), contiguous.end());
CHECK(json::from_cbor(contiguous, true, false).dump() == json::from_cbor(bytewise.begin(), bytewise.end(), true, false).dump());
SaxEventLogger from_contiguous;
SaxEventLogger from_bytewise;
CHECK(json::sax_parse(contiguous, &from_contiguous, json::input_format_t::cbor, true, false)
== json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::cbor, true, false));
CHECK(from_contiguous.events == from_bytewise.events);
}
}
TEST_CASE("CBOR indefinite-length strings do not recurse per chunk")
{
// Reading an indefinite-length string or byte array used to call itself
-28
View File
@@ -2311,31 +2311,3 @@ TEST_CASE("MessagePack arrays of scalars from contiguous and non-contiguous inpu
== json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::msgpack));
}
}
TEST_CASE("MessagePack arrays of one element from contiguous and non-contiguous input")
{
// contiguous input leaves an array of one element to the general path
// rather than to the loop that reads arrays of scalars; both inputs must
// still give the same events, value, or error for the complete document
// and for every truncation of it, with enough bytes after each array that
// the loop could have been entered
const json j = json::parse(R"({"a": [[[[1.5]]]], "b": [7], "c": [{"x": 2.5}], "d": ["s"], "e": [[1, 2]], "f": [null], "g": 1.25})");
const std::vector<std::uint8_t> bytes = json::to_msgpack(j);
CHECK(json::from_msgpack(bytes) == j);
for (std::size_t length = 0; length <= bytes.size(); ++length)
{
CAPTURE(length)
const std::vector<std::uint8_t> contiguous(bytes.begin(), bytes.begin() + static_cast<std::ptrdiff_t>(length));
const std::list<std::uint8_t> bytewise(contiguous.begin(), contiguous.end());
CHECK(json::from_msgpack(contiguous, true, false).dump() == json::from_msgpack(bytewise.begin(), bytewise.end(), true, false).dump());
SaxEventLogger from_contiguous;
SaxEventLogger from_bytewise;
CHECK(json::sax_parse(contiguous, &from_contiguous, json::input_format_t::msgpack, true, false)
== json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::msgpack, true, false));
CHECK(from_contiguous.events == from_bytewise.events);
}
}