Compare commits

..
Author SHA1 Message Date
Niels Lohmann 2241161307 insert(): strong guarantee for single elements, basic for the rest
Rework of the previous two commits, following the discussion in #5996:
the standard containers give the strong guarantee only for a single
element (std::vector: at the end, or if the exception does not come from
copying T; std::map: single-element insert), and copying a basic_json
allocates. Rebuilding a rollback for the count, range and object-range
overloads on top of array_t/object_t made the code verbose and slower.

- insert(pos, const basic_json&): at end() forward as before (the
  standard covers it); elsewhere copy first and insert by move, as
  libstdc++ does internally (libc++ shifts the tail before copying).
- insert_iterator() and object insert(first, last) set the parents in a
  scope guard, so a failed insertion leaves a valid value with
  JSON_DIAGNOSTICS too (the next assert_invariant() aborted before).
- Docs: strong guarantee for (1), basic guarantee for (2)-(5); the
  argument checks throw before anything is changed.
- Test: allocation-failure sweep checks the strong guarantee for (1) and
  validity, type and parents for the other overloads.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 08:54:34 +01:00
Claude 727a15c710 Fix clang-tidy and -Wnrvo findings in insert rollback helpers
Declare the rollback helpers non-movable, replace the nested conditional
and C array in object_insert_rollback, silence the intentional copy in the
allocator test, and avoid the missed NRVO in the test factory lambda.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>
2026-11-02 05:44:29 +00:00
Niels Lohmann c84e22cc62 Make basic_json::insert() provide its documented strong exception guarantee (#5986)
A failed allocation while the new elements were copied left the value
changed, and with JSON_DIAGNOSTICS stale parent pointers behind. The
extra work now happens almost only on the failure path:

- Array insert(pos, cnt, val) and insert(pos, first, last) append the
  copies at the end and rotate them into place. If a copy throws, the
  appended elements are erased again (noexcept) and the parents are set.
  Moving a basic_json cannot throw, so reallocation is safe. This is no
  slower than vector::insert (faster for numbers), needs no temporary
  array, and keeps the elements that libstdc++ lost on reallocation.
- Array insert(pos, val) copies a non-scalar value first and inserts it
  by move, except at the end, where vector::insert is strong. Scalars
  keep the direct path.
- The object range insert(first, last) into an empty object uses the
  container's own range insertion and clears the object on an exception.
  Into a non-empty object it inserts member by member and logs one bit
  per source member (a new member is detected by the size change; no
  allocation for the first 256 members); on an exception the members
  added are erased by walking the source range. No assumption that
  key_type is string_t.
- The parents are set when a failed insertion is undone.

The regression test fails every allocation of every insert() overload in
turn, for json and ordered_json, and checks the value and its parents.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 22:27:43 +01:00
19 changed files with 445 additions and 211 deletions

No files matched your search

+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
permissions: permissions:
actions: write actions: write
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
+1 -1
View File
@@ -19,7 +19,7 @@ permissions:
jobs: jobs:
save: save:
if: github.event_name == 'pull_request' if: github.event_name == 'pull_request'
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2 uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -16,7 +16,7 @@ concurrency:
jobs: jobs:
check_docs_links: check_docs_links:
if: github.repository == 'nlohmann/json' if: github.repository == 'nlohmann/json'
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
timeout-minutes: 30 timeout-minutes: 30
steps: steps:
- name: Harden Runner - name: Harden Runner
@@ -12,7 +12,7 @@ jobs:
comment: comment:
# push runs on develop have no PR to comment on (and no "pr" artifact) # push runs on develop have no PR to comment on (and no "pr" artifact)
if: ${{ github.event.workflow_run.conclusion == 'failure' && github.event.workflow_run.event == 'pull_request' }} if: ${{ github.event.workflow_run.conclusion == 'failure' && github.event.workflow_run.event == 'pull_request' }}
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
permissions: permissions:
contents: read contents: read
actions: read actions: read
+1 -1
View File
@@ -18,7 +18,7 @@ permissions:
jobs: jobs:
dependency-review: dependency-review:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2 uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -24,7 +24,7 @@ on:
jobs: jobs:
flawfinder: flawfinder:
name: Flawfinder name: Flawfinder
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
permissions: permissions:
actions: read actions: read
contents: read contents: read
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
contents: read contents: read
pull-requests: write pull-requests: write
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
+1 -1
View File
@@ -34,7 +34,7 @@ jobs:
contents: write contents: write
if: github.repository == 'nlohmann/json' if: github.repository == 'nlohmann/json'
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2 uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -24,7 +24,7 @@ permissions:
jobs: jobs:
analysis: analysis:
name: Scorecard analysis name: Scorecard analysis
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
permissions: permissions:
# Needed to upload the results to code-scanning dashboard. # Needed to upload the results to code-scanning dashboard.
security-events: write security-events: write
+1 -1
View File
@@ -8,7 +8,7 @@ permissions:
jobs: jobs:
stale: stale:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
permissions: permissions:
issues: write issues: write
+13 -13
View File
@@ -18,7 +18,7 @@ concurrency:
jobs: jobs:
ci_test_gcc: ci_test_gcc:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
container: gcc:latest container: gcc:latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -130,7 +130,7 @@ jobs:
run: cmake --build build --target ${{ matrix.target }} run: cmake --build build --target ${{ matrix.target }}
ci_static_analysis_clang: ci_static_analysis_clang:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
container: silkeh/clang:22 container: silkeh/clang:22
strategy: strategy:
matrix: matrix:
@@ -149,7 +149,7 @@ jobs:
run: cmake --build build --target ${{ matrix.target }} run: cmake --build build --target ${{ matrix.target }}
ci_cmake_options: ci_cmake_options:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
container: ubuntu:24.04 container: ubuntu:24.04
strategy: strategy:
matrix: matrix:
@@ -209,7 +209,7 @@ jobs:
fail-on-error: false fail-on-error: false
ci_test_compilers_gcc_old: ci_test_compilers_gcc_old:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
compiler: ['4.8', '4.9', '5', '6'] compiler: ['4.8', '4.9', '5', '6']
@@ -251,7 +251,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }} run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }}
ci_test_compilers_gcc: ci_test_compilers_gcc:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
# older GCC docker images (4, 5, 6) fail to check out code # older GCC docker images (4, 5, 6) fail to check out code
@@ -269,7 +269,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_default run: cmake --build build --target ci_test_compiler_default
ci_test_compilers_clang: ci_test_compilers_clang:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', '21', '22', 'latest'] compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', '21', '22', 'latest']
@@ -289,7 +289,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_default run: cmake --build build --target ci_test_compiler_default
ci_test_standards_gcc: ci_test_standards_gcc:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
container: gcc:latest container: gcc:latest
strategy: strategy:
matrix: matrix:
@@ -306,7 +306,7 @@ jobs:
run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }} run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }}
ci_test_standards_clang: ci_test_standards_clang:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
container: silkeh/clang:latest container: silkeh/clang:latest
strategy: strategy:
matrix: matrix:
@@ -330,7 +330,7 @@ jobs:
if: ${{ matrix.stdlib == 'libstdcxx' }} if: ${{ matrix.stdlib == 'libstdcxx' }}
ci_cuda_example: ci_cuda_example:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -356,7 +356,7 @@ jobs:
strategy: strategy:
matrix: matrix:
container: ['gcc:latest', 'silkeh/clang:latest'] container: ['gcc:latest', 'silkeh/clang:latest']
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
container: ${{ matrix.container }} container: ${{ matrix.container }}
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -383,7 +383,7 @@ jobs:
run: cmake --build build --target ci_module_cpp20 run: cmake --build build --target ci_module_cpp20
ci_icpc: ci_icpc:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
# Intel discontinued the classic icc/icpc compiler in oneAPI 2024.0; this is # Intel discontinued the classic icc/icpc compiler in oneAPI 2024.0; this is
# Intel's own last officially published image that still includes it. # Intel's own last officially published image that still includes it.
container: intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04 container: intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04
@@ -402,7 +402,7 @@ jobs:
run: cmake --build build --target ci_icpc run: cmake --build build --target ci_icpc
ci_icpx: ci_icpx:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
container: intel/oneapi-hpckit:latest container: intel/oneapi-hpckit:latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -416,7 +416,7 @@ jobs:
run: cmake --build build --target ci_icpx run: cmake --build build --target ci_icpx
ci_nvhpc: ci_nvhpc:
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04 container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+11 -1
View File
@@ -65,7 +65,17 @@ the same index, which is now a different value.
## Exception safety ## Exception safety
Strong exception safety: if an exception occurs, the original value stays intact. 1. Strong guarantee: if an exception is thrown, there are no changes in the JSON value.
2. Basic guarantee: if an exception is thrown while the elements are copied (for example, `std::bad_alloc`), the JSON
value may be partially modified, but it stays a valid array.
3. Basic guarantee: if an exception is thrown while the elements are copied (for example, `std::bad_alloc`), the JSON
value may be partially modified, but it stays a valid array.
4. Basic guarantee: if an exception is thrown while the elements are copied (for example, `std::bad_alloc`), the JSON
value may be partially modified, but it stays a valid array.
5. Basic guarantee: if an exception is thrown while the members are copied (for example, `std::bad_alloc`), the JSON
value may be partially modified, but it stays a valid object.
The exceptions listed below are thrown before the JSON value is changed.
## Exceptions ## Exceptions
+49 -62
View File
@@ -344,8 +344,8 @@ class binary_reader
nested document could declare a length that disagrees with where its nested document could declare a length that disagrees with where its
terminator actually falls and quietly hand the bytes in between to the terminator actually falls and quietly hand the bytes in between to the
enclosing document. A well-formed document is at least 5 bytes (the prefix enclosing document. A well-formed document is at least 5 bytes (the prefix
plus the terminator); open_bson_document() rejects a smaller size before plus the terminator); the equality also rejects those impossible sizes,
it is stored, so @a document_size is never negative here. since at least 5 bytes are always consumed.
@param[in] document_start value of chars_read before the size prefix @param[in] document_start value of chars_read before the size prefix
@param[in] document_size the declared document size @param[in] document_size the declared document size
@@ -353,7 +353,7 @@ class binary_reader
*/ */
bool check_bson_document_size(const std::size_t document_start, const std::int32_t document_size) bool check_bson_document_size(const std::size_t document_start, const std::int32_t document_size)
{ {
if (JSON_HEDLEY_UNLIKELY(static_cast<std::size_t>(document_size) != chars_read - document_start)) if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast<std::size_t>(document_size) != chars_read - document_start))
{ {
return bson_document_size_error(document_size, chars_read - document_start); return bson_document_size_error(document_size, chars_read - document_start);
} }
@@ -467,13 +467,11 @@ class binary_reader
const std::size_t element_type_parse_position = chars_read; const std::size_t element_type_parse_position = chars_read;
// an array's elements are named "0", "1", ... in the wire format, // an array's elements are named "0", "1", ... in the wire format,
// and those names are not passed on: they are skipped, and only // and those names are not passed on: they are skipped unless
// checked if error_handler_t::strict must reject ill-formed UTF-8 // error_handler_t::strict must reject ill-formed UTF-8 in them
const bool is_object = container_stack.back().is_object; const bool is_object = container_stack.back().is_object;
key.clear(); key.clear();
if (JSON_HEDLEY_UNLIKELY(!(is_object || error_handler != error_handler_t::strict if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key, is_object || error_handler == error_handler_t::strict)))
? get_bson_cstr(key, is_object)
: check_bson_cstr(key, std::integral_constant<bool, bulk_scan> {}))))
{ {
return false; return false;
} }
@@ -497,7 +495,7 @@ class binary_reader
@param[in] store whether the characters are needed; if not (the names of @param[in] store whether the characters are needed; if not (the names of
an array's elements), they are skipped and @a result may an array's elements), they are skipped and @a result may
be left unchanged, and they are not checked against be left unchanged, and they are not checked against
@ref error_handler (@ref check_bson_cstr checks them) @ref error_handler
@return `true` if the \\x00-byte indicating the end of the string was @return `true` if the \\x00-byte indicating the end of the string was
encountered before the EOF; false` indicates an unexpected EOF. encountered before the EOF; false` indicates an unexpected EOF.
*/ */
@@ -556,42 +554,6 @@ class binary_reader
return true; return true;
} }
/*!
@brief skip a C-style string that error_handler_t::strict must check
The names of an array's elements are not passed on. On contiguous input,
a well-formed name is checked in place and skipped without being copied.
Otherwise (no \\x00-byte in the input that can be scanned, or ill-formed
UTF-8), it is read by @ref get_bson_cstr, which reports the error.
@param[in,out] result an empty string, which may receive the name
@return whether the name was read and is well-formed
*/
bool check_bson_cstr(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining != 0)
{
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = find_zero_byte(data, remaining);
if (length != remaining && JSON_HEDLEY_LIKELY(valid_utf8_string(data, length)))
{
// consume the string and its \x00-byte, as get_bson_cstr does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
}
return get_bson_cstr(result);
}
/// input that is not contiguous: the name is read and checked byte by byte
bool check_bson_cstr(string_t& result, std::false_type /*bulk*/)
{
return get_bson_cstr(result);
}
/// input that is not contiguous: C-style strings are read byte by byte /// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept
{ {
@@ -3084,13 +3046,16 @@ class binary_reader
{ {
if (size_and_type.second != 0) if (size_and_type.second != 0)
{ {
for (std::size_t i = 0; i < size_and_type.first; ++i) if (size_and_type.second != 'N')
{ {
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, size_and_type.second))) for (std::size_t i = 0; i < size_and_type.first; ++i)
{ {
return false; if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, size_and_type.second)))
{
return false;
}
dim.push_back(dimlen);
} }
dim.push_back(dimlen);
} }
} }
else else
@@ -3292,9 +3257,11 @@ class binary_reader
// follow the documented _ArrayType_, _ArraySize_, _ArrayData_ order // follow the documented _ArrayType_, _ArraySize_, _ArrayData_ order
if (ndarray_dtype != 0) if (ndarray_dtype != 0)
{ {
// every optimized type marker is_valid_optimized_type() accepts for BJData has a name
const char* type_name = bjd_type_name(ndarray_dtype); const char* type_name = bjd_type_name(ndarray_dtype);
JSON_ASSERT(type_name != nullptr); if (JSON_HEDLEY_UNLIKELY(type_name == nullptr))
{
return unexpected_byte("invalid byte", "type");
}
string_t type_key = "_ArrayType_"; string_t type_key = "_ArrayType_";
string_t type = type_name; // sax->string() takes a reference string_t type = type_name; // sax->string() takes a reference
@@ -3377,6 +3344,12 @@ class binary_reader
if (current == '$') if (current == '$')
{ {
result.second = get(); // must not ignore 'N', because 'N' maybe the type result.second = get(); // must not ignore 'N', because 'N' maybe the type
if (input_format == input_format_t::bjdata
&& JSON_HEDLEY_UNLIKELY(is_bjd_excluded_optimized_type(result.second)))
{
return optimized_type_error();
}
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type"))) if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type")))
{ {
return false; return false;
@@ -3404,7 +3377,7 @@ class binary_reader
// an ndarray was read here only if the flag flipped; when it was // an ndarray was read here only if the flag flipped; when it was
// seeded true, get_ubjson_size_value() already rejected the nested // seeded true, get_ubjson_size_value() already rejected the nested
// dimension vector // dimension vector
if (is_ndarray && !inside_ndarray) if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
{ {
result.second |= (1 << 8); // use bit 8 to indicate ndarray, all UBJSON and BJData markers should be ASCII letters result.second |= (1 << 8); // use bit 8 to indicate ndarray, all UBJSON and BJData markers should be ASCII letters
} }
@@ -3414,7 +3387,7 @@ class binary_reader
if (current == '#') if (current == '#')
{ {
const bool is_error = get_ubjson_size_value(result.first, is_ndarray); const bool is_error = get_ubjson_size_value(result.first, is_ndarray);
if (is_ndarray && !inside_ndarray) if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
{ {
return last_byte_error(exception_id::unexpected_byte, "ndarray requires both type and size", "size"); return last_byte_error(exception_id::unexpected_byte, "ndarray requires both type and size", "size");
} }
@@ -3588,6 +3561,7 @@ class binary_reader
JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NEVER_INLINE
bool parse_ubjson_typed_run(std::size_t& remaining, const char_int_type type, std::true_type /*bulk*/) bool parse_ubjson_typed_run(std::size_t& remaining, const char_int_type type, std::true_type /*bulk*/)
{ {
const bool bjdata = input_format == input_format_t::bjdata;
switch (type) switch (type)
{ {
case 'U': case 'U':
@@ -3601,11 +3575,11 @@ class binary_reader
case 'L': case 'L':
return parse_ubjson_typed_run<std::int64_t>(remaining); return parse_ubjson_typed_run<std::int64_t>(remaining);
case 'u': case 'u':
return parse_ubjson_typed_run<std::uint16_t>(remaining); return !bjdata || parse_ubjson_typed_run<std::uint16_t>(remaining);
case 'm': case 'm':
return parse_ubjson_typed_run<std::uint32_t>(remaining); return !bjdata || parse_ubjson_typed_run<std::uint32_t>(remaining);
case 'M': case 'M':
return parse_ubjson_typed_run<std::uint64_t>(remaining); return !bjdata || parse_ubjson_typed_run<std::uint64_t>(remaining);
case 'd': case 'd':
return parse_ubjson_typed_run<float>(remaining); return parse_ubjson_typed_run<float>(remaining);
case 'D': case 'D':
@@ -3686,7 +3660,7 @@ class binary_reader
// if bit-8 of size_and_type.second is set to 1, encode bjdata ndarray as an object in JData annotated array format (https://github.com/NeuroJSON/jdata): // if bit-8 of size_and_type.second is set to 1, encode bjdata ndarray as an object in JData annotated array format (https://github.com/NeuroJSON/jdata):
// {"_ArrayType_" : "typeid", "_ArraySize_" : [n1, n2, ...], "_ArrayData_" : [v1, v2, ...]} // {"_ArrayType_" : "typeid", "_ArraySize_" : [n1, n2, ...], "_ArrayData_" : [v1, v2, ...]}
if ((size_and_type.second & (1 << 8)) != 0) if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
{ {
size_and_type.second &= ~(static_cast<char_int_type>(1) << 8); // use bit 8 to indicate ndarray, here we remove the bit to restore the type marker size_and_type.second &= ~(static_cast<char_int_type>(1) << 8); // use bit 8 to indicate ndarray, here we remove the bit to restore the type marker
@@ -3745,7 +3719,7 @@ class binary_reader
} }
// If BJData type marker is 'B' decode as binary // If BJData type marker is 'B' decode as binary
if (size_and_type.first != npos && size_and_type.second == 'B') if (input_format == input_format_t::bjdata && size_and_type.first != npos && size_and_type.second == 'B')
{ {
binary_t result; binary_t result;
return get_binary(size_and_type.first, result) && sax->binary(result); return get_binary(size_and_type.first, result) && sax->binary(result);
@@ -3794,7 +3768,7 @@ class binary_reader
} }
// do not accept ND-array size in objects in BJData // do not accept ND-array size in objects in BJData
if ((size_and_type.second & (1 << 8)) != 0) if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
{ {
return last_byte_error(exception_id::unexpected_byte, "BJData object does not support ND-array size in optimized format", "object"); return last_byte_error(exception_id::unexpected_byte, "BJData object does not support ND-array size in optimized format", "object");
} }
@@ -5075,9 +5049,8 @@ class binary_reader
JSON_PRIVATE_UNLESS_TESTED: JSON_PRIVATE_UNLESS_TESTED:
/*! /*!
@brief check whether @a marker is a type marker that may follow '$' @brief check whether @a marker is a type marker that may follow '$'
UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C. UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C
The markers BJData excludes (F H N S T Z [ {) are the ones (BJData's excluded markers are rejected separately).
binary_writer::is_bjdata_excluded_type_marker() lists; keep the two in sync.
*/ */
bool is_valid_optimized_type(const char_int_type marker) const noexcept bool is_valid_optimized_type(const char_int_type marker) const noexcept
{ {
@@ -5112,6 +5085,20 @@ class binary_reader
} }
} }
/*!
@brief whether @a marker is excluded from BJData's optimized ND-array types
@return whether @a marker is one of 'F', 'H', 'N', 'S', 'T', 'Z', '[', '{'
Mirrors binary_writer's @ref binary_writer::is_bjdata_excluded_type_marker
"is_bjdata_excluded_type_marker()`, which encodes the same list the other
way; keep the two in sync.
*/
static constexpr bool is_bjd_excluded_optimized_type(const char_int_type marker) noexcept
{
return marker == '[' || marker == '{' || marker == 'S' || marker == 'H'
|| marker == 'T' || marker == 'F' || marker == 'N' || marker == 'Z';
}
/*! /*!
@brief look up the ND-array element type name for a BJData dtype marker @brief look up the ND-array element type name for a BJData dtype marker
@return the type name ("uint8", "int8", ...), or nullptr if @a marker does @return the type name ("uint8", "int8", ...), or nullptr if @a marker does
@@ -3129,19 +3129,22 @@ class binary_writer
// negative fixnum // negative fixnum
write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer)); write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
} }
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)()) else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int8_t>::max)())
{ {
// int 8 // int 8
oa.write_character(to_char_type(0xD0)); oa.write_character(to_char_type(0xD0));
write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer)); write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
} }
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)()) else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int16_t>::max)())
{ {
// int 16 // int 16
oa.write_character(to_char_type(0xD1)); oa.write_character(to_char_type(0xD1));
write_number(static_cast<std::int16_t>(j.m_data.m_value.number_integer)); write_number(static_cast<std::int16_t>(j.m_data.m_value.number_integer));
} }
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)()) else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int32_t>::max)())
{ {
// int 32 // int 32
oa.write_character(to_char_type(0xD2)); oa.write_character(to_char_type(0xD2));
@@ -4057,7 +4060,7 @@ class binary_writer
Containers, strings, high-precision numbers, booleans and null cannot be Containers, strings, high-precision numbers, booleans and null cannot be
declared as the single type of an optimized container in BJData; such a declared as the single type of an optimized container in BJData; such a
container is written unoptimized. The reader rejects them with the same container is written unoptimized. The reader rejects them with the same
list (binary_reader::is_valid_optimized_type()). list (binary_reader::is_bjd_excluded_optimized_type()).
*/ */
static constexpr bool is_bjdata_excluded_type_marker(const CharType marker) noexcept static constexpr bool is_bjdata_excluded_type_marker(const CharType marker) noexcept
{ {
+36 -4
View File
@@ -4914,6 +4914,30 @@ public:
return {it, res.second}; return {it, res.second};
} }
private:
/// Sets the parents of a value when it goes out of scope, also when an
/// exception is thrown: a container insertion that throws may already
/// have added elements, whose parents must be set for the value to stay
/// valid (basic exception guarantee). Does nothing without JSON_DIAGNOSTICS.
class set_parents_on_exit
{
public:
explicit set_parents_on_exit(basic_json* v) noexcept : m_value(v) {}
set_parents_on_exit(const set_parents_on_exit&) = delete;
set_parents_on_exit& operator=(const set_parents_on_exit&) = delete;
set_parents_on_exit(set_parents_on_exit&&) = delete;
set_parents_on_exit& operator=(set_parents_on_exit&&) = delete;
~set_parents_on_exit()
{
m_value->set_parents();
}
private:
basic_json* m_value;
};
public:
/// Helper for insertion of an iterator /// Helper for insertion of an iterator
/// @note: This uses std::distance to support GCC 4.8, /// @note: This uses std::distance to support GCC 4.8,
/// see https://github.com/nlohmann/json/pull/1257 /// see https://github.com/nlohmann/json/pull/1257
@@ -4923,6 +4947,7 @@ public:
iterator result(this); iterator result(this);
JSON_ASSERT(m_data.m_value.array != nullptr); JSON_ASSERT(m_data.m_value.array != nullptr);
const set_parents_on_exit parents(this);
auto insert_pos = std::distance(m_data.m_value.array->begin(), pos.m_it.array_iterator); auto insert_pos = std::distance(m_data.m_value.array->begin(), pos.m_it.array_iterator);
m_data.m_value.array->insert(pos.m_it.array_iterator, std::forward<Args>(args)...); m_data.m_value.array->insert(pos.m_it.array_iterator, std::forward<Args>(args)...);
result.m_it.array_iterator = m_data.m_value.array->begin() + insert_pos; result.m_it.array_iterator = m_data.m_value.array->begin() + insert_pos;
@@ -4931,7 +4956,6 @@ public:
// result.m_it.array_iterator = m_data.m_value.array->insert(pos.m_it.array_iterator, cnt, val); // result.m_it.array_iterator = m_data.m_value.array->insert(pos.m_it.array_iterator, cnt, val);
// but the return value of insert is missing in GCC 4.8, so it is written this way instead. // but the return value of insert is missing in GCC 4.8, so it is written this way instead.
set_parents();
return result; return result;
} }
@@ -4951,8 +4975,16 @@ public:
throw_iterator_from_other_value(this); throw_iterator_from_other_value(this);
} }
// insert to array and return iterator // vector::insert has no effects if copying a single element to the
return insert_iterator(pos, val); // end throws; elsewhere, it may already have shifted the elements, so
// copy first and insert by move, which cannot throw but by allocating
if (pos.m_it.array_iterator == m_data.m_value.array->end())
{
return insert_iterator(pos, val);
}
basic_json tmp(val);
return insert_iterator(pos, std::move(tmp));
} }
/// @brief inserts element into array /// @brief inserts element into array
@@ -5084,8 +5116,8 @@ public:
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterators first and last must point to objects", this); detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterators first and last must point to objects", this);
} }
const set_parents_on_exit parents(this);
m_data.m_value.object->insert(first.m_it.object_iterator, last.m_it.object_iterator); m_data.m_value.object->insert(first.m_it.object_iterator, last.m_it.object_iterator);
set_parents();
} }
/// @brief updates a JSON object from another object, overwriting existing keys /// @brief updates a JSON object from another object, overwriting existing keys
+92 -70
View File
@@ -18351,8 +18351,8 @@ class binary_reader
nested document could declare a length that disagrees with where its nested document could declare a length that disagrees with where its
terminator actually falls and quietly hand the bytes in between to the terminator actually falls and quietly hand the bytes in between to the
enclosing document. A well-formed document is at least 5 bytes (the prefix enclosing document. A well-formed document is at least 5 bytes (the prefix
plus the terminator); open_bson_document() rejects a smaller size before plus the terminator); the equality also rejects those impossible sizes,
it is stored, so @a document_size is never negative here. since at least 5 bytes are always consumed.
@param[in] document_start value of chars_read before the size prefix @param[in] document_start value of chars_read before the size prefix
@param[in] document_size the declared document size @param[in] document_size the declared document size
@@ -18360,7 +18360,7 @@ class binary_reader
*/ */
bool check_bson_document_size(const std::size_t document_start, const std::int32_t document_size) bool check_bson_document_size(const std::size_t document_start, const std::int32_t document_size)
{ {
if (JSON_HEDLEY_UNLIKELY(static_cast<std::size_t>(document_size) != chars_read - document_start)) if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast<std::size_t>(document_size) != chars_read - document_start))
{ {
return bson_document_size_error(document_size, chars_read - document_start); return bson_document_size_error(document_size, chars_read - document_start);
} }
@@ -18474,13 +18474,11 @@ class binary_reader
const std::size_t element_type_parse_position = chars_read; const std::size_t element_type_parse_position = chars_read;
// an array's elements are named "0", "1", ... in the wire format, // an array's elements are named "0", "1", ... in the wire format,
// and those names are not passed on: they are skipped, and only // and those names are not passed on: they are skipped unless
// checked if error_handler_t::strict must reject ill-formed UTF-8 // error_handler_t::strict must reject ill-formed UTF-8 in them
const bool is_object = container_stack.back().is_object; const bool is_object = container_stack.back().is_object;
key.clear(); key.clear();
if (JSON_HEDLEY_UNLIKELY(!(is_object || error_handler != error_handler_t::strict if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key, is_object || error_handler == error_handler_t::strict)))
? get_bson_cstr(key, is_object)
: check_bson_cstr(key, std::integral_constant<bool, bulk_scan> {}))))
{ {
return false; return false;
} }
@@ -18504,7 +18502,7 @@ class binary_reader
@param[in] store whether the characters are needed; if not (the names of @param[in] store whether the characters are needed; if not (the names of
an array's elements), they are skipped and @a result may an array's elements), they are skipped and @a result may
be left unchanged, and they are not checked against be left unchanged, and they are not checked against
@ref error_handler (@ref check_bson_cstr checks them) @ref error_handler
@return `true` if the \\x00-byte indicating the end of the string was @return `true` if the \\x00-byte indicating the end of the string was
encountered before the EOF; false` indicates an unexpected EOF. encountered before the EOF; false` indicates an unexpected EOF.
*/ */
@@ -18563,42 +18561,6 @@ class binary_reader
return true; return true;
} }
/*!
@brief skip a C-style string that error_handler_t::strict must check
The names of an array's elements are not passed on. On contiguous input,
a well-formed name is checked in place and skipped without being copied.
Otherwise (no \\x00-byte in the input that can be scanned, or ill-formed
UTF-8), it is read by @ref get_bson_cstr, which reports the error.
@param[in,out] result an empty string, which may receive the name
@return whether the name was read and is well-formed
*/
bool check_bson_cstr(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining != 0)
{
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = find_zero_byte(data, remaining);
if (length != remaining && JSON_HEDLEY_LIKELY(valid_utf8_string(data, length)))
{
// consume the string and its \x00-byte, as get_bson_cstr does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
}
return get_bson_cstr(result);
}
/// input that is not contiguous: the name is read and checked byte by byte
bool check_bson_cstr(string_t& result, std::false_type /*bulk*/)
{
return get_bson_cstr(result);
}
/// input that is not contiguous: C-style strings are read byte by byte /// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept
{ {
@@ -21091,13 +21053,16 @@ class binary_reader
{ {
if (size_and_type.second != 0) if (size_and_type.second != 0)
{ {
for (std::size_t i = 0; i < size_and_type.first; ++i) if (size_and_type.second != 'N')
{ {
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, size_and_type.second))) for (std::size_t i = 0; i < size_and_type.first; ++i)
{ {
return false; if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, size_and_type.second)))
{
return false;
}
dim.push_back(dimlen);
} }
dim.push_back(dimlen);
} }
} }
else else
@@ -21299,9 +21264,11 @@ class binary_reader
// follow the documented _ArrayType_, _ArraySize_, _ArrayData_ order // follow the documented _ArrayType_, _ArraySize_, _ArrayData_ order
if (ndarray_dtype != 0) if (ndarray_dtype != 0)
{ {
// every optimized type marker is_valid_optimized_type() accepts for BJData has a name
const char* type_name = bjd_type_name(ndarray_dtype); const char* type_name = bjd_type_name(ndarray_dtype);
JSON_ASSERT(type_name != nullptr); if (JSON_HEDLEY_UNLIKELY(type_name == nullptr))
{
return unexpected_byte("invalid byte", "type");
}
string_t type_key = "_ArrayType_"; string_t type_key = "_ArrayType_";
string_t type = type_name; // sax->string() takes a reference string_t type = type_name; // sax->string() takes a reference
@@ -21384,6 +21351,12 @@ class binary_reader
if (current == '$') if (current == '$')
{ {
result.second = get(); // must not ignore 'N', because 'N' maybe the type result.second = get(); // must not ignore 'N', because 'N' maybe the type
if (input_format == input_format_t::bjdata
&& JSON_HEDLEY_UNLIKELY(is_bjd_excluded_optimized_type(result.second)))
{
return optimized_type_error();
}
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type"))) if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type")))
{ {
return false; return false;
@@ -21411,7 +21384,7 @@ class binary_reader
// an ndarray was read here only if the flag flipped; when it was // an ndarray was read here only if the flag flipped; when it was
// seeded true, get_ubjson_size_value() already rejected the nested // seeded true, get_ubjson_size_value() already rejected the nested
// dimension vector // dimension vector
if (is_ndarray && !inside_ndarray) if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
{ {
result.second |= (1 << 8); // use bit 8 to indicate ndarray, all UBJSON and BJData markers should be ASCII letters result.second |= (1 << 8); // use bit 8 to indicate ndarray, all UBJSON and BJData markers should be ASCII letters
} }
@@ -21421,7 +21394,7 @@ class binary_reader
if (current == '#') if (current == '#')
{ {
const bool is_error = get_ubjson_size_value(result.first, is_ndarray); const bool is_error = get_ubjson_size_value(result.first, is_ndarray);
if (is_ndarray && !inside_ndarray) if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
{ {
return last_byte_error(exception_id::unexpected_byte, "ndarray requires both type and size", "size"); return last_byte_error(exception_id::unexpected_byte, "ndarray requires both type and size", "size");
} }
@@ -21595,6 +21568,7 @@ class binary_reader
JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NEVER_INLINE
bool parse_ubjson_typed_run(std::size_t& remaining, const char_int_type type, std::true_type /*bulk*/) bool parse_ubjson_typed_run(std::size_t& remaining, const char_int_type type, std::true_type /*bulk*/)
{ {
const bool bjdata = input_format == input_format_t::bjdata;
switch (type) switch (type)
{ {
case 'U': case 'U':
@@ -21608,11 +21582,11 @@ class binary_reader
case 'L': case 'L':
return parse_ubjson_typed_run<std::int64_t>(remaining); return parse_ubjson_typed_run<std::int64_t>(remaining);
case 'u': case 'u':
return parse_ubjson_typed_run<std::uint16_t>(remaining); return !bjdata || parse_ubjson_typed_run<std::uint16_t>(remaining);
case 'm': case 'm':
return parse_ubjson_typed_run<std::uint32_t>(remaining); return !bjdata || parse_ubjson_typed_run<std::uint32_t>(remaining);
case 'M': case 'M':
return parse_ubjson_typed_run<std::uint64_t>(remaining); return !bjdata || parse_ubjson_typed_run<std::uint64_t>(remaining);
case 'd': case 'd':
return parse_ubjson_typed_run<float>(remaining); return parse_ubjson_typed_run<float>(remaining);
case 'D': case 'D':
@@ -21693,7 +21667,7 @@ class binary_reader
// if bit-8 of size_and_type.second is set to 1, encode bjdata ndarray as an object in JData annotated array format (https://github.com/NeuroJSON/jdata): // if bit-8 of size_and_type.second is set to 1, encode bjdata ndarray as an object in JData annotated array format (https://github.com/NeuroJSON/jdata):
// {"_ArrayType_" : "typeid", "_ArraySize_" : [n1, n2, ...], "_ArrayData_" : [v1, v2, ...]} // {"_ArrayType_" : "typeid", "_ArraySize_" : [n1, n2, ...], "_ArrayData_" : [v1, v2, ...]}
if ((size_and_type.second & (1 << 8)) != 0) if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
{ {
size_and_type.second &= ~(static_cast<char_int_type>(1) << 8); // use bit 8 to indicate ndarray, here we remove the bit to restore the type marker size_and_type.second &= ~(static_cast<char_int_type>(1) << 8); // use bit 8 to indicate ndarray, here we remove the bit to restore the type marker
@@ -21752,7 +21726,7 @@ class binary_reader
} }
// If BJData type marker is 'B' decode as binary // If BJData type marker is 'B' decode as binary
if (size_and_type.first != npos && size_and_type.second == 'B') if (input_format == input_format_t::bjdata && size_and_type.first != npos && size_and_type.second == 'B')
{ {
binary_t result; binary_t result;
return get_binary(size_and_type.first, result) && sax->binary(result); return get_binary(size_and_type.first, result) && sax->binary(result);
@@ -21801,7 +21775,7 @@ class binary_reader
} }
// do not accept ND-array size in objects in BJData // do not accept ND-array size in objects in BJData
if ((size_and_type.second & (1 << 8)) != 0) if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
{ {
return last_byte_error(exception_id::unexpected_byte, "BJData object does not support ND-array size in optimized format", "object"); return last_byte_error(exception_id::unexpected_byte, "BJData object does not support ND-array size in optimized format", "object");
} }
@@ -23082,9 +23056,8 @@ class binary_reader
JSON_PRIVATE_UNLESS_TESTED: JSON_PRIVATE_UNLESS_TESTED:
/*! /*!
@brief check whether @a marker is a type marker that may follow '$' @brief check whether @a marker is a type marker that may follow '$'
UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C. UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C
The markers BJData excludes (F H N S T Z [ {) are the ones (BJData's excluded markers are rejected separately).
binary_writer::is_bjdata_excluded_type_marker() lists; keep the two in sync.
*/ */
bool is_valid_optimized_type(const char_int_type marker) const noexcept bool is_valid_optimized_type(const char_int_type marker) const noexcept
{ {
@@ -23119,6 +23092,20 @@ class binary_reader
} }
} }
/*!
@brief whether @a marker is excluded from BJData's optimized ND-array types
@return whether @a marker is one of 'F', 'H', 'N', 'S', 'T', 'Z', '[', '{'
Mirrors binary_writer's @ref binary_writer::is_bjdata_excluded_type_marker
"is_bjdata_excluded_type_marker()`, which encodes the same list the other
way; keep the two in sync.
*/
static constexpr bool is_bjd_excluded_optimized_type(const char_int_type marker) noexcept
{
return marker == '[' || marker == '{' || marker == 'S' || marker == 'H'
|| marker == 'T' || marker == 'F' || marker == 'N' || marker == 'Z';
}
/*! /*!
@brief look up the ND-array element type name for a BJData dtype marker @brief look up the ND-array element type name for a BJData dtype marker
@return the type name ("uint8", "int8", ...), or nullptr if @a marker does @return the type name ("uint8", "int8", ...), or nullptr if @a marker does
@@ -31076,19 +31063,22 @@ class binary_writer
// negative fixnum // negative fixnum
write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer)); write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
} }
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)()) else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int8_t>::max)())
{ {
// int 8 // int 8
oa.write_character(to_char_type(0xD0)); oa.write_character(to_char_type(0xD0));
write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer)); write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
} }
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)()) else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int16_t>::max)())
{ {
// int 16 // int 16
oa.write_character(to_char_type(0xD1)); oa.write_character(to_char_type(0xD1));
write_number(static_cast<std::int16_t>(j.m_data.m_value.number_integer)); write_number(static_cast<std::int16_t>(j.m_data.m_value.number_integer));
} }
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)()) else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int32_t>::max)())
{ {
// int 32 // int 32
oa.write_character(to_char_type(0xD2)); oa.write_character(to_char_type(0xD2));
@@ -32004,7 +31994,7 @@ class binary_writer
Containers, strings, high-precision numbers, booleans and null cannot be Containers, strings, high-precision numbers, booleans and null cannot be
declared as the single type of an optimized container in BJData; such a declared as the single type of an optimized container in BJData; such a
container is written unoptimized. The reader rejects them with the same container is written unoptimized. The reader rejects them with the same
list (binary_reader::is_valid_optimized_type()). list (binary_reader::is_bjd_excluded_optimized_type()).
*/ */
static constexpr bool is_bjdata_excluded_type_marker(const CharType marker) noexcept static constexpr bool is_bjdata_excluded_type_marker(const CharType marker) noexcept
{ {
@@ -42716,6 +42706,30 @@ public:
return {it, res.second}; return {it, res.second};
} }
private:
/// Sets the parents of a value when it goes out of scope, also when an
/// exception is thrown: a container insertion that throws may already
/// have added elements, whose parents must be set for the value to stay
/// valid (basic exception guarantee). Does nothing without JSON_DIAGNOSTICS.
class set_parents_on_exit
{
public:
explicit set_parents_on_exit(basic_json* v) noexcept : m_value(v) {}
set_parents_on_exit(const set_parents_on_exit&) = delete;
set_parents_on_exit& operator=(const set_parents_on_exit&) = delete;
set_parents_on_exit(set_parents_on_exit&&) = delete;
set_parents_on_exit& operator=(set_parents_on_exit&&) = delete;
~set_parents_on_exit()
{
m_value->set_parents();
}
private:
basic_json* m_value;
};
public:
/// Helper for insertion of an iterator /// Helper for insertion of an iterator
/// @note: This uses std::distance to support GCC 4.8, /// @note: This uses std::distance to support GCC 4.8,
/// see https://github.com/nlohmann/json/pull/1257 /// see https://github.com/nlohmann/json/pull/1257
@@ -42725,6 +42739,7 @@ public:
iterator result(this); iterator result(this);
JSON_ASSERT(m_data.m_value.array != nullptr); JSON_ASSERT(m_data.m_value.array != nullptr);
const set_parents_on_exit parents(this);
auto insert_pos = std::distance(m_data.m_value.array->begin(), pos.m_it.array_iterator); auto insert_pos = std::distance(m_data.m_value.array->begin(), pos.m_it.array_iterator);
m_data.m_value.array->insert(pos.m_it.array_iterator, std::forward<Args>(args)...); m_data.m_value.array->insert(pos.m_it.array_iterator, std::forward<Args>(args)...);
result.m_it.array_iterator = m_data.m_value.array->begin() + insert_pos; result.m_it.array_iterator = m_data.m_value.array->begin() + insert_pos;
@@ -42733,7 +42748,6 @@ public:
// result.m_it.array_iterator = m_data.m_value.array->insert(pos.m_it.array_iterator, cnt, val); // result.m_it.array_iterator = m_data.m_value.array->insert(pos.m_it.array_iterator, cnt, val);
// but the return value of insert is missing in GCC 4.8, so it is written this way instead. // but the return value of insert is missing in GCC 4.8, so it is written this way instead.
set_parents();
return result; return result;
} }
@@ -42753,8 +42767,16 @@ public:
throw_iterator_from_other_value(this); throw_iterator_from_other_value(this);
} }
// insert to array and return iterator // vector::insert has no effects if copying a single element to the
return insert_iterator(pos, val); // end throws; elsewhere, it may already have shifted the elements, so
// copy first and insert by move, which cannot throw but by allocating
if (pos.m_it.array_iterator == m_data.m_value.array->end())
{
return insert_iterator(pos, val);
}
basic_json tmp(val);
return insert_iterator(pos, std::move(tmp));
} }
/// @brief inserts element into array /// @brief inserts element into array
@@ -42886,8 +42908,8 @@ public:
detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterators first and last must point to objects", this); detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterators first and last must point to objects", this);
} }
const set_parents_on_exit parents(this);
m_data.m_value.object->insert(first.m_it.object_iterator, last.m_it.object_iterator); m_data.m_value.object->insert(first.m_it.object_iterator, last.m_it.object_iterator);
set_parents();
} }
/// @brief updates a JSON object from another object, overwriting existing keys /// @brief updates a JSON object from another object, overwriting existing keys
+220
View File
@@ -12,6 +12,7 @@
#include <nlohmann/json.hpp> #include <nlohmann/json.hpp>
using nlohmann::json; using nlohmann::json;
#include <functional>
#include <valarray> #include <valarray>
#if JSON_HAS_RANGES #if JSON_HAS_RANGES
#include <ranges> #include <ranges>
@@ -804,3 +805,222 @@ TEST_CASE("a failed allocation leaves the value unchanged")
#endif #endif
} }
#endif #endif
// the no-exceptions CI job has no std::bad_alloc to catch; with iterator
// debugging (MSVC STL debug builds) containers allocate inside noexcept
// functions, so failing an allocation there terminates the program
#if !defined(JSON_NOEXCEPTION) && !(defined(_ITERATOR_DEBUG_LEVEL) && _ITERATOR_DEBUG_LEVEL > 0)
namespace
{
// whether every child of j names j as its parent (always true without
// JSON_DIAGNOSTICS, where no parents are stored)
template<class BasicJsonType>
bool children_name_their_parent(const BasicJsonType& j)
{
#if JSON_DIAGNOSTICS
if (j.is_array())
{
for (const auto& element : *j.m_data.m_value.array)
{
if (element.m_parent != &j)
{
return false;
}
}
}
else if (j.is_object())
{
for (const auto& element : *j.m_data.m_value.object)
{
if (element.second.m_parent != &j)
{
return false;
}
}
}
#else
static_cast<void>(j);
#endif
return true;
}
enum class guarantee
{
basic, // the value stays valid and keeps its type
strong // the value is unchanged
};
// Runs op on a fresh value make() with the 0th, 1st, 2nd, ... allocation made
// by the library failing in turn, until op completes. Every failed attempt
// must have thrown std::bad_alloc and left the value valid, with all parent
// pointers intact (copying a value asserts the invariant), and for the strong
// guarantee exactly as it was.
template<class BasicJsonType>
void check_exception_guarantee(guarantee g,
const std::function<BasicJsonType()>& make,
const std::function<void(BasicJsonType&)>& op)
{
fail_at_alloc_call = -1;
bool completed = false;
std::size_t failures = 0;
for (std::size_t n = 0; !completed; ++n)
{
CAPTURE(n)
REQUIRE(n < 1000);
BasicJsonType j = make();
const std::string before = j.dump();
const auto type_before = j.type();
alloc_call_count = 0;
fail_at_alloc_call = static_cast<long>(n);
try
{
op(j);
completed = true;
}
catch (const std::bad_alloc&)
{
++failures;
}
fail_at_alloc_call = -1;
if (!completed)
{
CHECK((j.type() == type_before));
CHECK(children_name_their_parent(j));
const BasicJsonType copy(j); // NOLINT(performance-unnecessary-copy-initialization): the copy is the point, it allocates
CHECK(copy.dump() == j.dump());
if (g == guarantee::strong)
{
CHECK(j.dump() == before);
}
}
}
// the operation allocates, so at least one attempt must have failed
CHECK(failures > 0);
}
template<class BasicJsonType>
void check_insert_exception_guarantees()
{
using json_t = BasicJsonType;
using make_t = std::function<json_t()>;
// strings of this length do not fit into the small string buffer, so
// copying them allocates
const json_t val = std::string(64, 'x');
const json_t source = json_t::array({std::string(64, 'p'), std::string(64, 'q'), std::string(64, 'r')});
for (const bool spare_capacity :
{
false, true
})
{
CAPTURE(spare_capacity)
// [ "aaa...", "bbb...", "ccc..." ], with a spare slot if requested
const make_t make = [spare_capacity]()
{
json_t j = json_t::array({std::string(64, 'a'), std::string(64, 'b'), std::string(64, 'c')});
if (spare_capacity)
{
j.push_back(std::string(64, 'd'));
j.erase(3);
}
return json_t(std::move(j));
};
for (const bool at_end :
{
false, true
})
{
CAPTURE(at_end)
SECTION("insert(pos, const basic_json&)")
{
check_exception_guarantee<json_t>(guarantee::strong, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, val);
});
}
SECTION("insert(pos, basic_json&&)")
{
check_exception_guarantee<json_t>(guarantee::strong, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, json_t(val));
});
}
SECTION("insert(pos, size_type, const basic_json&)")
{
check_exception_guarantee<json_t>(guarantee::basic, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, 3, val);
});
}
SECTION("insert(pos, const_iterator, const_iterator)")
{
check_exception_guarantee<json_t>(guarantee::basic, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, source.begin(), source.end());
});
}
SECTION("insert(pos, initializer_list_t)")
{
check_exception_guarantee<json_t>(guarantee::basic, make, [&](json_t& j)
{
j.insert(at_end ? j.end() : j.begin() + 1, {val, val});
});
}
}
}
SECTION("insert(const_iterator, const_iterator) for objects")
{
const json_t members = {{"m", std::string(64, 'm')}, {"n", std::string(64, 'n')}, {"o", std::string(64, 'o')}};
SECTION("no member exists yet")
{
check_exception_guarantee<json_t>(guarantee::basic, []()
{
return json_t({{"a", 1}});
}, [&](json_t& j)
{
j.insert(members.begin(), members.end());
});
}
SECTION("some members exist already")
{
check_exception_guarantee<json_t>(guarantee::basic, []()
{
return json_t({{"a", 1}, {"n", std::string(64, 'x')}});
}, [&](json_t& j)
{
j.insert(members.begin(), members.end());
});
}
}
}
} // namespace
TEST_CASE("insert() provides its documented exception guarantees (#5986)")
{
SECTION("std::map-backed object_t")
{
check_insert_exception_guarantees<nlohmann::json::with_allocator_t<nth_alloc_fails_allocator>>();
}
SECTION("ordered_map-backed object_t")
{
check_insert_exception_guarantees<nlohmann::ordered_json::with_allocator_t<nth_alloc_fails_allocator>>();
}
}
#endif
+7 -9
View File
@@ -78,25 +78,23 @@ TEST_CASE("BJData")
{ {
SECTION("binary_reader BJData lookup tables") SECTION("binary_reader BJData lookup tables")
{ {
// bjd_type_name() is static; is_valid_optimized_type() needs a BJData reader // both lookups are static member functions
std::vector<std::uint8_t> const data; std::vector<std::uint8_t> const data;
using reader_t = nlohmann::detail::binary_reader<json, decltype(nlohmann::detail::input_adapter(data))>; using reader_t = nlohmann::detail::binary_reader<json, decltype(nlohmann::detail::input_adapter(data))>;
// the markers BJData excludes as optimized types must match // the excluded optimized-type markers must match binary_writer's
// binary_writer's is_bjdata_excluded_type_marker(), which encodes the // is_bjdata_excluded_type_marker(), which encodes the same 8 markers
// same 8 markers
reader_t reader(nlohmann::detail::input_adapter(data), json::input_format_t::bjdata);
for (const char marker : for (const char marker :
{'[', '{', 'S', 'H', 'T', 'F', 'N', 'Z', 'x' {'[', '{', 'S', 'H', 'T', 'F', 'N', 'Z'
}) })
{ {
CHECK(!reader.is_valid_optimized_type(static_cast<unsigned char>(marker))); CHECK(reader_t::is_bjd_excluded_optimized_type(static_cast<unsigned char>(marker)));
} }
for (const char marker : for (const char marker :
{'U', 'i', 'u', 'I', 'm', 'l', 'M', 'L', 'd', 'D', 'C', 'B', 'h' {'U', 'i', 'u', 'I', 'm', 'l', 'M', 'L', 'd', 'D', 'C', 'B', 'x'
}) })
{ {
CHECK(reader.is_valid_optimized_type(static_cast<unsigned char>(marker))); CHECK(!reader_t::is_bjd_excluded_optimized_type(static_cast<unsigned char>(marker)));
} }
// every dtype marker must round-trip to its ND-array type name // every dtype marker must round-trip to its ND-array type name
-38
View File
@@ -23,7 +23,6 @@ using nlohmann::json;
#include <cstdint> #include <cstdint>
#include <fstream> #include <fstream>
#include <limits> #include <limits>
#include <list>
#include <sstream> #include <sstream>
#include <vector> #include <vector>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
@@ -1200,31 +1199,6 @@ TEST_CASE("BSON keys from contiguous and stream input")
from_stream = e.what(); from_stream = e.what();
} }
CHECK(from_vector == from_stream); CHECK(from_vector == from_stream);
// error_handler_t::strict checks the array element names in place on
// contiguous input and byte by byte otherwise; the names are
// well-formed, so both give the same value or error as above
std::string strict_from_vector;
std::string strict_from_list;
try
{
strict_from_vector = json::from_bson(input, true, true, json::error_handler_t::strict).dump();
}
catch (const json::parse_error& e)
{
strict_from_vector = e.what();
}
try
{
const std::list<std::uint8_t> list(input.begin(), input.end());
strict_from_list = json::from_bson(list.begin(), list.end(), true, true, json::error_handler_t::strict).dump();
}
catch (const json::parse_error& e)
{
strict_from_list = e.what();
}
CHECK(strict_from_vector == from_vector);
CHECK(strict_from_list == from_vector);
} }
} }
@@ -1253,18 +1227,6 @@ TEST_CASE("BSON array element names")
std::istringstream stream(std::string(reinterpret_cast<const char*>(bson.data()), bson.size())); std::istringstream stream(std::string(reinterpret_cast<const char*>(bson.data()), bson.size()));
CHECK(json::from_bson(stream) == expected); CHECK(json::from_bson(stream) == expected);
// input that is not contiguous reads the names byte by byte and must reject
// the same name
const std::list<std::uint8_t> list(bson.begin(), bson.end());
CHECK_THROWS_WITH_AS(_ = json::from_bson(list.begin(), list.end(), true, true, json::error_handler_t::strict), "[json.exception.parse_error.113] parse error at byte 14: syntax error while parsing BSON key: invalid string: ill-formed UTF-8 byte", json::parse_error&);
// without exceptions, the rejected name ends the parse
CHECK(json::from_bson(bson, true, false, json::error_handler_t::strict).is_discarded());
// well-formed names pass the check without being stored
const json valid = {{"a", {true, false, {{"k", json::array({1, 2})}}}}};
CHECK(json::from_bson(json::to_bson(valid), true, true, json::error_handler_t::strict) == valid);
} }
#endif #endif