Compare commits

..
Author SHA1 Message Date
Niels Lohmann 16ed40b2d4 CBOR/MessagePack readers: leave one-element arrays to the general path
enter_cbor_array and enter_msgpack_array called the out-of-line scalar
run for every non-empty array. When the array's element is an object or
an array, as on every level of [[...]] or the single ring of a GeoJSON
polygon, the call reads nothing; it made 1,000 arrays nested in each
other cost 4 % more instructions (GCC 13) and up to 8 % more time (Apple
clang) than without the run. #5975 had fixed that with an inline
pre-check, which was later dropped after measuring canada only.

The run is now entered only for arrays of two or more elements: the same
comparison as before, so no other row changes. Arrays of one scalar,
which the run read faster, now take the general path; they are rare in
documents (one in citm_catalog, none in twitter or canada). The new
benchmark FromBinaryShape/singles measures them, next to
FromBinaryShape/nested.

Instructions per repetition, new/old, GCC 13 callgrind: nested arrays
0.973 (CBOR) and 0.971 (MessagePack), arrays of one scalar 1.037 and
1.021, every other row within 0.15 %.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 08:22:32 +01:00
Niels Lohmann b2674a4a2f BSON reader: check array element names in place in strict mode (#5984) 2026-11-01 21:03:35 +01:00
Niels Lohmann e2423a1857 CI: pin ubuntu-latest jobs to ubuntu-26.04 (#5983)
GitHub is moving the ubuntu-latest label from Ubuntu 24.04 to 26.04
between October 19 and November 19, 2026 (actions/runner-images#14748).
Pin every ubuntu-latest job to ubuntu-26.04 so the switch does not
change the toolchain under our jobs, and so the behaviour does not
depend on when GitHub reaches our runs.

Jobs pinned to ubuntu-24.04 and ubuntu-24.04-arm are unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 20:43:07 +01:00
Niels Lohmann 67b103e506 Fix clang-tidy errors in json_view tests on develop (#5989)
readability-math-missing-parentheses in unit-json_view_builder.cpp (#5974)
and performance-inefficient-string-concatenation in
unit-json_view_positions.cpp.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 20:42:58 +01:00
25 changed files with 251 additions and 224 deletions

No files matched your search

+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
permissions: permissions:
actions: write actions: write
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
+1 -1
View File
@@ -19,7 +19,7 @@ permissions:
jobs: jobs:
save: save:
if: github.event_name == 'pull_request' if: github.event_name == 'pull_request'
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2 uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -16,7 +16,7 @@ concurrency:
jobs: jobs:
check_docs_links: check_docs_links:
if: github.repository == 'nlohmann/json' if: github.repository == 'nlohmann/json'
runs-on: ubuntu-latest runs-on: ubuntu-26.04
timeout-minutes: 30 timeout-minutes: 30
steps: steps:
- name: Harden Runner - name: Harden Runner
@@ -12,7 +12,7 @@ jobs:
comment: comment:
# push runs on develop have no PR to comment on (and no "pr" artifact) # push runs on develop have no PR to comment on (and no "pr" artifact)
if: ${{ github.event.workflow_run.conclusion == 'failure' && github.event.workflow_run.event == 'pull_request' }} if: ${{ github.event.workflow_run.conclusion == 'failure' && github.event.workflow_run.event == 'pull_request' }}
runs-on: ubuntu-latest runs-on: ubuntu-26.04
permissions: permissions:
contents: read contents: read
actions: read actions: read
+1 -1
View File
@@ -18,7 +18,7 @@ permissions:
jobs: jobs:
dependency-review: dependency-review:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2 uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -24,7 +24,7 @@ on:
jobs: jobs:
flawfinder: flawfinder:
name: Flawfinder name: Flawfinder
runs-on: ubuntu-latest runs-on: ubuntu-26.04
permissions: permissions:
actions: read actions: read
contents: read contents: read
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
contents: read contents: read
pull-requests: write pull-requests: write
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
+1 -1
View File
@@ -34,7 +34,7 @@ jobs:
contents: write contents: write
if: github.repository == 'nlohmann/json' if: github.repository == 'nlohmann/json'
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2 uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -24,7 +24,7 @@ permissions:
jobs: jobs:
analysis: analysis:
name: Scorecard analysis name: Scorecard analysis
runs-on: ubuntu-latest runs-on: ubuntu-26.04
permissions: permissions:
# Needed to upload the results to code-scanning dashboard. # Needed to upload the results to code-scanning dashboard.
security-events: write security-events: write
+1 -1
View File
@@ -8,7 +8,7 @@ permissions:
jobs: jobs:
stale: stale:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
permissions: permissions:
issues: write issues: write
+13 -13
View File
@@ -18,7 +18,7 @@ concurrency:
jobs: jobs:
ci_test_gcc: ci_test_gcc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: gcc:latest container: gcc:latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -130,7 +130,7 @@ jobs:
run: cmake --build build --target ${{ matrix.target }} run: cmake --build build --target ${{ matrix.target }}
ci_static_analysis_clang: ci_static_analysis_clang:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: silkeh/clang:22 container: silkeh/clang:22
strategy: strategy:
matrix: matrix:
@@ -149,7 +149,7 @@ jobs:
run: cmake --build build --target ${{ matrix.target }} run: cmake --build build --target ${{ matrix.target }}
ci_cmake_options: ci_cmake_options:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: ubuntu:24.04 container: ubuntu:24.04
strategy: strategy:
matrix: matrix:
@@ -209,7 +209,7 @@ jobs:
fail-on-error: false fail-on-error: false
ci_test_compilers_gcc_old: ci_test_compilers_gcc_old:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
strategy: strategy:
matrix: matrix:
compiler: ['4.8', '4.9', '5', '6'] compiler: ['4.8', '4.9', '5', '6']
@@ -251,7 +251,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }} run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }}
ci_test_compilers_gcc: ci_test_compilers_gcc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
strategy: strategy:
matrix: matrix:
# older GCC docker images (4, 5, 6) fail to check out code # older GCC docker images (4, 5, 6) fail to check out code
@@ -269,7 +269,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_default run: cmake --build build --target ci_test_compiler_default
ci_test_compilers_clang: ci_test_compilers_clang:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
strategy: strategy:
matrix: matrix:
compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', '21', '22', 'latest'] compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', '21', '22', 'latest']
@@ -289,7 +289,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_default run: cmake --build build --target ci_test_compiler_default
ci_test_standards_gcc: ci_test_standards_gcc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: gcc:latest container: gcc:latest
strategy: strategy:
matrix: matrix:
@@ -306,7 +306,7 @@ jobs:
run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }} run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }}
ci_test_standards_clang: ci_test_standards_clang:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: silkeh/clang:latest container: silkeh/clang:latest
strategy: strategy:
matrix: matrix:
@@ -330,7 +330,7 @@ jobs:
if: ${{ matrix.stdlib == 'libstdcxx' }} if: ${{ matrix.stdlib == 'libstdcxx' }}
ci_cuda_example: ci_cuda_example:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -356,7 +356,7 @@ jobs:
strategy: strategy:
matrix: matrix:
container: ['gcc:latest', 'silkeh/clang:latest'] container: ['gcc:latest', 'silkeh/clang:latest']
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: ${{ matrix.container }} container: ${{ matrix.container }}
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -383,7 +383,7 @@ jobs:
run: cmake --build build --target ci_module_cpp20 run: cmake --build build --target ci_module_cpp20
ci_icpc: ci_icpc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
# Intel discontinued the classic icc/icpc compiler in oneAPI 2024.0; this is # Intel discontinued the classic icc/icpc compiler in oneAPI 2024.0; this is
# Intel's own last officially published image that still includes it. # Intel's own last officially published image that still includes it.
container: intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04 container: intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04
@@ -402,7 +402,7 @@ jobs:
run: cmake --build build --target ci_icpc run: cmake --build build --target ci_icpc
ci_icpx: ci_icpx:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: intel/oneapi-hpckit:latest container: intel/oneapi-hpckit:latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -416,7 +416,7 @@ jobs:
run: cmake --build build --target ci_icpx run: cmake --build build --target ci_icpx
ci_nvhpc: ci_nvhpc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04 container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-10
View File
@@ -120,16 +120,6 @@ overload (3).
See [Number conversion](../../features/types/number_handling.md#number-conversion) See [Number conversion](../../features/types/number_handling.md#number-conversion)
for more information. for more information.
!!! note "Booleans as numbers"
A JSON boolean converts to `#!cpp 0` or `#!cpp 1` for every arithmetic type **except** [`number_integer_t`](number_integer_t.md),
[`number_unsigned_t`](number_unsigned_t.md) and [`number_float_t`](number_float_t.md) (by default `#!cpp std::int64_t`,
`#!cpp std::uint64_t`, and `#!cpp double`), for which `#!cpp get<T>()` throws
[`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) ("type must be number, but is boolean"). So
`#!cpp json(true).get<int>()` is `#!cpp 1`, while `#!cpp json(true).get<std::int64_t>()` and
`#!cpp json(true).get<double>()` throw. This is the behavior of all 3.x versions;
[`basic_json_view::get<T>()`](../basic_json_view/get.md) behaves the same.
!!! note "`std::optional` conversions" !!! note "`std::optional` conversions"
Prior to version 3.13.0, `#!cpp get<std::optional<T>>()` (and other conversions to `std::optional<T>`) failed to Prior to version 3.13.0, `#!cpp get<std::optional<T>>()` (and other conversions to `std::optional<T>`) failed to
+1 -2
View File
@@ -11,8 +11,7 @@ For the types below, the conversion works directly on the flat index — no `Bas
- `#!cpp bool` - `#!cpp bool`
- arithmetic types other than `#!cpp bool` (for a boolean, `#!cpp 0`/`#!cpp 1`, exactly as for - arithmetic types other than `#!cpp bool` (for a boolean, `#!cpp 0`/`#!cpp 1`, exactly as for
[`BasicJsonType::get<T>()`](../basic_json/get.md#notes), except for `number_integer_t`, `number_unsigned_t` and [`BasicJsonType::get<T>()`](../basic_json/get.md))
`number_float_t`, which throw `type_error.302` for a boolean, as `BasicJsonType::get<T>()` does)
- `#!cpp std::nullptr_t` - `#!cpp std::nullptr_t`
- `#!cpp std::basic_string<char, Traits, Alloc>` (including `string_t`) — a copy of the string - `#!cpp std::basic_string<char, Traits, Alloc>` (including `string_t`) — a copy of the string
- [`string_view_t`](index.md#member-types) — **no copy**: the returned view points into the document's - [`string_view_t`](index.md#member-types) — **no copy**: the returned view points into the document's
@@ -467,11 +467,13 @@ class binary_reader
const std::size_t element_type_parse_position = chars_read; const std::size_t element_type_parse_position = chars_read;
// an array's elements are named "0", "1", ... in the wire format, // an array's elements are named "0", "1", ... in the wire format,
// and those names are not passed on: they are skipped unless // and those names are not passed on: they are skipped, and only
// error_handler_t::strict must reject ill-formed UTF-8 in them // checked if error_handler_t::strict must reject ill-formed UTF-8
const bool is_object = container_stack.back().is_object; const bool is_object = container_stack.back().is_object;
key.clear(); key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key, is_object || error_handler == error_handler_t::strict))) if (JSON_HEDLEY_UNLIKELY(!(is_object || error_handler != error_handler_t::strict
? get_bson_cstr(key, is_object)
: check_bson_cstr(key, std::integral_constant<bool, bulk_scan> {}))))
{ {
return false; return false;
} }
@@ -495,7 +497,7 @@ class binary_reader
@param[in] store whether the characters are needed; if not (the names of @param[in] store whether the characters are needed; if not (the names of
an array's elements), they are skipped and @a result may an array's elements), they are skipped and @a result may
be left unchanged, and they are not checked against be left unchanged, and they are not checked against
@ref error_handler @ref error_handler (@ref check_bson_cstr checks them)
@return `true` if the \\x00-byte indicating the end of the string was @return `true` if the \\x00-byte indicating the end of the string was
encountered before the EOF; false` indicates an unexpected EOF. encountered before the EOF; false` indicates an unexpected EOF.
*/ */
@@ -554,6 +556,42 @@ class binary_reader
return true; return true;
} }
/*!
@brief skip a C-style string that error_handler_t::strict must check
The names of an array's elements are not passed on. On contiguous input,
a well-formed name is checked in place and skipped without being copied.
Otherwise (no \\x00-byte in the input that can be scanned, or ill-formed
UTF-8), it is read by @ref get_bson_cstr, which reports the error.
@param[in,out] result an empty string, which may receive the name
@return whether the name was read and is well-formed
*/
bool check_bson_cstr(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining != 0)
{
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = find_zero_byte(data, remaining);
if (length != remaining && JSON_HEDLEY_LIKELY(valid_utf8_string(data, length)))
{
// consume the string and its \x00-byte, as get_bson_cstr does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
}
return get_bson_cstr(result);
}
/// input that is not contiguous: the name is read and checked byte by byte
bool check_bson_cstr(string_t& result, std::false_type /*bulk*/)
{
return get_bson_cstr(result);
}
/// input that is not contiguous: C-style strings are read byte by byte /// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept
{ {
@@ -1818,14 +1856,18 @@ class binary_reader
Only the start of an array is tried, which covers arrays of numbers Only the start of an array is tried, which covers arrays of numbers
(e.g. coordinates) but costs arrays of other values only one call, not one (e.g. coordinates) but costs arrays of other values only one call, not one
per element. per element. An array of one element is not tried at all: that element is
mostly an object or an array (`[[...]]`, the single ring of a GeoJSON
polygon), for which the call would find no scalar, while a single scalar,
which the run would read faster, is rare. The benchmarks
`FromBinaryShape/nested` and `FromBinaryShape/singles` show both sides.
@param[in] len the number of elements @param[in] len the number of elements
@return whether the SAX parser accepted the array and the values read @return whether the SAX parser accepted the array and the values read
*/ */
bool enter_cbor_array(const std::size_t len) bool enter_cbor_array(const std::size_t len)
{ {
return enter_array(len) && (len == 0 || parse_cbor_scalar_run(std::integral_constant<bool, bulk_scan> {})); return enter_array(len) && (len <= 1 || parse_cbor_scalar_run(std::integral_constant<bool, bulk_scan> {}));
} }
/// input that is not contiguous: every value is read by parse_cbor_value /// input that is not contiguous: every value is read by parse_cbor_value
@@ -2726,12 +2768,16 @@ class binary_reader
/*! /*!
@brief open a MessagePack array and read the scalars it begins with (see @brief open a MessagePack array and read the scalars it begins with (see
@ref parse_msgpack_scalar_run) @ref parse_msgpack_scalar_run)
An array of one element is not tried, for the reason given at
@ref enter_cbor_array.
@param[in] len the number of elements @param[in] len the number of elements
@return whether the SAX parser accepted the array and the values read @return whether the SAX parser accepted the array and the values read
*/ */
bool enter_msgpack_array(const std::size_t len) bool enter_msgpack_array(const std::size_t len)
{ {
return enter_array(len) && (len == 0 || parse_msgpack_scalar_run(std::integral_constant<bool, bulk_scan> {})); return enter_array(len) && (len <= 1 || parse_msgpack_scalar_run(std::integral_constant<bool, bulk_scan> {}));
} }
/// input that is not contiguous: every value is read by parse_msgpack_value /// input that is not contiguous: every value is read by parse_msgpack_value
+3 -28
View File
@@ -33,37 +33,12 @@ namespace view
template<typename T> template<typename T>
struct value_tag {}; struct value_tag {};
/// whether basic_json reads T with a dedicated from_json() overload that does
/// not accept a boolean: exactly number_integer_t, number_unsigned_t and
/// number_float_t (the other arithmetic types convert a boolean to 0 or 1)
template<typename T, typename BasicJsonType>
struct rejects_boolean : std::integral_constant < bool,
std::is_same<T, typename BasicJsonType::number_integer_t>::value
|| std::is_same<T, typename BasicJsonType::number_unsigned_t>::value
|| std::is_same<T, typename BasicJsonType::number_float_t>::value >
{};
/// a boolean as T: 0 or 1
template<typename T>
NLOHMANN_VIEW_ALWAYS_INLINE T boolean_as(const document_data& /*d*/, const node& n, std::false_type /*rejects*/)
{
return static_cast<T>((n.flags & node_flags::is_true) != 0);
}
/// a boolean as number_integer_t, number_unsigned_t or number_float_t: type_error.302, as in basic_json
template<typename T>
[[noreturn]] NLOHMANN_VIEW_ALWAYS_INLINE T boolean_as(const document_data& d, const node& n, std::true_type /*rejects*/)
{
throw_type_error(error_site(&d, &n), exception_id::type_mismatch, "type must be number, but is ", value_type_name(static_cast<value_t>(n.kind)));
}
/*! /*!
@brief the number or boolean of a node converted to an arithmetic type @brief the number or boolean of a node converted to an arithmetic type
As basic_json's get<T>() for arithmetic types: integers and floats are As basic_json's get<T>() for arithmetic types: integers and floats are
converted with static_cast, booleans give 0 or 1 (but throw type_error.302 for converted with static_cast, booleans give 0 or 1, and other types throw
number_integer_t, number_unsigned_t and number_float_t, as basic_json does), type_error.302.
and other types throw type_error.302.
*/ */
template<typename T, typename BasicJsonType> template<typename T, typename BasicJsonType>
NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const node& n) NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const node& n)
@@ -77,7 +52,7 @@ NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const nod
case value_t::number_float: case value_t::number_float:
return static_cast<T>(float_value<typename BasicJsonType::number_float_t>(d, n)); return static_cast<T>(float_value<typename BasicJsonType::number_float_t>(d, n));
case value_t::boolean: case value_t::boolean:
return boolean_as<T>(d, n, rejects_boolean<T, BasicJsonType> {}); return static_cast<T>((n.flags & node_flags::is_true) != 0);
case value_t::null: case value_t::null:
case value_t::object: case value_t::object:
case value_t::array: case value_t::array:
+53 -7
View File
@@ -18474,11 +18474,13 @@ class binary_reader
const std::size_t element_type_parse_position = chars_read; const std::size_t element_type_parse_position = chars_read;
// an array's elements are named "0", "1", ... in the wire format, // an array's elements are named "0", "1", ... in the wire format,
// and those names are not passed on: they are skipped unless // and those names are not passed on: they are skipped, and only
// error_handler_t::strict must reject ill-formed UTF-8 in them // checked if error_handler_t::strict must reject ill-formed UTF-8
const bool is_object = container_stack.back().is_object; const bool is_object = container_stack.back().is_object;
key.clear(); key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key, is_object || error_handler == error_handler_t::strict))) if (JSON_HEDLEY_UNLIKELY(!(is_object || error_handler != error_handler_t::strict
? get_bson_cstr(key, is_object)
: check_bson_cstr(key, std::integral_constant<bool, bulk_scan> {}))))
{ {
return false; return false;
} }
@@ -18502,7 +18504,7 @@ class binary_reader
@param[in] store whether the characters are needed; if not (the names of @param[in] store whether the characters are needed; if not (the names of
an array's elements), they are skipped and @a result may an array's elements), they are skipped and @a result may
be left unchanged, and they are not checked against be left unchanged, and they are not checked against
@ref error_handler @ref error_handler (@ref check_bson_cstr checks them)
@return `true` if the \\x00-byte indicating the end of the string was @return `true` if the \\x00-byte indicating the end of the string was
encountered before the EOF; false` indicates an unexpected EOF. encountered before the EOF; false` indicates an unexpected EOF.
*/ */
@@ -18561,6 +18563,42 @@ class binary_reader
return true; return true;
} }
/*!
@brief skip a C-style string that error_handler_t::strict must check
The names of an array's elements are not passed on. On contiguous input,
a well-formed name is checked in place and skipped without being copied.
Otherwise (no \\x00-byte in the input that can be scanned, or ill-formed
UTF-8), it is read by @ref get_bson_cstr, which reports the error.
@param[in,out] result an empty string, which may receive the name
@return whether the name was read and is well-formed
*/
bool check_bson_cstr(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining != 0)
{
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = find_zero_byte(data, remaining);
if (length != remaining && JSON_HEDLEY_LIKELY(valid_utf8_string(data, length)))
{
// consume the string and its \x00-byte, as get_bson_cstr does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
}
return get_bson_cstr(result);
}
/// input that is not contiguous: the name is read and checked byte by byte
bool check_bson_cstr(string_t& result, std::false_type /*bulk*/)
{
return get_bson_cstr(result);
}
/// input that is not contiguous: C-style strings are read byte by byte /// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept
{ {
@@ -19825,14 +19863,18 @@ class binary_reader
Only the start of an array is tried, which covers arrays of numbers Only the start of an array is tried, which covers arrays of numbers
(e.g. coordinates) but costs arrays of other values only one call, not one (e.g. coordinates) but costs arrays of other values only one call, not one
per element. per element. An array of one element is not tried at all: that element is
mostly an object or an array (`[[...]]`, the single ring of a GeoJSON
polygon), for which the call would find no scalar, while a single scalar,
which the run would read faster, is rare. The benchmarks
`FromBinaryShape/nested` and `FromBinaryShape/singles` show both sides.
@param[in] len the number of elements @param[in] len the number of elements
@return whether the SAX parser accepted the array and the values read @return whether the SAX parser accepted the array and the values read
*/ */
bool enter_cbor_array(const std::size_t len) bool enter_cbor_array(const std::size_t len)
{ {
return enter_array(len) && (len == 0 || parse_cbor_scalar_run(std::integral_constant<bool, bulk_scan> {})); return enter_array(len) && (len <= 1 || parse_cbor_scalar_run(std::integral_constant<bool, bulk_scan> {}));
} }
/// input that is not contiguous: every value is read by parse_cbor_value /// input that is not contiguous: every value is read by parse_cbor_value
@@ -20733,12 +20775,16 @@ class binary_reader
/*! /*!
@brief open a MessagePack array and read the scalars it begins with (see @brief open a MessagePack array and read the scalars it begins with (see
@ref parse_msgpack_scalar_run) @ref parse_msgpack_scalar_run)
An array of one element is not tried, for the reason given at
@ref enter_cbor_array.
@param[in] len the number of elements @param[in] len the number of elements
@return whether the SAX parser accepted the array and the values read @return whether the SAX parser accepted the array and the values read
*/ */
bool enter_msgpack_array(const std::size_t len) bool enter_msgpack_array(const std::size_t len)
{ {
return enter_array(len) && (len == 0 || parse_msgpack_scalar_run(std::integral_constant<bool, bulk_scan> {})); return enter_array(len) && (len <= 1 || parse_msgpack_scalar_run(std::integral_constant<bool, bulk_scan> {}));
} }
/// input that is not contiguous: every value is read by parse_msgpack_value /// input that is not contiguous: every value is read by parse_msgpack_value
+3 -28
View File
@@ -10067,37 +10067,12 @@ namespace view
template<typename T> template<typename T>
struct value_tag {}; struct value_tag {};
/// whether basic_json reads T with a dedicated from_json() overload that does
/// not accept a boolean: exactly number_integer_t, number_unsigned_t and
/// number_float_t (the other arithmetic types convert a boolean to 0 or 1)
template<typename T, typename BasicJsonType>
struct rejects_boolean : std::integral_constant < bool,
std::is_same<T, typename BasicJsonType::number_integer_t>::value
|| std::is_same<T, typename BasicJsonType::number_unsigned_t>::value
|| std::is_same<T, typename BasicJsonType::number_float_t>::value >
{};
/// a boolean as T: 0 or 1
template<typename T>
NLOHMANN_VIEW_ALWAYS_INLINE T boolean_as(const document_data& /*d*/, const node& n, std::false_type /*rejects*/)
{
return static_cast<T>((n.flags & node_flags::is_true) != 0);
}
/// a boolean as number_integer_t, number_unsigned_t or number_float_t: type_error.302, as in basic_json
template<typename T>
[[noreturn]] NLOHMANN_VIEW_ALWAYS_INLINE T boolean_as(const document_data& d, const node& n, std::true_type /*rejects*/)
{
throw_type_error(error_site(&d, &n), exception_id::type_mismatch, "type must be number, but is ", value_type_name(static_cast<value_t>(n.kind)));
}
/*! /*!
@brief the number or boolean of a node converted to an arithmetic type @brief the number or boolean of a node converted to an arithmetic type
As basic_json's get<T>() for arithmetic types: integers and floats are As basic_json's get<T>() for arithmetic types: integers and floats are
converted with static_cast, booleans give 0 or 1 (but throw type_error.302 for converted with static_cast, booleans give 0 or 1, and other types throw
number_integer_t, number_unsigned_t and number_float_t, as basic_json does), type_error.302.
and other types throw type_error.302.
*/ */
template<typename T, typename BasicJsonType> template<typename T, typename BasicJsonType>
NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const node& n) NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const node& n)
@@ -10111,7 +10086,7 @@ NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const nod
case value_t::number_float: case value_t::number_float:
return static_cast<T>(float_value<typename BasicJsonType::number_float_t>(d, n)); return static_cast<T>(float_value<typename BasicJsonType::number_float_t>(d, n));
case value_t::boolean: case value_t::boolean:
return boolean_as<T>(d, n, rejects_boolean<T, BasicJsonType> {}); return static_cast<T>((n.flags & node_flags::is_true) != 0);
case value_t::null: case value_t::null:
case value_t::object: case value_t::object:
case value_t::array: case value_t::array:
+18
View File
@@ -629,6 +629,22 @@ static json make_containers()
return j; return j;
} }
/// many arrays of one number each: one-element arrays are left to the general
/// path (CBOR and MessagePack), which reads a lone number slower than the loop
/// for arrays of scalars would
static json make_singles()
{
json data = json::array();
for (std::size_t i = 0; i < 100000; ++i)
{
data.push_back(json::array({i}));
}
json j = json::object();
j["data"] = std::move(data);
return j;
}
/// one flat array of numbers: the scalar decoding path, which must not move /// one flat array of numbers: the scalar decoding path, which must not move
static json make_scalars() static json make_scalars()
{ {
@@ -676,6 +692,8 @@ BENCHMARK_CAPTURE(FromBinaryShape, containers / ubjson_optimized, make_container
BENCHMARK_CAPTURE(FromBinaryShape, containers / bjdata, make_containers, binary_format::bjdata); BENCHMARK_CAPTURE(FromBinaryShape, containers / bjdata, make_containers, binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryShape, containers / bson, make_containers, binary_format::bson); BENCHMARK_CAPTURE(FromBinaryShape, containers / bson, make_containers, binary_format::bson);
BENCHMARK_CAPTURE(FromBinaryShape, containers / bon8, make_containers, binary_format::bon8); BENCHMARK_CAPTURE(FromBinaryShape, containers / bon8, make_containers, binary_format::bon8);
BENCHMARK_CAPTURE(FromBinaryShape, singles / cbor, make_singles, binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryShape, singles / msgpack, make_singles, binary_format::msgpack);
// BSON names every array element, so a large array measures key generation // BSON names every array element, so a large array measures key generation
// rather than scalar decoding and is left out here // rather than scalar decoding and is left out here
BENCHMARK_CAPTURE(FromBinaryShape, scalars / cbor, make_scalars, binary_format::cbor); BENCHMARK_CAPTURE(FromBinaryShape, scalars / cbor, make_scalars, binary_format::cbor);
+38
View File
@@ -23,6 +23,7 @@ using nlohmann::json;
#include <cstdint> #include <cstdint>
#include <fstream> #include <fstream>
#include <limits> #include <limits>
#include <list>
#include <sstream> #include <sstream>
#include <vector> #include <vector>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
@@ -1199,6 +1200,31 @@ TEST_CASE("BSON keys from contiguous and stream input")
from_stream = e.what(); from_stream = e.what();
} }
CHECK(from_vector == from_stream); CHECK(from_vector == from_stream);
// error_handler_t::strict checks the array element names in place on
// contiguous input and byte by byte otherwise; the names are
// well-formed, so both give the same value or error as above
std::string strict_from_vector;
std::string strict_from_list;
try
{
strict_from_vector = json::from_bson(input, true, true, json::error_handler_t::strict).dump();
}
catch (const json::parse_error& e)
{
strict_from_vector = e.what();
}
try
{
const std::list<std::uint8_t> list(input.begin(), input.end());
strict_from_list = json::from_bson(list.begin(), list.end(), true, true, json::error_handler_t::strict).dump();
}
catch (const json::parse_error& e)
{
strict_from_list = e.what();
}
CHECK(strict_from_vector == from_vector);
CHECK(strict_from_list == from_vector);
} }
} }
@@ -1227,6 +1253,18 @@ TEST_CASE("BSON array element names")
std::istringstream stream(std::string(reinterpret_cast<const char*>(bson.data()), bson.size())); std::istringstream stream(std::string(reinterpret_cast<const char*>(bson.data()), bson.size()));
CHECK(json::from_bson(stream) == expected); CHECK(json::from_bson(stream) == expected);
// input that is not contiguous reads the names byte by byte and must reject
// the same name
const std::list<std::uint8_t> list(bson.begin(), bson.end());
CHECK_THROWS_WITH_AS(_ = json::from_bson(list.begin(), list.end(), true, true, json::error_handler_t::strict), "[json.exception.parse_error.113] parse error at byte 14: syntax error while parsing BSON key: invalid string: ill-formed UTF-8 byte", json::parse_error&);
// without exceptions, the rejected name ends the parse
CHECK(json::from_bson(bson, true, false, json::error_handler_t::strict).is_discarded());
// well-formed names pass the check without being stored
const json valid = {{"a", {true, false, {{"k", json::array({1, 2})}}}}};
CHECK(json::from_bson(json::to_bson(valid), true, true, json::error_handler_t::strict) == valid);
} }
#endif #endif
+28
View File
@@ -2520,6 +2520,34 @@ TEST_CASE("CBOR arrays of scalars from contiguous and non-contiguous input")
} }
} }
TEST_CASE("CBOR arrays of one element from contiguous and non-contiguous input")
{
// contiguous input leaves an array of one element to the general path
// rather than to the loop that reads arrays of scalars; both inputs must
// still give the same events, value, or error for the complete document
// and for every truncation of it, with enough bytes after each array that
// the loop could have been entered
const json j = json::parse(R"({"a": [[[[1.5]]]], "b": [7], "c": [{"x": 2.5}], "d": ["s"], "e": [[1, 2]], "f": [null], "g": 1.25})");
const std::vector<std::uint8_t> bytes = json::to_cbor(j);
CHECK(json::from_cbor(bytes) == j);
for (std::size_t length = 0; length <= bytes.size(); ++length)
{
CAPTURE(length)
const std::vector<std::uint8_t> contiguous(bytes.begin(), bytes.begin() + static_cast<std::ptrdiff_t>(length));
const std::list<std::uint8_t> bytewise(contiguous.begin(), contiguous.end());
CHECK(json::from_cbor(contiguous, true, false).dump() == json::from_cbor(bytewise.begin(), bytewise.end(), true, false).dump());
SaxEventLogger from_contiguous;
SaxEventLogger from_bytewise;
CHECK(json::sax_parse(contiguous, &from_contiguous, json::input_format_t::cbor, true, false)
== json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::cbor, true, false));
CHECK(from_contiguous.events == from_bytewise.events);
}
}
TEST_CASE("CBOR indefinite-length strings do not recurse per chunk") TEST_CASE("CBOR indefinite-length strings do not recurse per chunk")
{ {
// Reading an indefinite-length string or byte array used to call itself // Reading an indefinite-length string or byte array used to call itself
-38
View File
@@ -1260,44 +1260,6 @@ TEST_CASE("value conversion")
#undef JSON_HAS_CPP_17 #undef JSON_HAS_CPP_17
#endif #endif
TEST_CASE("get<T>() of a boolean (#5988)")
{
// a boolean converts to 0/1 for every arithmetic type except number_integer_t, number_unsigned_t and
// number_float_t, whose dedicated from_json() overloads throw
const json t = true;
const json f = false;
SECTION("0 and 1")
{
// long and unsigned long are left out: they are std::int64_t and std::uint64_t on some platforms
CHECK(t.get<bool>() == true);
CHECK(f.get<bool>() == false);
CHECK(t.get<char>() == 1);
CHECK(t.get<signed char>() == 1);
CHECK(t.get<unsigned char>() == 1);
CHECK(t.get<short>() == 1); // NOLINT(google-runtime-int)
CHECK(t.get<unsigned short>() == 1); // NOLINT(google-runtime-int)
CHECK(t.get<int>() == 1);
CHECK(t.get<unsigned>() == 1);
CHECK(t.get<float>() == 1.0F);
CHECK(t.get<long double>() == 1.0L);
CHECK(f.get<int>() == 0);
CHECK(f.get<float>() == 0.0F);
}
#if !defined(JSON_NOEXCEPTION)
SECTION("number_integer_t, number_unsigned_t and number_float_t throw")
{
CHECK_THROWS_WITH_AS(t.get<json::number_integer_t>(), "[json.exception.type_error.302] type must be number, but is boolean", json::type_error&);
CHECK_THROWS_WITH_AS(t.get<json::number_unsigned_t>(), "[json.exception.type_error.302] type must be number, but is boolean", json::type_error&);
CHECK_THROWS_WITH_AS(t.get<json::number_float_t>(), "[json.exception.type_error.302] type must be number, but is boolean", json::type_error&);
CHECK_THROWS_WITH_AS(f.get<json::number_integer_t>(), "[json.exception.type_error.302] type must be number, but is boolean", json::type_error&);
CHECK_THROWS_WITH_AS(f.get<json::number_unsigned_t>(), "[json.exception.type_error.302] type must be number, but is boolean", json::type_error&);
CHECK_THROWS_WITH_AS(f.get<json::number_float_t>(), "[json.exception.type_error.302] type must be number, but is boolean", json::type_error&);
}
#endif
}
#ifdef JSON_HAS_CPP_14 #ifdef JSON_HAS_CPP_14
#undef JSON_HAS_CPP_14 #undef JSON_HAS_CPP_14
#endif #endif
-79
View File
@@ -1551,85 +1551,6 @@ TEST_CASE("json_view values")
} }
} }
namespace
{
// get<T>() of a boolean: the same value or the same exception in basic_json and in the view (#5988)
template<typename T>
void check_boolean_get_parity()
{
for (const bool b :
{
true, false
})
{
const std::string text = b ? "true" : "false";
const json j = json::parse(text);
const json_document d = json_document::parse(text);
CAPTURE(text)
#if !defined(JSON_NOEXCEPTION)
T dom_value {};
T view_value{};
const std::string dom_error = json_view_test::exception_of_call([&] { dom_value = j.get<T>(); });
const std::string view_error = json_view_test::exception_of_call([&] { view_value = d.root().template get<T>(); });
CHECK(dom_error == view_error);
if (dom_error.empty())
{
CHECK(dom_value == view_value);
CHECK(view_value == static_cast<T>(b));
}
#else
// without exceptions, the combinations that throw (number_integer_t, number_unsigned_t, number_float_t)
// cannot be run: JSON_THROW aborts
const bool throws = std::is_same<T, json::number_integer_t>::value
|| std::is_same<T, json::number_unsigned_t>::value
|| std::is_same<T, json::number_float_t>::value;
if (!throws)
{
CHECK(j.get<T>() == d.root().template get<T>());
}
#endif
}
}
} // namespace
TEST_CASE("json_view get<T>() of a boolean (#5988)")
{
check_boolean_get_parity<bool>();
check_boolean_get_parity<char>();
check_boolean_get_parity<signed char>();
check_boolean_get_parity<unsigned char>();
check_boolean_get_parity<short>(); // NOLINT(google-runtime-int)
check_boolean_get_parity<unsigned short>(); // NOLINT(google-runtime-int)
check_boolean_get_parity<int>();
check_boolean_get_parity<unsigned>();
check_boolean_get_parity<long>(); // NOLINT(google-runtime-int)
check_boolean_get_parity<unsigned long>(); // NOLINT(google-runtime-int)
check_boolean_get_parity<long long>(); // NOLINT(google-runtime-int)
check_boolean_get_parity<unsigned long long>(); // NOLINT(google-runtime-int)
check_boolean_get_parity<std::int64_t>();
check_boolean_get_parity<std::uint64_t>();
check_boolean_get_parity<float>();
check_boolean_get_parity<double>();
check_boolean_get_parity<long double>();
#if !defined(JSON_NOEXCEPTION)
SECTION("the exceptions of the exact number types")
{
const json_document d = json_document::parse("true");
CHECK_THROWS_WITH_AS(d.root().get<json::number_integer_t>(), json_view_test::message("type_error.302", "", 0, 4, "type must be number, but is boolean").c_str(), json::type_error&);
CHECK_THROWS_WITH_AS(d.root().get<json::number_unsigned_t>(), json_view_test::message("type_error.302", "", 0, 4, "type must be number, but is boolean").c_str(), json::type_error&);
CHECK_THROWS_WITH_AS(d.root().get<json::number_float_t>(), json_view_test::message("type_error.302", "", 0, 4, "type must be number, but is boolean").c_str(), json::type_error&);
CHECK(d.root().get<int>() == 1);
// get_to() and value() go through get<T>()
json::number_float_t x = 0;
CHECK_THROWS_AS(d.root().get_to(x), json::type_error&);
const json_document o = json_document::parse("{\"b\":true}");
CHECK_THROWS_AS(static_cast<void>(o.root().value("b", 0.0)), json::type_error&);
CHECK(o.root().value("b", 0) == 1);
}
#endif
}
TEST_CASE("json_view JSON pointers") TEST_CASE("json_view JSON pointers")
{ {
SECTION("every value of generated documents") SECTION("every value of generated documents")
+1 -1
View File
@@ -692,7 +692,7 @@ TEST_CASE("json_view builder: fraction digits")
check_same(number); check_same(number);
check_same("[" + number + "]"); check_same("[" + number + "]");
std::string pair; std::string pair;
pair.reserve(number.size() * 2 + 3); pair.reserve((number.size() * 2) + 3);
pair += '['; pair += '[';
pair += number; pair += number;
pair += ','; pair += ',';
+2 -1
View File
@@ -521,7 +521,8 @@ TEST_CASE("json_view exceptions name the path and the bytes of the value")
std::string text = "7"; std::string text = "7";
for (int level = 0; level < 40; ++level) for (int level = 0; level < 40; ++level)
{ {
text = R"([[1, [2]], )" + text + R"(, {"k": [3, {"l": [4]}]}, [5, [6, [7]]]])"; text.insert(0, R"([[1, [2]], )");
text += R"(, {"k": [3, {"l": [4]}]}, [5, [6, [7]]]])";
} }
CAPTURE(text) CAPTURE(text)
const json j = json::parse(text); const json j = json::parse(text);
+28
View File
@@ -2311,3 +2311,31 @@ TEST_CASE("MessagePack arrays of scalars from contiguous and non-contiguous inpu
== json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::msgpack)); == json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::msgpack));
} }
} }
TEST_CASE("MessagePack arrays of one element from contiguous and non-contiguous input")
{
// contiguous input leaves an array of one element to the general path
// rather than to the loop that reads arrays of scalars; both inputs must
// still give the same events, value, or error for the complete document
// and for every truncation of it, with enough bytes after each array that
// the loop could have been entered
const json j = json::parse(R"({"a": [[[[1.5]]]], "b": [7], "c": [{"x": 2.5}], "d": ["s"], "e": [[1, 2]], "f": [null], "g": 1.25})");
const std::vector<std::uint8_t> bytes = json::to_msgpack(j);
CHECK(json::from_msgpack(bytes) == j);
for (std::size_t length = 0; length <= bytes.size(); ++length)
{
CAPTURE(length)
const std::vector<std::uint8_t> contiguous(bytes.begin(), bytes.begin() + static_cast<std::ptrdiff_t>(length));
const std::list<std::uint8_t> bytewise(contiguous.begin(), contiguous.end());
CHECK(json::from_msgpack(contiguous, true, false).dump() == json::from_msgpack(bytewise.begin(), bytewise.end(), true, false).dump());
SaxEventLogger from_contiguous;
SaxEventLogger from_bytewise;
CHECK(json::sax_parse(contiguous, &from_contiguous, json::input_format_t::msgpack, true, false)
== json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::msgpack, true, false));
CHECK(from_contiguous.events == from_bytewise.events);
}
}