Compare commits

...
Author SHA1 Message Date
Claude 3c38455c7d BSON writer: drop the SFINAE guard that Visual Studio 2017 cannot deduce
write_bson_document_single_pass() was a member template enabled by
is_detected<sink_written_t, SinkType>. MSVC 19.16 (Visual Studio 2017)
fails to deduce it through the class-scope alias template (C2672, C2783)
when it compiles to_bson() and to_bon8(), so every Visual Studio 2017
job on AppVeyor fails since #5979.

The guard is not needed: as a non-template member of the class template,
the function is only instantiated when called, and its only caller is
the write_bson_document() overload for sinks that have written(), which
already calls oa.written() and oa.rewind() itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>
2026-11-02 10:35:13 +00:00
Claude 4571446da7 CI: give the Valgrind tests a timeout of one hour
Under Valgrind, test-json_view_builder_cpp11 and test-ordered_json_cpp11
took 1435 s and 1390 s on develop (ddc34f5), close to ctest's default
timeout of 1500 s. On the next run (b2674a4) the whole job was about 10 %
slower and both timed out, failing ci_test_valgrind on develop and on
every PR based on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>
2026-11-02 10:29:38 +00:00
Niels Lohmann b2674a4a2f BSON reader: check array element names in place in strict mode (#5984) 2026-11-01 21:03:35 +01:00
Niels Lohmann e2423a1857 CI: pin ubuntu-latest jobs to ubuntu-26.04 (#5983)
GitHub is moving the ubuntu-latest label from Ubuntu 24.04 to 26.04
between October 19 and November 19, 2026 (actions/runner-images#14748).
Pin every ubuntu-latest job to ubuntu-26.04 so the switch does not
change the toolchain under our jobs, and so the behaviour does not
depend on when GitHub reaches our runs.

Jobs pinned to ubuntu-24.04 and ubuntu-24.04-arm are unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 20:43:07 +01:00
16 changed files with 148 additions and 34 deletions

No files matched your search

+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
permissions: permissions:
actions: write actions: write
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
+1 -1
View File
@@ -19,7 +19,7 @@ permissions:
jobs: jobs:
save: save:
if: github.event_name == 'pull_request' if: github.event_name == 'pull_request'
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2 uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -16,7 +16,7 @@ concurrency:
jobs: jobs:
check_docs_links: check_docs_links:
if: github.repository == 'nlohmann/json' if: github.repository == 'nlohmann/json'
runs-on: ubuntu-latest runs-on: ubuntu-26.04
timeout-minutes: 30 timeout-minutes: 30
steps: steps:
- name: Harden Runner - name: Harden Runner
@@ -12,7 +12,7 @@ jobs:
comment: comment:
# push runs on develop have no PR to comment on (and no "pr" artifact) # push runs on develop have no PR to comment on (and no "pr" artifact)
if: ${{ github.event.workflow_run.conclusion == 'failure' && github.event.workflow_run.event == 'pull_request' }} if: ${{ github.event.workflow_run.conclusion == 'failure' && github.event.workflow_run.event == 'pull_request' }}
runs-on: ubuntu-latest runs-on: ubuntu-26.04
permissions: permissions:
contents: read contents: read
actions: read actions: read
+1 -1
View File
@@ -18,7 +18,7 @@ permissions:
jobs: jobs:
dependency-review: dependency-review:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2 uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -24,7 +24,7 @@ on:
jobs: jobs:
flawfinder: flawfinder:
name: Flawfinder name: Flawfinder
runs-on: ubuntu-latest runs-on: ubuntu-26.04
permissions: permissions:
actions: read actions: read
contents: read contents: read
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
contents: read contents: read
pull-requests: write pull-requests: write
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
+1 -1
View File
@@ -34,7 +34,7 @@ jobs:
contents: write contents: write
if: github.repository == 'nlohmann/json' if: github.repository == 'nlohmann/json'
runs-on: ubuntu-latest runs-on: ubuntu-26.04
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2 uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+1 -1
View File
@@ -24,7 +24,7 @@ permissions:
jobs: jobs:
analysis: analysis:
name: Scorecard analysis name: Scorecard analysis
runs-on: ubuntu-latest runs-on: ubuntu-26.04
permissions: permissions:
# Needed to upload the results to code-scanning dashboard. # Needed to upload the results to code-scanning dashboard.
security-events: write security-events: write
+1 -1
View File
@@ -8,7 +8,7 @@ permissions:
jobs: jobs:
stale: stale:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
permissions: permissions:
issues: write issues: write
+13 -13
View File
@@ -18,7 +18,7 @@ concurrency:
jobs: jobs:
ci_test_gcc: ci_test_gcc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: gcc:latest container: gcc:latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -130,7 +130,7 @@ jobs:
run: cmake --build build --target ${{ matrix.target }} run: cmake --build build --target ${{ matrix.target }}
ci_static_analysis_clang: ci_static_analysis_clang:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: silkeh/clang:22 container: silkeh/clang:22
strategy: strategy:
matrix: matrix:
@@ -149,7 +149,7 @@ jobs:
run: cmake --build build --target ${{ matrix.target }} run: cmake --build build --target ${{ matrix.target }}
ci_cmake_options: ci_cmake_options:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: ubuntu:24.04 container: ubuntu:24.04
strategy: strategy:
matrix: matrix:
@@ -209,7 +209,7 @@ jobs:
fail-on-error: false fail-on-error: false
ci_test_compilers_gcc_old: ci_test_compilers_gcc_old:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
strategy: strategy:
matrix: matrix:
compiler: ['4.8', '4.9', '5', '6'] compiler: ['4.8', '4.9', '5', '6']
@@ -251,7 +251,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }} run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }}
ci_test_compilers_gcc: ci_test_compilers_gcc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
strategy: strategy:
matrix: matrix:
# older GCC docker images (4, 5, 6) fail to check out code # older GCC docker images (4, 5, 6) fail to check out code
@@ -269,7 +269,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_default run: cmake --build build --target ci_test_compiler_default
ci_test_compilers_clang: ci_test_compilers_clang:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
strategy: strategy:
matrix: matrix:
compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', '21', '22', 'latest'] compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', '21', '22', 'latest']
@@ -289,7 +289,7 @@ jobs:
run: cmake --build build --target ci_test_compiler_default run: cmake --build build --target ci_test_compiler_default
ci_test_standards_gcc: ci_test_standards_gcc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: gcc:latest container: gcc:latest
strategy: strategy:
matrix: matrix:
@@ -306,7 +306,7 @@ jobs:
run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }} run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }}
ci_test_standards_clang: ci_test_standards_clang:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: silkeh/clang:latest container: silkeh/clang:latest
strategy: strategy:
matrix: matrix:
@@ -330,7 +330,7 @@ jobs:
if: ${{ matrix.stdlib == 'libstdcxx' }} if: ${{ matrix.stdlib == 'libstdcxx' }}
ci_cuda_example: ci_cuda_example:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -356,7 +356,7 @@ jobs:
strategy: strategy:
matrix: matrix:
container: ['gcc:latest', 'silkeh/clang:latest'] container: ['gcc:latest', 'silkeh/clang:latest']
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: ${{ matrix.container }} container: ${{ matrix.container }}
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -383,7 +383,7 @@ jobs:
run: cmake --build build --target ci_module_cpp20 run: cmake --build build --target ci_module_cpp20
ci_icpc: ci_icpc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
# Intel discontinued the classic icc/icpc compiler in oneAPI 2024.0; this is # Intel discontinued the classic icc/icpc compiler in oneAPI 2024.0; this is
# Intel's own last officially published image that still includes it. # Intel's own last officially published image that still includes it.
container: intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04 container: intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04
@@ -402,7 +402,7 @@ jobs:
run: cmake --build build --target ci_icpc run: cmake --build build --target ci_icpc
ci_icpx: ci_icpx:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: intel/oneapi-hpckit:latest container: intel/oneapi-hpckit:latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -416,7 +416,7 @@ jobs:
run: cmake --build build --target ci_icpx run: cmake --build build --target ci_icpx
ci_nvhpc: ci_nvhpc:
runs-on: ubuntu-latest runs-on: ubuntu-26.04
container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04 container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+3 -1
View File
@@ -178,8 +178,10 @@ function(_json_test_add_test test_name file main cxx_standard)
COMMAND ${memcheck_command} $<TARGET_FILE:${test_target}> ${DOCTEST_TEST_FILTER} COMMAND ${memcheck_command} $<TARGET_FILE:${test_target}> ${DOCTEST_TEST_FILTER}
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
) )
# under Valgrind the slowest tests take close to 25 minutes, so ctest's
# default timeout of 1500 seconds leaves no headroom
set_tests_properties(${test_target}_valgrind PROPERTIES set_tests_properties(${test_target}_valgrind PROPERTIES
LABELS "valgrind" FIXTURES_REQUIRED TEST_DATA LABELS "valgrind" FIXTURES_REQUIRED TEST_DATA TIMEOUT 3600
) )
endif() endif()
endfunction() endfunction()
@@ -467,11 +467,13 @@ class binary_reader
const std::size_t element_type_parse_position = chars_read; const std::size_t element_type_parse_position = chars_read;
// an array's elements are named "0", "1", ... in the wire format, // an array's elements are named "0", "1", ... in the wire format,
// and those names are not passed on: they are skipped unless // and those names are not passed on: they are skipped, and only
// error_handler_t::strict must reject ill-formed UTF-8 in them // checked if error_handler_t::strict must reject ill-formed UTF-8
const bool is_object = container_stack.back().is_object; const bool is_object = container_stack.back().is_object;
key.clear(); key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key, is_object || error_handler == error_handler_t::strict))) if (JSON_HEDLEY_UNLIKELY(!(is_object || error_handler != error_handler_t::strict
? get_bson_cstr(key, is_object)
: check_bson_cstr(key, std::integral_constant<bool, bulk_scan> {}))))
{ {
return false; return false;
} }
@@ -495,7 +497,7 @@ class binary_reader
@param[in] store whether the characters are needed; if not (the names of @param[in] store whether the characters are needed; if not (the names of
an array's elements), they are skipped and @a result may an array's elements), they are skipped and @a result may
be left unchanged, and they are not checked against be left unchanged, and they are not checked against
@ref error_handler @ref error_handler (@ref check_bson_cstr checks them)
@return `true` if the \\x00-byte indicating the end of the string was @return `true` if the \\x00-byte indicating the end of the string was
encountered before the EOF; false` indicates an unexpected EOF. encountered before the EOF; false` indicates an unexpected EOF.
*/ */
@@ -554,6 +556,42 @@ class binary_reader
return true; return true;
} }
/*!
@brief skip a C-style string that error_handler_t::strict must check
The names of an array's elements are not passed on. On contiguous input,
a well-formed name is checked in place and skipped without being copied.
Otherwise (no \\x00-byte in the input that can be scanned, or ill-formed
UTF-8), it is read by @ref get_bson_cstr, which reports the error.
@param[in,out] result an empty string, which may receive the name
@return whether the name was read and is well-formed
*/
bool check_bson_cstr(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining != 0)
{
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = find_zero_byte(data, remaining);
if (length != remaining && JSON_HEDLEY_LIKELY(valid_utf8_string(data, length)))
{
// consume the string and its \x00-byte, as get_bson_cstr does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
}
return get_bson_cstr(result);
}
/// input that is not contiguous: the name is read and checked byte by byte
bool check_bson_cstr(string_t& result, std::false_type /*bulk*/)
{
return get_bson_cstr(result);
}
/// input that is not contiguous: C-style strings are read byte by byte /// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept
{ {
@@ -2422,7 +2422,6 @@ class binary_writer
nothing large has been written then, and the caller writes the nothing large has been written then, and the caller writes the
document in two passes, which reports that error document in two passes, which reports that error
*/ */
template<typename SinkType = OutputSinkType, typename std::enable_if<is_detected<sink_written_t, SinkType>::value, int>::type = 0>
bool write_bson_document_single_pass(const BasicJsonType& document) bool write_bson_document_single_pass(const BasicJsonType& document)
{ {
const std::size_t start = oa.written(); const std::size_t start = oa.written();
+42 -5
View File
@@ -18474,11 +18474,13 @@ class binary_reader
const std::size_t element_type_parse_position = chars_read; const std::size_t element_type_parse_position = chars_read;
// an array's elements are named "0", "1", ... in the wire format, // an array's elements are named "0", "1", ... in the wire format,
// and those names are not passed on: they are skipped unless // and those names are not passed on: they are skipped, and only
// error_handler_t::strict must reject ill-formed UTF-8 in them // checked if error_handler_t::strict must reject ill-formed UTF-8
const bool is_object = container_stack.back().is_object; const bool is_object = container_stack.back().is_object;
key.clear(); key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key, is_object || error_handler == error_handler_t::strict))) if (JSON_HEDLEY_UNLIKELY(!(is_object || error_handler != error_handler_t::strict
? get_bson_cstr(key, is_object)
: check_bson_cstr(key, std::integral_constant<bool, bulk_scan> {}))))
{ {
return false; return false;
} }
@@ -18502,7 +18504,7 @@ class binary_reader
@param[in] store whether the characters are needed; if not (the names of @param[in] store whether the characters are needed; if not (the names of
an array's elements), they are skipped and @a result may an array's elements), they are skipped and @a result may
be left unchanged, and they are not checked against be left unchanged, and they are not checked against
@ref error_handler @ref error_handler (@ref check_bson_cstr checks them)
@return `true` if the \\x00-byte indicating the end of the string was @return `true` if the \\x00-byte indicating the end of the string was
encountered before the EOF; false` indicates an unexpected EOF. encountered before the EOF; false` indicates an unexpected EOF.
*/ */
@@ -18561,6 +18563,42 @@ class binary_reader
return true; return true;
} }
/*!
@brief skip a C-style string that error_handler_t::strict must check
The names of an array's elements are not passed on. On contiguous input,
a well-formed name is checked in place and skipped without being copied.
Otherwise (no \\x00-byte in the input that can be scanned, or ill-formed
UTF-8), it is read by @ref get_bson_cstr, which reports the error.
@param[in,out] result an empty string, which may receive the name
@return whether the name was read and is well-formed
*/
bool check_bson_cstr(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining != 0)
{
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = find_zero_byte(data, remaining);
if (length != remaining && JSON_HEDLEY_LIKELY(valid_utf8_string(data, length)))
{
// consume the string and its \x00-byte, as get_bson_cstr does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
}
return get_bson_cstr(result);
}
/// input that is not contiguous: the name is read and checked byte by byte
bool check_bson_cstr(string_t& result, std::false_type /*bulk*/)
{
return get_bson_cstr(result);
}
/// input that is not contiguous: C-style strings are read byte by byte /// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept bool get_bson_cstr_bulk(string_t& /*result*/, const bool /*store*/, std::false_type /*bulk*/) const noexcept
{ {
@@ -30356,7 +30394,6 @@ class binary_writer
nothing large has been written then, and the caller writes the nothing large has been written then, and the caller writes the
document in two passes, which reports that error document in two passes, which reports that error
*/ */
template<typename SinkType = OutputSinkType, typename std::enable_if<is_detected<sink_written_t, SinkType>::value, int>::type = 0>
bool write_bson_document_single_pass(const BasicJsonType& document) bool write_bson_document_single_pass(const BasicJsonType& document)
{ {
const std::size_t start = oa.written(); const std::size_t start = oa.written();
+38
View File
@@ -23,6 +23,7 @@ using nlohmann::json;
#include <cstdint> #include <cstdint>
#include <fstream> #include <fstream>
#include <limits> #include <limits>
#include <list>
#include <sstream> #include <sstream>
#include <vector> #include <vector>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
@@ -1199,6 +1200,31 @@ TEST_CASE("BSON keys from contiguous and stream input")
from_stream = e.what(); from_stream = e.what();
} }
CHECK(from_vector == from_stream); CHECK(from_vector == from_stream);
// error_handler_t::strict checks the array element names in place on
// contiguous input and byte by byte otherwise; the names are
// well-formed, so both give the same value or error as above
std::string strict_from_vector;
std::string strict_from_list;
try
{
strict_from_vector = json::from_bson(input, true, true, json::error_handler_t::strict).dump();
}
catch (const json::parse_error& e)
{
strict_from_vector = e.what();
}
try
{
const std::list<std::uint8_t> list(input.begin(), input.end());
strict_from_list = json::from_bson(list.begin(), list.end(), true, true, json::error_handler_t::strict).dump();
}
catch (const json::parse_error& e)
{
strict_from_list = e.what();
}
CHECK(strict_from_vector == from_vector);
CHECK(strict_from_list == from_vector);
} }
} }
@@ -1227,6 +1253,18 @@ TEST_CASE("BSON array element names")
std::istringstream stream(std::string(reinterpret_cast<const char*>(bson.data()), bson.size())); std::istringstream stream(std::string(reinterpret_cast<const char*>(bson.data()), bson.size()));
CHECK(json::from_bson(stream) == expected); CHECK(json::from_bson(stream) == expected);
// input that is not contiguous reads the names byte by byte and must reject
// the same name
const std::list<std::uint8_t> list(bson.begin(), bson.end());
CHECK_THROWS_WITH_AS(_ = json::from_bson(list.begin(), list.end(), true, true, json::error_handler_t::strict), "[json.exception.parse_error.113] parse error at byte 14: syntax error while parsing BSON key: invalid string: ill-formed UTF-8 byte", json::parse_error&);
// without exceptions, the rejected name ends the parse
CHECK(json::from_bson(bson, true, false, json::error_handler_t::strict).is_discarded());
// well-formed names pass the check without being stored
const json valid = {{"a", {true, false, {{"k", json::array({1, 2})}}}}};
CHECK(json::from_bson(json::to_bson(valid), true, true, json::error_handler_t::strict) == valid);
} }
#endif #endif