Trim the compiler-appended NUL from wide/UTF string literals too (#5702)

With JSON_STRICT_NUL_HANDLING defined to 1, parsing a wide, UTF-16,
UTF-32, or (C++20) UTF-8 string literal (e.g. json::parse(L"[1]"))
failed with parse_error.101 at the terminating NUL of the literal,
and accept() returned false. The array overload of input_adapter()
only dropped the compiler-added trailing '\0' for arrays of char,
so for wchar_t, char16_t, char32_t, and char8_t arrays that
terminator was passed to the parser as data, which the macro then
rejected.

Broaden the trimming to every character type that a string literal
can use (char, wchar_t, char16_t, char32_t, and, since C++20,
char8_t). Arrays of any other element type (unsigned char,
std::uint8_t, ...), as used for CBOR/MessagePack, are unaffected: a
trailing zero byte there is still read as genuine data.

Fixes #5658.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann authored and GitHub committed 2026-09-30 20:07:37 +02:00
1 parent bfe0f32d71
commit 4bb1b14b06
4 files changed
+79 -28

No files matched your search

@@ -16,7 +16,8 @@ byte is still not rejected:
[`from_msgpack`](../basic_json/from_msgpack.md), [`from_ubjson`](../basic_json/from_ubjson.md)) are never affected: there, `0x00` is ordinary data. [`from_msgpack`](../basic_json/from_msgpack.md), [`from_ubjson`](../basic_json/from_ubjson.md)) are never affected: there, `0x00` is ordinary data.
- A bare `const char*` pointer has no length of its own, so its length is still determined with `strlen()`. The first - A bare `const char*` pointer has no length of its own, so its length is still determined with `strlen()`. The first
NUL byte therefore still marks the end of the input, and nothing after it is read. NUL byte therefore still marks the end of the input, and nothing after it is read.
- One trailing `'\0'` at the end of a `char` array (e.g., a string literal) is trimmed; see the warning below. - One trailing `'\0'` at the end of a `char`, `wchar_t`, `char16_t`, `char32_t`, or (C++20) `char8_t` array (e.g., a
string literal) is trimmed; see the warning below.
## Default definition ## Default definition
@@ -57,13 +58,15 @@ The default value is `0` (disabled — existing behavior is preserved).
This macro must be defined **before** including `<nlohmann/json.hpp>`. Defining it after the include has no This macro must be defined **before** including `<nlohmann/json.hpp>`. Defining it after the include has no
effect. effect.
Enabling it also changes how a `char` array (including a string literal, e.g. `json::parse("123")`) is read: such Enabling it also changes how an array of a text-literal element type (`char`, `wchar_t`, `char16_t`, `char32_t`,
an array normally carries a trailing `'\0'` contributed by the compiler, not by the source text. With this macro or, since C++20, `char8_t` - including a string literal, e.g. `json::parse("123")` or `json::parse(L"123")`) is
enabled, that one trailing byte is trimmed if present so that parsing a string literal keeps working; every other read: such an array normally carries a trailing `'\0'` contributed by the compiler, not by the source text. With
byte in the array - including any `'\0'` that is not the very last element - is read as real data and rejected this macro enabled, that one trailing element is trimmed if present so that parsing a string literal keeps
like any other unexpected byte. Arrays of any other element type (`unsigned char`, `std::uint8_t`, ...), as used working, for any of these character types; every other element in the array - including any `'\0'` that is not
for CBOR or MessagePack, are never affected by this trimming; their full extent - including a genuine trailing the very last element - is read as real data and rejected like any other unexpected byte. Arrays of any other
`0x00` - is always preserved, in both states of this macro. element type (`unsigned char`, `std::uint8_t`, ...), as used for CBOR or MessagePack, are never affected by this
trimming; their full extent - including a genuine trailing `0x00` - is always preserved, in both states of this
macro.
!!! note "ABI compatibility" !!! note "ABI compatibility"
@@ -855,16 +855,28 @@ template<typename T, std::size_t N>
auto input_adapter(T (&array)[N]) -> decltype(input_adapter(array, array + N)) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) auto input_adapter(T (&array)[N]) -> decltype(input_adapter(array, array + N)) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
{ {
#if JSON_STRICT_NUL_HANDLING #if JSON_STRICT_NUL_HANDLING
// A `char` array from string-literal initialization (e.g. json::parse("123")) // A text-literal array from string-literal initialization (e.g.
// carries a trailing '\0' contributed by the compiler, not by the source // json::parse("123") or json::parse(L"123")) carries a trailing '\0'
// text; drop exactly that one byte so it is not mistaken for real trailing // contributed by the compiler, not by the source text; drop exactly that
// data. Every other element type (unsigned char, std::uint8_t, ...) keeps // one byte so it is not mistaken for real trailing data. This covers all
// the full extent unconditionally, since a trailing zero byte there is // character types that string literals can use: char, wchar_t, char16_t,
// genuine data (e.g. CBOR/MessagePack). This intentionally does not // char32_t, and (C++20) char8_t. Every other element type (unsigned char,
// strlen()-scan the array (as the pointer overload above does for a // std::uint8_t, ...) keeps the full extent unconditionally, since a
// null-delimited string): for a `char` array that is not NUL-terminated // trailing zero byte there is genuine data (e.g. CBOR/MessagePack). This
// within its bounds, that would read past the end of the array. // intentionally does not strlen()-scan the array (as the pointer overload
if (std::is_same<typename std::remove_cv<T>::type, char>::value && N > 0 && array[N - 1] == 0) // above does for a null-delimited string): for an array that is not
// NUL-terminated within its bounds, that would read past the end of the
// array.
using char_t = typename std::remove_cv<T>::type;
constexpr bool is_text_literal_type = std::is_same<char_t, char>::value
|| std::is_same<char_t, wchar_t>::value
|| std::is_same<char_t, char16_t>::value
|| std::is_same<char_t, char32_t>::value
#if defined(__cpp_char8_t)
|| std::is_same<char_t, char8_t>::value
#endif
;
if (is_text_literal_type && N > 0 && array[N - 1] == 0)
{ {
return input_adapter(array, array + N - 1); return input_adapter(array, array + N - 1);
} }
+22 -10
View File
@@ -8411,16 +8411,28 @@ template<typename T, std::size_t N>
auto input_adapter(T (&array)[N]) -> decltype(input_adapter(array, array + N)) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) auto input_adapter(T (&array)[N]) -> decltype(input_adapter(array, array + N)) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
{ {
#if JSON_STRICT_NUL_HANDLING #if JSON_STRICT_NUL_HANDLING
// A `char` array from string-literal initialization (e.g. json::parse("123")) // A text-literal array from string-literal initialization (e.g.
// carries a trailing '\0' contributed by the compiler, not by the source // json::parse("123") or json::parse(L"123")) carries a trailing '\0'
// text; drop exactly that one byte so it is not mistaken for real trailing // contributed by the compiler, not by the source text; drop exactly that
// data. Every other element type (unsigned char, std::uint8_t, ...) keeps // one byte so it is not mistaken for real trailing data. This covers all
// the full extent unconditionally, since a trailing zero byte there is // character types that string literals can use: char, wchar_t, char16_t,
// genuine data (e.g. CBOR/MessagePack). This intentionally does not // char32_t, and (C++20) char8_t. Every other element type (unsigned char,
// strlen()-scan the array (as the pointer overload above does for a // std::uint8_t, ...) keeps the full extent unconditionally, since a
// null-delimited string): for a `char` array that is not NUL-terminated // trailing zero byte there is genuine data (e.g. CBOR/MessagePack). This
// within its bounds, that would read past the end of the array. // intentionally does not strlen()-scan the array (as the pointer overload
if (std::is_same<typename std::remove_cv<T>::type, char>::value && N > 0 && array[N - 1] == 0) // above does for a null-delimited string): for an array that is not
// NUL-terminated within its bounds, that would read past the end of the
// array.
using char_t = typename std::remove_cv<T>::type;
constexpr bool is_text_literal_type = std::is_same<char_t, char>::value
|| std::is_same<char_t, wchar_t>::value
|| std::is_same<char_t, char16_t>::value
|| std::is_same<char_t, char32_t>::value
#if defined(__cpp_char8_t)
|| std::is_same<char_t, char8_t>::value
#endif
;
if (is_text_literal_type && N > 0 && array[N - 1] == 0)
{ {
return input_adapter(array, array + N - 1); return input_adapter(array, array + N - 1);
} }
+24
View File
@@ -645,6 +645,30 @@ TEST_CASE("parser class")
// carries a compiler-appended trailing '\0') still works, // carries a compiler-appended trailing '\0') still works,
// even though a NUL byte is now rejected everywhere else // even though a NUL byte is now rejected everywhere else
CHECK(json::parse("123") == json(123)); CHECK(json::parse("123") == json(123));
// regression test for issue #5658: the same holds for wide,
// UTF-16, UTF-32, and (C++20) UTF-8 string literals, whose
// compiler-appended trailing '\0' is not of type `char`
CHECK(json::parse(L"[1]") == json({1}));
CHECK(json::accept(L"[1]"));
CHECK(json::parse(u"[1]") == json({1}));
CHECK(json::accept(u"[1]"));
CHECK(json::parse(U"[1]") == json({1}));
CHECK(json::accept(U"[1]"));
#if defined(__cpp_char8_t)
CHECK(json::parse(u8"[1]") == json({1}));
CHECK(json::accept(u8"[1]"));
#endif
// a NUL byte inside such a literal, as opposed to the single
// compiler-appended trailing one, is still rejected
{
json _; // NOLINT(readability-identifier-naming)
CHECK_THROWS_WITH_AS(_ = json::parse(L"[1\0]"),
"[json.exception.parse_error.101] parse error at line 1, column 3: syntax error while parsing array - invalid literal; last read: '1<U+0000>'; expected ']'",
json::parse_error&);
CHECK_FALSE(json::accept(L"[1\0]"));
}
} }
#endif #endif
} }