Niels Lohmann
abbe52d6de
* docs: qualify the operator>> stream positioning guarantee operator>>'s notes state that it leaves the stream positioned right after the parsed value, so that concatenated JSON values can be read back to back. That does not hold when the value is a number: a number is only terminated by the character that follows it, and the lexer's unget() is simulated (it rewinds only the lexer's own bookkeeping), so that character stays consumed from the stream. Document the actual behaviour: the guarantee holds for all value types except numbers, which must be followed by whitespace. Also qualify the cross-reference on the JSON Lines page, which repeated the unqualified claim. Documentation only; the behaviour itself is tracked in #5340. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * fix: restore the character that terminates a number (#5340) operator>> is documented to leave the stream positioned right after the parsed value, so that concatenated JSON values can be read back to back. That did not hold for numbers: a number is only terminated by the character following it, and lexer::scan_number() reads that character and calls unget() -- which is simulated and rewinds only the lexer's own bookkeeping. input_stream_adapter consumes via sbumpc() with no matching sungetc(), so the terminating character stayed consumed and the next extraction started one byte too late ('1true' left the stream at 'rue'). Propagating unget() to the adapter directly does not work: next_unget makes the following get() replay the cached character, so the terminator would be delivered twice. Instead, restore the still-pending character once at the end of a non-strict parse, where the input is handed back to the caller: - input_stream_adapter gains unget_character() (sungetc()) and advertises it via supports_unget, detected the same way as supports_seek. - lexer::restore_pending_unget() turns a pending simulated unget of a real (non-EOF) character into a real one and clears next_unget so the character is not also replayed. It is a no-op for adapters that cannot unget, and reports failure when sungetc() fails, in which case the input is left as it was before. - parser calls it on the three non-strict paths, i.e. for operator>> and sax_parse(strict = false). Strict parse()/accept() are unaffected: they require the input to end after the value, so the character is consumed by the end-of-input check anyway. Parse error messages and reported positions are unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * tests: fix CI failures in the #5340 test helpers Four CI failures, all in the new test code: - GCC (-Werror=useless-cast): drop the `json(...)` wrapper around `json::parse(...)`, which already returns a `json`. - GCC (-Werror=unused-result): assign the discarded `json::parse()` result to a dummy, the idiom used elsewhere in the test suite, and catch `json::parse_error&` for consistency. - clang-tidy (google-default-arguments): remove the default argument from the `pbackfail()` override; `sungetc()` supplies the base declaration's default. - MSVC (bad allocation): `no_putback_streambuf::underflow()` set a one-character get area without advancing `m_pos`, so an implementation whose `istream::get` peeks before it bumps re-read the same character forever. Keep no get area at all: `underflow()` peeks, `uflow()` consumes, and `sungetc()` still always lands in `pbackfail()`, which is what the test needs. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * fix: leave the character that terminates a number in the input Read the character following a number without consuming it, instead of consuming it and putting it back. input_stream_adapter now peeks with sgetc() and only steps over the character when the next one is requested or when the adapter is destroyed, so releasing it cannot fail - no putback position is required from the streambuf. Suggested by gregmarr in #5344. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: match the version history wording to the peek-based fix Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: drop the whitespace-separator caveat from the parsing pages The caveat added in #5343 describes the behavior this branch fixes: a number no longer consumes the character that terminates it, so concatenated values need no separator. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * refactor: split the strict and non-strict paths in parser Folding the release_lookahead() call into the existing strict check left the "in strict mode" comment on an else-if branch, and made the strict condition in sax_parse() redundant with the branch it followed. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Put the stream position fix behind JSON_PRECISE_STREAM_POSITION Leaving the character that terminates a number in the stream is observable: reading "1,2,3" with repeated operator>> works today only because the comma after each number is swallowed, and std::getline after a number skips the line break. Both break with the fix, so make it opt-in for 3.x, as suggested by @gregmarr in the review. - JSON_PRECISE_STREAM_POSITION (default 0) selects the peek-based input_stream_adapter. Without it, the adapter is the consuming one from develop and has no supports_lookahead, so lexer::release_lookahead() and the parser's calls to it compile to nothing. - The macro changes input_stream_adapter's layout and member functions, so it gets the ABI tag _psp, after _bics. The ABI config tests, the natvis generator, and nlohmann_json.natvis (regenerated) know the tag. - The tests for the fix move to unit-precise-stream-position.cpp, which defines the macro itself and runs in every build, and gain the two cases above. unit-deserialization.cpp pins the default behavior instead. - The docs describe the default behavior again and point to the new macro page; version history says "added in 3.13.0, planned default in 4.0.0". Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
11 KiB
Supported Macros
Some aspects of the library can be configured by defining preprocessor macros before including the json.hpp header.
See also the API documentation for macros for examples and more information.
JSON_ASSERT(x)
This macro controls which code is executed for runtime assertions of the library.
See full documentation of JSON_ASSERT(x).
JSON_BRACE_INIT_COPY_SEMANTICS
When defined to 1, single-element brace initialization of a basic_json value (e.g., #!cpp json j{value};) is
treated as a copy/move of the element rather than wrapping it in a single-element array. The default value is 0, which
preserves the existing behavior.
See full documentation of JSON_BRACE_INIT_COPY_SEMANTICS.
JSON_CATCH_USER(exception)
This macro overrides #!cpp catch calls inside the library.
See full documentation of JSON_CATCH_USER(exception).
JSON_DIAGNOSTICS
This macro enables extended diagnostics for exception messages. Possible values are 1 to enable or 0 to disable
(default).
When enabled, exception messages contain a JSON Pointer to the JSON value that triggered the exception, see Extended diagnostic messages for an example. Note that enabling this macro increases the size of every JSON value by one pointer and adds some runtime overhead.
The diagnostics messages can also be controlled with the CMake option
JSON_Diagnostics (OFF by default) which sets JSON_DIAGNOSTICS
accordingly.
See full documentation of JSON_DIAGNOSTICS.
JSON_DIAGNOSTIC_POSITIONS
When enabled, two new member functions start_pos() and
end_pos() are added to basic_json values. If the value
was created by calling theparse function, then these functions allow querying the byte
positions of the value in the input it was parsed from. The byte positions are also used in exceptions to help locate
errors.
The diagnostics positions can also be controlled with the CMake option
JSON_Diagnostic_Positions (OFF by default) which sets
JSON_DIAGNOSTIC_POSITIONS accordingly.
See full documentation of JSON_DIAGNOSTIC_POSITIONS
JSON_HAS_CPP_11, JSON_HAS_CPP_14, JSON_HAS_CPP_17, JSON_HAS_CPP_20, JSON_HAS_CPP_23, JSON_HAS_CPP_26
The library targets C++11, but also supports some features introduced in later C++ versions (e.g., std::string_view
support for C++17). For these new features, the library implements some preprocessor checks to determine the C++
standard. By defining any of these symbols, the internal check is overridden and the provided C++ version is
unconditionally assumed. This can be helpful for compilers that only implement parts of the standard and would be
detected incorrectly.
JSON_HAS_FILESYSTEM, JSON_HAS_EXPERIMENTAL_FILESYSTEM
When compiling with C++17, the library provides conversions from and to std::filesystem::path. As compiler support
for filesystem is limited, the library tries to detect whether <filesystem>/std::filesystem (JSON_HAS_FILESYSTEM)
or <experimental/filesystem>/std::experimental::filesystem (JSON_HAS_EXPERIMENTAL_FILESYSTEM) should be used.
To override the built-in check, define JSON_HAS_FILESYSTEM or JSON_HAS_EXPERIMENTAL_FILESYSTEM to 1.
See full documentation of JSON_HAS_FILESYSTEM and JSON_HAS_EXPERIMENTAL_FILESYSTEM.
JSON_NOEXCEPTION
Exceptions can be switched off by defining the symbol JSON_NOEXCEPTION.
See full documentation of JSON_NOEXCEPTION.
JSON_DISABLE_ENUM_SERIALIZATION
When defined, default parse and serialize functions for enums are excluded and have to be provided by the user, for example, using NLOHMANN_JSON_SERIALIZE_ENUM.
See full documentation of JSON_DISABLE_ENUM_SERIALIZATION.
JSON_NO_IO
When defined, headers <cstdio>, <ios>, <iosfwd>, <istream>, and <ostream> are not included and parse functions
relying on these headers are excluded. This is relevant for environment where these I/O functions are disallowed for
security reasons (e.g., Intel Software Guard Extensions (SGX)).
See full documentation of JSON_NO_IO.
JSON_NO_THREAD_LOCAL
When defined, the library does not use #!cpp thread_local storage. Copying a value then always avoids the call stack
rather than descending into a bounded number of levels first, which is slower but yields the same values.
See full documentation of JSON_NO_THREAD_LOCAL.
JSON_PRECISE_STREAM_POSITION
When defined to 1, operator>> and non-strict
sax_parse leave an input stream positioned right after the parsed value, instead of
also consuming the character that terminates a number. The default value is 0, which preserves the existing behavior;
this is planned to become the default in version 4.0.0.
See full documentation of JSON_PRECISE_STREAM_POSITION.
JSON_SKIP_LIBRARY_VERSION_CHECK
When defined, the library will not create a compiler warning when a different version of the library was already included.
See full documentation of JSON_SKIP_LIBRARY_VERSION_CHECK.
JSON_SKIP_UNSUPPORTED_COMPILER_CHECK
When defined, the library will not create a compile error when a known unsupported compiler is detected. This allows using the library with compilers that do not fully support C++11 and may only work if unsupported features are not used.
See full documentation of JSON_SKIP_UNSUPPORTED_COMPILER_CHECK.
JSON_STRICT_NUL_HANDLING
When defined to 1, a '\0' (NUL) byte anywhere in the input is rejected with parse_error.101, like any other
unexpected byte, instead of being silently treated as end of input (see the
FAQ entry for background). The default value is 0, which preserves the
existing behavior; this is planned to become the default in version 4.0.0.
The strict handling can also be enabled with the CMake option
JSON_StrictNulHandling (OFF by default) which sets
JSON_STRICT_NUL_HANDLING accordingly.
See full documentation of JSON_STRICT_NUL_HANDLING.
JSON_THROW_USER(exception)
This macro overrides #!cpp throw calls inside the library. The argument is the exception to be thrown.
See full documentation of JSON_THROW_USER(exception).
JSON_TRY_USER
This macro overrides #!cpp try calls inside the library.
See full documentation of JSON_TRY_USER.
JSON_USE_IMPLICIT_CONVERSIONS
When defined to 0, implicit conversions are switched off. By default, implicit conversions are switched on.
See full documentation of JSON_USE_IMPLICIT_CONVERSIONS.
JSON_USE_GLOBAL_UDLS
When defined to 1 (default), the user-defined string literals operator""_json and operator""_json_pointer are
placed into the global namespace instead of nlohmann::literals::json_literals.
See full documentation of JSON_USE_GLOBAL_UDLS.
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON
When defined to 1, the library restores the legacy behavior in which a discarded value compared equal to itself. This
behavior is deprecated and switched off (0) by default.
See full documentation of JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON.
JSON_USE_SIMDUTF
When defined, UTF-8 validation of JSON strings read from contiguous byte input is delegated to the simdutf library instead of the built-in scalar validator. This is an opt-in external dependency and is not defined by default.
See full documentation of JSON_USE_SIMDUTF.
NLOHMANN_DEFINE_TYPE_*(...), NLOHMANN_DEFINE_DERIVED_TYPE_*(...)
The library defines 12 macros to simplify the serialization/deserialization of types. See the page on arbitrary type conversion for a detailed discussion.
NLOHMANN_JSON_NAMESPACE, NLOHMANN_JSON_NAMESPACE_BEGIN, NLOHMANN_JSON_NAMESPACE_END, NLOHMANN_JSON_NAMESPACE_NO_VERSION
These macros relate to the versioned, inline nlohmann namespace:
NLOHMANN_JSON_NAMESPACEevaluates to the full name of thenlohmannnamespace (including the inline ABI namespace).NLOHMANN_JSON_NAMESPACE_BEGIN/NLOHMANN_JSON_NAMESPACE_ENDopen and close the namespace (for example, to add specializations).NLOHMANN_JSON_NAMESPACE_NO_VERSION, when defined to1, omits the version component from the inline namespace.
See the nlohmann Namespace page, and the full documentation of
NLOHMANN_JSON_NAMESPACE,
NLOHMANN_JSON_NAMESPACE_BEGIN / NLOHMANN_JSON_NAMESPACE_END, and
NLOHMANN_JSON_NAMESPACE_NO_VERSION.
NLOHMANN_JSON_SERIALIZE_ENUM(type, ...)
This macro simplifies the serialization/deserialization of enum types. See Specializing enum conversion for more information.
See full documentation of NLOHMANN_JSON_SERIALIZE_ENUM.
A strict variant NLOHMANN_JSON_SERIALIZE_ENUM_STRICT throws an
exception on undefined input instead of falling back to the first mapping.
NLOHMANN_JSON_VERSION_MAJOR, NLOHMANN_JSON_VERSION_MINOR, NLOHMANN_JSON_VERSION_PATCH
These macros are defined by the library and contain the version numbers according to Semantic Versioning 2.0.0.