wt/json

0 Fork 0

mirror of https://github.com/nlohmann/json.git synced 2026-11-12 16:50:22 +00:00

Files

T

Niels Lohmann 73aea7ce54 Add JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE to set the UBJSON whole-input budget (#6083 )

* Add JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE to set the UBJSON whole-input budget

The whole-input budget for the elements of optimized UBJSON arrays of
type Z, T and F (1048576 plus 4 per byte read) rejects files written by
3.12.0 that have long arrays of these types, such as 1100 rows of 1000
nulls with a `$Z` annotation, and there was no way to read them. The
per-byte rate is now the macro JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE.

Semantics:
- default 4, so the default build is unchanged (the generated code of
  from_ubjson is identical at -O2 up to source line numbers);
- the value must be a positive integer: 0, negative, non-integral, bool,
  character and wider-than-uintmax_t values fail to compile with a
  static_assert that names the macro;
- a value that does not fit into std::size_t is the largest size_t, so
  one spelling works for 32- and 64-bit targets;
- the limit base + rate * bytes_read saturates at the largest size_t
  instead of wrapping around, for any rate and input size; a rate of
  131072 (2^20 / 8) or more cannot be exceeded by any input (the
  shortest header of an array of more than 32767 elements is 8 bytes),
  which switches the whole-input budget off;
- the limit of one array (1048576) and the base (1048576) stay fixed
  constants, so the writer fallback still follows the base only and a
  reader with any rate reads everything the writer produces;
- the macro is not part of the ABI tag and must have the same value in
  every translation unit; it is documented that way.

The memory the arrays can take in a DOM is about 20 bytes per rate and
byte of input (measured, macOS, containers of 9 * rate elements:
25 bytes per input byte at rate 1, 81 at 4, 312 at 16, 1226 at 64, 5493
at 256) plus about 33 MB for the base. A 3.12.0 row of N values has a
header of b bytes (5 up to 255, 6 up to 32767, 8 above), so a table is
read completely if N <= rate * b, and otherwise up to
(1048576 + h * rate, h = outer header: 6, 7 or 9 bytes) / (N - rate * b) rows; 1100 rows of 1000 nulls need
a rate of 8. The new page of the macro documents the formulas, the
measurements, and two examples that are run by the documentation checks.

The tests of the budget move to tests/src/unit-ubjson_valueless.cpp, which
derives every boundary from the macro and is built with the rates 1,
1000000 and 0xFFFFFFFFFFFFFFFF in addition to the default (the last two
check that nothing is refused and that the limit does not wrap around);
they fail with the unchanged header for rate 1, 1000000 and the maximum.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Refuse char8_t for JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE and clarify its documentation

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

2026-11-11 22:37:35 +01:00

10 KiB

Raw Blame History

Macros

Some aspects of the library can be configured by defining preprocessor macros before including the json.hpp header. See also the macro overview page.

Runtime assertions

JSON_ASSERT(x) - control behavior of runtime assertions

Exceptions

JSON_CATCH_USER(exception)
JSON_THROW_USER(exception)
JSON_TRY_USER - control exceptions
JSON_DIAGNOSTICS - control extended diagnostics
JSON_DIAGNOSTIC_POSITIONS - access positions of elements
JSON_NOEXCEPTION - switch off exceptions

Parsing

JSON_DISABLE_STREAM_BULK_SCAN - read a std::istream one character at a time instead of scanning its stream buffer in bulk
JSON_DISABLE_UNLOCKED_FILE_IO - read a FILE* with std::fgetc instead of locking it once and using the unlocked stdio functions
JSON_PRECISE_STREAM_POSITION - opt in to leaving an input stream positioned right after a parsed number
JSON_STRICT_BINARY_UTF8 - opt in to checking strings for valid UTF-8 in the CBOR, UBJSON, BJData, and BSON writers
JSON_STRICT_NUL_HANDLING - opt in to rejecting a NUL byte in the input instead of treating it as end of input
JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE - set how many elements of UBJSON arrays of null, true, or false an input may announce per byte it has

Language support

JSON_HAS_CPP_11
JSON_HAS_CPP_14
JSON_HAS_CPP_17
JSON_HAS_CPP_20 - set supported C++ standard
JSON_HAS_FILESYSTEM
JSON_HAS_EXPERIMENTAL_FILESYSTEM - control std::filesystem support
JSON_HAS_RANGE_VIEW_CONVERSION - control construction from std::ranges views
JSON_HAS_RANGES - control std::ranges support
JSON_HAS_STATIC_RTTI - control RTTI (run time type information) support
JSON_HAS_STD_FORMAT - control std::format/std::formatter support
JSON_HAS_THREE_WAY_COMPARISON - control 3-way comparison support
JSON_NO_AUTOMATIC_UDLS - do not include the user-defined string literals (UDLs) automatically
JSON_NO_IO - switch off functions relying on certain C++ I/O headers
JSON_NO_SIMD - use only portable code instead of vector instructions (SSE2, SSSE3, NEON)
JSON_NO_THREAD_LOCAL - switch off the use of thread_local storage
JSON_SKIP_UNSUPPORTED_COMPILER_CHECK - do not warn about unsupported compilers
JSON_USE_GLOBAL_UDLS - place user-defined string literals (UDLs) into the global namespace
JSON_USE_SIMDUTF - use the simdutf library to accelerate UTF-8 validation

Library version

JSON_SKIP_LIBRARY_VERSION_CHECK - skip library version check
NLOHMANN_JSON_VERSION_MAJOR
NLOHMANN_JSON_VERSION_MINOR
NLOHMANN_JSON_VERSION_PATCH - library version information

Library namespace

NLOHMANN_JSON_NAMESPACE - full name of the nlohmann namespace
NLOHMANN_JSON_NAMESPACE_BEGIN
NLOHMANN_JSON_NAMESPACE_END - open and close the library namespace
NLOHMANN_JSON_NAMESPACE_NO_VERSION - disable the version component of the inline namespace

Type conversions

JSON_BRACE_INIT_COPY_SEMANTICS - opt in to copy/move semantics for single-element brace initialization
JSON_DISABLE_ENUM_SERIALIZATION - switch off default serialization/deserialization functions for enums
JSON_DISABLE_TUPLE_REFERENCE_CONVERSION - switch off conversion from a one-element tuple of a JSON reference
JSON_USE_IMPLICIT_CONVERSIONS - control implicit conversions
JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS - opt in to storing maps with enum keys as objects

Deprecated functions

JSON_DELETE_DEPRECATED_FUNCTIONS - opt in to deleting the deprecated functions ahead of their removal in version 4.0.0

Comparison behavior

JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON - control comparison of discarded values

Serialization/deserialization macros

Enums

NLOHMANN_JSON_SERIALIZE_ENUM - serialize/deserialize an enum
NLOHMANN_JSON_SERIALIZE_ENUM_STRICT - serialize/deserialize an enum with exceptions

Classes and structs

NLOHMANN_DEFINE_TYPE_INTRUSIVE - serialize/deserialize a non-derived class with private members
NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT - serialize/deserialize a non-derived class with private members; uses default values
NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE - serialize a non-derived class with private members
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE - serialize/deserialize a non-derived class
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT - serialize/deserialize a non-derived class; uses default values
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE - serialize a non-derived class
NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE - serialize/deserialize a derived class with private members
NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT - serialize/deserialize a derived class with private members; uses default values
NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE - serialize a derived class with private members
NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE - serialize/deserialize a derived class
NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT - serialize/deserialize a derived class; uses default values
NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE - serialize a derived class
NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_NAMES - serialize/deserialize a non-derived class with private members; uses custom names
NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT_WITH_NAMES - serialize/deserialize a non-derived class with private members; uses default values; uses custom names
NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE_WITH_NAMES - serialize a non-derived class with private members; uses custom names
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_NAMES - serialize/deserialize a non-derived class; uses custom names
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT_WITH_NAMES - serialize/deserialize a non-derived class; uses default values; uses custom names
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_WITH_NAMES - serialize a non-derived class; uses custom names
NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_NAMES - serialize/deserialize a derived class with private members; uses custom names
NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT_WITH_NAMES - serialize/deserialize a derived class with private members; uses default values; uses custom names
NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE_WITH_NAMES - serialize a derived class with private members; uses custom names
NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_NAMES - serialize/deserialize a derived class; uses custom names
NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT_WITH_NAMES - serialize/deserialize a derived class; uses default values; uses custom names
NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_WITH_NAMES - serialize a derived class; uses custom names

10 KiB Raw Blame History