* Bound the total number of valueless UBJSON elements per input An optimized UBJSON array of type Z, T or F has no payload, so its declared count is not bounded by the input. The limit of 1048576 applied to each array on its own, so 9 bytes (`[$Z#l` and a count) produced 1048576 values and every further 9 bytes another 1048576: 362 bytes allocated 661 MB in a DOM, and a SAX handler received 1M events per 9 bytes. The reader now adds the counts of all such arrays of one input and rejects the input with out_of_range.408 when the sum exceeds 1048576 plus 4 for every byte read so far. The base is the per-array limit, so an array that was accepted on its own before is still accepted at the start of an input. These arrays add about 80 bytes of memory per input byte (measured 77; one `Z` per byte needs 32), plus the base; the memory of a document as a whole is not bounded by this. The check runs once per valueless array, before its start event, with plain size_t arithmetic that cannot overflow for a 32-bit size_t. Containers with a payload type and objects (every member needs a key) are bounded by the input already and are unchanged. BJData rejects these types. to_ubjson counts the elements of the typed arrays it writes in one call and writes an array in the unoptimized form (`[#n Z Z ...`) once it would bring the total above 1048576, which a reader accepts at any input size. The check is made only for a typed array of a valueless type, so BJData and other arrays do not pay for it. Everything to_ubjson writes in version 3.13.0 can therefore be read back. Two consequences: - to_ubjson output changes for values with more than 1048576 elements in arrays of type Z, T or F: such an array is written at one byte per element, e.g. 270000 rows of 100 nulls take 26.8 MB instead of 1.35 MB. - Output of version 3.12.0 and earlier (typed arrays without such a total) is rejected by from_ubjson when the arrays hold more than 1048576 elements plus 4 per byte of input. 3.12.0 writes a row of up to 255 nulls in 5 bytes (6 above 255), so a table is rejected from 1075 rows of 1000 nulls or 209723 rows of 25 (accepted: 1000 rows of 1000, 209000 rows of 25); rows of 20 or fewer are never rejected. Measured on the test data: no file of json_test_data (converted with to_ubjson in all three modes, 846 round trips) holds an array of this kind with more than 0.17 values per byte. All of them are read back. Reproducer (K arrays of 1048576 nulls), Apple clang -O2: K=40 (362 bytes): before 41943040 values, 664 MB; after rejected, 33 MB K=2 (20 bytes): before 2097152 values, 53 MB; after rejected, 33 MB The new tests count events with a SAX handler where they can: building and comparing million-element values made the first version take 176 s under Valgrind, the current one about 22 s. Behaviour change: an input that holds more than 1048576 elements in arrays of type Z, T or F at more than 4 per byte is now rejected. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Report an empty array for UBJSON `[$N#<n>` instead of n elements A no-op (N) is not a value, so an optimized array of type N holds no element whatever count it declares. The reader skipped the elements but still passed the declared count to the SAX parser: a handler was told start_array(n) and no element followed, and the DOM parser reserved min(n, 16384) values (256 KB) for an array that stays empty. Nine bytes (`[$N#l 0x7FFFFFFF`) therefore committed 256 KB: 2000 of them in one input, 18 KB, committed 512 MB of heap (capacity 163,840,000 values for the 10000 arrays of the original probe: 2.6 GB for 70 KB of input). Untouched pages keep the resident size small, but the commit charge and a 32-bit address space are consumed. The reader now announces 0 elements for an optimized array of type N. The loop that skips the elements is unchanged (it never ran). A count that does not fit into std::size_t is still rejected by the count reader; a count that is too large for the array type of the DOM (before: out_of_range.408 "excessive array size" from the DOM handler) is accepted now, because the array is empty. Behaviour change: sax_parse reports start_array(0) for `[$N#n` (before: n); from_ubjson no longer throws out_of_range.408 for an N-typed array whose count exceeds the maximum size of the array type. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bound what the DOM parsers reserve for arrays that announce a size Cause: #5476 made the DOM parsers reserve storage (up to 16384 elements) for a counted array of CBOR, MessagePack, UBJSON and BJData when it opens, before any input backs the count. The arrays that are open at the same time keep their storage until the parse ends, so N nested headers that announce 16384 elements and are never completed hold N * 16384 * sizeof(json): 100,000 nested CBOR headers `99 40 00` (300,000 bytes) peaked at 1.65 GiB, 3.12.0 at 12 MiB; the same for MessagePack `DC 40 00`, UBJSON `[#I 16384` and BJData. Fix: json_sax_dom_error_state, the base of the three DOM parsers, keeps a credit of elements that may still be reserved: 65,536 at the start plus 2 for every byte the reader has read. An array that announces no more than the credit left is reserved as before (one comparison and one subtraction per counted array); a larger one is reserved what is left and grows as its elements arrive. binary_reader tells a DOM parser (a handler whose pointer converts to json_sax_dom_error_state*; a handler that derives from one privately or protectedly, or any other SAX handler, is not told anything and reserves as in the base of this change) where to find its count of characters read for the duration of sax_parse(). An RAII guard puts back the previous binding on every exit, also by exception, so the parser never keeps a pointer to a reader that is gone (unbound, only the per-array cap of 16384 applies); the public SAX interface is unchanged, and start_array() still reports the announced number. A bound per array by the remaining input would not help, since the same bytes would back every open level; the bound has to be over all arrays. The elements of an array take at least one byte of input each, so the credit is more than honest input needs: none of the 19 inputs measured (canada, citm_catalog, twitter, mesh, update-center, random, numbers, animation, ik_rig, instruments, geo_response, jeopardy and 7 synthetic ones, each as CBOR, MessagePack, UBJSON and BJData, with and without size/type optimization) was given less than it announced. A value that nests five or more arrays of 16384 elements, each with an array as its first element, is the exception: the later ones grow as before. Measurements (clang -O2, macOS arm64, peak memory footprint of `/usr/bin/time -l`, reproducer: n nested headers): n = 20,000 n = 100,000 develop fix 3.12.0 develop fix 3.12.0 CBOR 338.7 5.9 8.2 1687.7 23.3 11.8 MiB MessagePack 338.9 6.0 7.3 1687.9 23.8 11.9 MiB UBJSON 339.0 7.6 8.0 1687.9 30.1 11.6 MiB BJData 339.0 7.3 7.9 1688.0 30.8 11.6 MiB That is at most (65,536 + 2 per input byte) values: 1 MiB plus 32 bytes per input byte with 16-byte values (a value is 24 bytes with JSON_DIAGNOSTICS and up to 40 with JSON_DIAGNOSTIC_POSITIONS), as documented in untrusted_input.md. Lower fixed caps do not get there: a cap of 64 elements still peaks at 113 MiB (n = 100,000) and costs 4.7 % on arrays of 300 elements (mat300.cbor) and 5.0 % on arrays of 10,000 (arrays_of_big.cbor). Reader instructions retired (min of 7, 108 rows: 18 inputs in the 6 forms CBOR, MessagePack, UBJSON and BJData plain and optimized; jeopardy, 46 MB, is within 0.1 %), compared with the base of this change: -0.19 % to +0.37 % in the formats that announce sizes (canada.cbor 81.9 M -> 82.1 M, many_small.cbor 333.5 M -> 334.2 M); the benefit of #5476 is kept (without any reserve canada.cbor takes 116.8 M, +42.6 %, many_small.cbor +71.5 %, mat300.cbor +81.6 %). Rows of plain UBJSON and BJData, whose arrays announce no size, move by -2.7 % to +2.6 % without any code of theirs changing: the base alternates between two instruction counts per process (452 M and 463 M for many_pairs.ubjson, equal in cycles), and a struct with three unused members added to the same class reproduces the 474 M mode. Tests (CBOR, MessagePack, UBJSON, BJData): 200 nested headers that announce 16384 elements peak at 52,433,600 bytes of DOM allocations on the base and below 128 Ki values with the fix (scaled by sizeof(value), so that it holds with the diagnostic macros) (counting allocator, no process memory measured); an array that completes is still reserved in one allocation; CBOR also checks that a DOM parser is not bound to a finished parse (ASan: stack-use-after-scope without the guard), that wrappers derived publicly (also with a parse inside the parse), privately and protectedly compile and are bound or not as described, that an array that got less than it announced decodes and that a SAX handler is told the announced size. The new tests take 4.6 s under Valgrind at -O0 (gcc:12, all four files). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Clarify the reservation note for announced array counts Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Report an empty array for UBJSON [$N#<n>] instead of n elements (#6079) * Report an empty array for UBJSON `[$N#<n>` instead of n elements A no-op (N) is not a value, so an optimized array of type N holds no element whatever count it declares. The reader skipped the elements but still passed the declared count to the SAX parser: a handler was told start_array(n) and no element followed, and the DOM parser reserved min(n, 16384) values (256 KB) for an array that stays empty. Nine bytes (`[$N#l 0x7FFFFFFF`) therefore committed 256 KB: 2000 of them in one input, 18 KB, committed 512 MB of heap (capacity 163,840,000 values for the 10000 arrays of the original probe: 2.6 GB for 70 KB of input). Untouched pages keep the resident size small, but the commit charge and a 32-bit address space are consumed. The reader now announces 0 elements for an optimized array of type N. The loop that skips the elements is unchanged (it never ran). A count that does not fit into std::size_t is still rejected by the count reader; a count that is too large for the array type of the DOM (before: out_of_range.408 "excessive array size" from the DOM handler) is accepted now, because the array is empty. Behaviour change: sax_parse reports start_array(0) for `[$N#n` (before: n); from_ubjson no longer throws out_of_range.408 for an N-typed array whose count exceeds the maximum size of the array type. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Clarify the reservation note for announced array counts Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE to set the UBJSON whole-input budget (#6083) * Add JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE to set the UBJSON whole-input budget The whole-input budget for the elements of optimized UBJSON arrays of type Z, T and F (1048576 plus 4 per byte read) rejects files written by 3.12.0 that have long arrays of these types, such as 1100 rows of 1000 nulls with a `$Z` annotation, and there was no way to read them. The per-byte rate is now the macro JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE. Semantics: - default 4, so the default build is unchanged (the generated code of from_ubjson is identical at -O2 up to source line numbers); - the value must be a positive integer: 0, negative, non-integral, bool, character and wider-than-uintmax_t values fail to compile with a static_assert that names the macro; - a value that does not fit into std::size_t is the largest size_t, so one spelling works for 32- and 64-bit targets; - the limit base + rate * bytes_read saturates at the largest size_t instead of wrapping around, for any rate and input size; a rate of 131072 (2^20 / 8) or more cannot be exceeded by any input (the shortest header of an array of more than 32767 elements is 8 bytes), which switches the whole-input budget off; - the limit of one array (1048576) and the base (1048576) stay fixed constants, so the writer fallback still follows the base only and a reader with any rate reads everything the writer produces; - the macro is not part of the ABI tag and must have the same value in every translation unit; it is documented that way. The memory the arrays can take in a DOM is about 20 bytes per rate and byte of input (measured, macOS, containers of 9 * rate elements: 25 bytes per input byte at rate 1, 81 at 4, 312 at 16, 1226 at 64, 5493 at 256) plus about 33 MB for the base. A 3.12.0 row of N values has a header of b bytes (5 up to 255, 6 up to 32767, 8 above), so a table is read completely if N <= rate * b, and otherwise up to (1048576 + h * rate, h = outer header: 6, 7 or 9 bytes) / (N - rate * b) rows; 1100 rows of 1000 nulls need a rate of 8. The new page of the macro documents the formulas, the measurements, and two examples that are run by the documentation checks. The tests of the budget move to tests/src/unit-ubjson_valueless.cpp, which derives every boundary from the macro and is built with the rates 1, 1000000 and 0xFFFFFFFFFFFFFFFF in addition to the default (the last two check that nothing is refused and that the limit does not wrap around); they fail with the unchanged header for rate 1, 1000000 and the maximum. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Refuse char8_t for JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE and clarify its documentation Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use fseek instead of rewind in the UBJSON FILE* test clang-tidy (bugprone-unsafe-functions) rejects rewind() because it has no error detection. Check the result of fseek() instead. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Charge an announced array only what is reserved for it An array that announced more than reserve_cap() elements but no more than the credit left was charged the announced count, while reserve_array() reserved at most reserve_cap() of them. An outer array of 20000 elements therefore left a third nested array of 16384 with 12796 instead of the full reservation. reserve_count() now caps the count first and charges that, and the DOM parsers reserve it through reserve_array_capped(), which does not cap again, so the fast path costs the same as before (instructions retired within +-0.06 % on canada, citm_catalog and twitter in CBOR, MessagePack and UBJSON). Also re-wrap the comment of valueless_elements_base. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
17 KiB
UBJSON
Universal Binary JSON (UBJSON) is a binary form directly imitating JSON, but requiring fewer bytes of data. It aims to achieve the generality of JSON, combined with being much easier to process than JSON.
!!! abstract "References"
- [UBJSON Website](https://ubjson.org)
Byte layout
UBJSON is built from just a one-byte, human-readable ASCII type marker (for example, i for int8, S for string, [ for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an unoptimized container that is opened with [ or { and closed with ] or }, or as an optimized container that declares a count (#) after the opening marker, optionally preceded by an element type ($); an element type is only allowed together with a count. An optimized container saves the closing marker and, with a type, the per-element markers.
Consider the following JSON value (shown formatted for readability):
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
to_ubjson encodes it to 51 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: type/marker length/count/size data.
| JSON | Bytes | Meaning |
|---|---|---|
{ | 7B | object start ({) |
"id": 300, | 69 02 69 64 49 01 2C | key: length 2 as int8 (i), then UTF-8 bytes; value: int16 (I), big-endian |
"name": "json", | 69 04 6E 61 6D 65 53 69 04 6A 73 6F 6E | key: length 4; value: string (S), length 4 as int8, UTF-8 bytes |
"pi": 3.14, | 69 02 70 69 44 40 09 1E B8 51 EB 85 1F | key: length 2; value: float64 (D), big-endian |
"tags": [ | 69 04 74 61 67 73 5B | key: length 4; value: array start ([) |
"a", | 53 69 01 61 | string, length 1 |
"b" | 53 69 01 62 | string, length 1 |
] | 5D | array end (]) |
} | 7D | object end (}) |
Serialization
The library uses the following mapping from JSON values types to UBJSON types according to the UBJSON specification:
| JSON value type | value/range | UBJSON type | marker |
|---|---|---|---|
| null | null |
null | Z |
| boolean | true |
true | T |
| boolean | false |
false | F |
| number_integer | -9223372036854775808..-2147483649 | int64 | L |
| number_integer | -2147483648..-32769 | int32 | l |
| number_integer | -32768..-129 | int16 | I |
| number_integer | -128..127 | int8 | i |
| number_integer | 128..255 | uint8 | U |
| number_integer | 256..32767 | int16 | I |
| number_integer | 32768..2147483647 | int32 | l |
| number_integer | 2147483648..9223372036854775807 | int64 | L |
| number_unsigned | 0..127 | int8 | i |
| number_unsigned | 128..255 | uint8 | U |
| number_unsigned | 256..32767 | int16 | I |
| number_unsigned | 32768..2147483647 | int32 | l |
| number_unsigned | 2147483648..9223372036854775807 | int64 | L |
| number_unsigned | 9223372036854775808..18446744073709551615 | high-precision | H |
| number_float | any value | float64 (float32 if number_float_t is float) |
D (d) |
| string | with shortest length indicator | string | S |
| array | see notes on optimized format | array | [ |
| object | see notes on optimized format | object | { |
!!! success "Complete mapping"
The mapping is **complete** in the sense that any JSON value type can be converted to a UBJSON value.
Any UBJSON output created by `to_ubjson` can be successfully parsed by `from_ubjson`.
!!! warning "Size constraints"
The following values can **not** be converted to a UBJSON value:
- strings with more than 9223372036854775807 bytes (theoretical)
!!! warning "UTF-8 validation of string values and object keys"
UBJSON's required string encoding is UTF-8. By default (the [`error_handler`](../../api/basic_json/to_ubjson.md)
parameter left at `keep`), `to_ubjson()` writes the bytes of string values and object keys unchanged, even if they
are not valid UTF-8. With `error_handler_t::strict`, it throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for ill-formed UTF-8 instead;
`replace`/`ignore` sanitize the string. [`JSON_STRICT_BINARY_UTF8`](../../api/macros/json_strict_binary_utf8.md)
makes `strict` the default.
!!! info "Unused UBJSON markers"
The following markers are not used in the conversion:
- `N`: no-op values are never written (they are skipped when reading).
- `C`: single-byte strings are serialized with `S` markers.
!!! info "NaN/infinity handling"
NaN and infinity are written as IEEE 754 float64 values (`D`) and read back unchanged. This differs from
[`dump()`](../../api/basic_json/dump.md), which serializes them to `null`, and from the UBJSON specification, which
encodes infinity as `null` (it does not mention NaN); other UBJSON readers may therefore not accept these values.
!!! info "Optimized formats"
The optimized formats for containers are supported: Parameter `use_size` adds size information to the beginning of a
container and removes the closing marker. Parameter `use_type` further checks whether all elements of a container
have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be
used together with `use_size = true`: combining `use_type` with `use_size = false` throws
[`other_error.502`](../../home/exceptions.md#jsonexceptionother_error502) if the value contains a non-empty array,
object, or binary value.
Two elements have the same type if they are written with the same type marker. For example, `[1, 2]` is written with
the type `i`, but `[1, 200]` (`i`, `U`), `[true, false]` (`T`, `F`), and `[1, 2.5]` (`i`, `D`) are not; an empty
container is never given a `$` annotation of its own (an array of empty arrays is still typed `[`).
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the
receiving side is immediately informed on the number of elements of the container.
An array whose type marker is `Z` (null), `T` (true) or `F` (false) stores no payload at all, because the marker
already is the value. Its declared count is therefore the only thing that decides how much memory the receiving
side allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
[`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576
(`1 << 20`), or when the counts of all such arrays in the input add up to more than 1,048,576 plus _K_ for every byte
read so far, where _K_ is the value of the macro
[`JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE`](../../api/macros/json_ubjson_valueless_elements_per_byte.md) (4 by
default). `to_ubjson` therefore writes an array of these types in the counted but untyped form when it is longer
than that, or when it would bring the total of the typed arrays written so far above 1,048,576: the `$` type
marker is dropped, the `#` count is kept, and every element is written with its own marker
(`[#l 1048577 Z Z ...`), so everything it produces can be read back. Only input from another encoder, or written by version 3.12.0
and earlier of this library, can exceed the limit: for example, 1,100 rows of 1,000 `null` values or 210,000 rows
of 25, each with a `$Z` annotation. Version 3.12.0 writes a row in 5 bytes (6 above 255 elements, 8 above 32,767),
which raises the limit by 5 _K_ (6 _K_, 8 _K_), so rows of up to 5 _K_ values (20 at the default) are never
rejected. To read such a file, define the macro to a larger value (`8` is enough for the 1,100 rows of 1,000
`null` values; memory for the arrays then rises to about 20 _K_ bytes per byte of input in a DOM); the macro's
page explains how to choose a value and what it costs. The limit of one array of 1,048,576 elements does not
depend on the macro. Objects and untyped counted arrays are not subject to this limit.
An optimized array of type `N` (no-op) is empty whatever its count is, because a no-op is not a value. Its count is
not passed on: `sax_parse` reports `start_array(0)` (versions before 3.13.0 reported the declared count), and the DOM
parser reserves nothing for it.
An optimized array of type `N` (no-op) is empty whatever its count is, because a no-op is not a value. Its count is
not passed on: `sax_parse` reports `start_array(0)` (versions before 3.13.0 reported the declared count), and the DOM
parser reserves nothing for it.
!!! info "Binary values"
If the JSON data contains the binary type, the value stored is a list of integers, as suggested by the UBJSON
documentation. In particular, this means that serialization and the deserialization of a JSON containing binary
values into UBJSON and back will result in a different JSON object.
??? example "Example: serialize JSON values to UBJSON, with and without size/type optimization"
```cpp
--8<-- "examples/to_ubjson.cpp"
```
Output:
```c
--8<-- "examples/to_ubjson.output"
```
Deserialization
The library maps UBJSON types to JSON value types as follows:
| UBJSON type | JSON value type | marker |
|---|---|---|
| no-op | no value, next value is read | N |
| null | null |
Z |
| false | false |
F |
| true | true |
T |
| float32 | number_float | d |
| float64 | number_float | D |
| uint8 | number_unsigned | U |
| int8 | number_integer | i |
| int16 | number_integer | I |
| int32 | number_integer | l |
| int64 | number_integer | L |
| string | string | S |
| char | string | C |
| high-precision | number_unsigned, number_integer, or number_float | H |
| array | array (optimized values are supported) | [ |
| object | object (optimized values are supported) | { |
!!! info "High-precision numbers"
The payload of `H` is a length-prefixed string that is parsed as a JSON number (surrounding whitespace is ignored).
Integer text in the range of `number_unsigned_t` (non-negative) or `number_integer_t` (negative) is stored as such;
any other valid number text (larger or smaller integers, fractions, and exponents such as `1e2`) is stored as
`number_float`, so precision may be lost. Text that is not a valid JSON number raises
[`parse_error.115`](../../home/exceptions.md#jsonexceptionparse_error115), a number outside the range of
`number_float_t` (`1e999`) raises [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406).
!!! success "Complete mapping"
Every well-formed UBJSON value can be converted to a JSON value, with these exceptions: a high-precision number
outside the range of `number_float_t` ([`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406)),
optimized arrays of `Z`, `T`, or `F` with more elements than the limit allows (1,048,576 plus
[`JSON_UBJSON_VALUELESS_ELEMENTS_PER_BYTE`](../../api/macros/json_ubjson_valueless_elements_per_byte.md), 4 by default,
per byte read),
and a count that exceeds the capacity of the container
([`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408)). High-precision numbers that do not
fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected with a
parse error.
!!! info "Duplicate object keys"
If an object contains the same key more than once, the last value is kept (the UBJSON specification does not
address duplicate keys).
!!! info "Type marker of optimized containers"
The type marker after `$` must be one of `Z`, `N`, `T`, `F`, `i`, `U`, `I`, `l`, `L`, `d`, `D`, `H`, `C`, `S`, `[`, or
`{`. It is checked even if the count is 0; any other marker raises
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112). A counted object typed `N` (`{$N#i1 ...`)
is not supported.
!!! warning "Ill-formed UTF-8 in string values and object keys"
UBJSON's required string encoding is UTF-8, but checking it on read is opt-in: with the
[`error_handler`](../../api/basic_json/from_ubjson.md) parameter left at `keep` (the default), `from_ubjson()`
accepts a string value or object key whose bytes are not valid UTF-8 and hands them back unchanged. Passing
`error_handler_t::strict` makes `from_ubjson()` check and throw
[`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) for ill-formed UTF-8, and
`replace`/`ignore` sanitize the string instead of keeping it. However,
[`dump()`](../../api/basic_json/dump.md) still requires valid UTF-8 and throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for a value read with the default
`keep` handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. `to_ubjson()`'s
own `error_handler` parameter defaults to `keep` (see above), so such a value is written back unchanged.
??? example "Example: deserialize a JSON value from UBJSON"
```cpp
--8<-- "examples/from_ubjson.cpp"
```
Output:
```json
--8<-- "examples/from_ubjson.output"
```
See also
- Binary formats - overview and comparison of the supported binary formats
- Binary values - how binary values and subtypes map to UBJSON
to_ubjson- serialize a JSON value to UBJSONfrom_ubjson- parse a JSON value from UBJSONJSON_STRICT_BINARY_UTF8- reject ill-formed UTF-8 in strings- SAX interface - decode binary formats without building a value