wt/json
1
0 Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-27 12:25:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

deploy: 94fe31eb9b

Browse Source
This commit is contained in:
nlohmann committed 2026-10-26 21:24:35 +00:00
1 parent 41c601a7d1
commit d2e264bd84
1400 files changed
+11859 -5440
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

features/binary_formats/bjdata.md
+172 -126
View File
@@ -2,9 +2,9 @@
The [BJData format](https://neurojson.org) was derived from and improved upon
[Universal Binary JSON(UBJSON)](https://ubjson.org) specification (Draft 12). Specifically, it introduces an optimized
array container for efficient storage of N-dimensional packed arrays (**ND-arrays**); it also adds 5 new type markers -
`[u] - uint16`, `[m] - uint32`, `[M] - uint64`, `[h] - float16` and `[B] - byte` - to unambiguously map common binary
numeric types; furthermore, it uses little-endian (LE) to store all numerics instead of big-endian (BE) as in UBJSON to
array container for efficient storage of N-dimensional packed arrays (**ND-arrays**); it also adds four new type markers -
`[u] - uint16`, `[m] - uint32`, `[M] - uint64`, and `[h] - float16` - to unambiguously map common binary numeric types
(Draft 3 adds `[B] - byte`); furthermore, it uses little-endian (LE) to store all numerics instead of big-endian (BE) as in UBJSON to
avoid unnecessary conversions on commonly available platforms.
Compared to other binary JSON-like formats such as MessagePack and CBOR, both BJData and UBJSON demonstrate a rare
@@ -15,41 +15,93 @@ simple processing.
!!! abstract "References"
- [BJData Specification](https://neurojson.org/bjdata/draft2)
- [BJData Specification (Draft 3)](https://neurojson.org/bjdata/draft3)
- [BJData Specification (Draft 2)](https://neurojson.org/bjdata/draft2)
!!! info "Supported drafts"
The library supports BJData Draft 2 (the default) and Draft 3, except for column-major N-dimensional arrays (see
below). The draft used for writing is selected with the
[`version` parameter](../../api/basic_json/bjdata_version_t.md) of
[`to_bjdata`](../../api/basic_json/to_bjdata.md); [`from_bjdata`](../../api/basic_json/from_bjdata.md) has no such
parameter and reads the markers of both drafts.
!!! warning "Draft 4 and column-major ND-arrays are not supported"
BJData Draft 4 (April 2026) adds structure-of-arrays (SoA) containers (`[$` or `{$` followed by a schema object)
and the extension type `E`. `from_bjdata` does not implement them and throws
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) for such input; `to_bjdata` never writes
them.
Likewise, column-major ND-arrays of Draft 3 (the `#` marker followed by an array that wraps the dimension vector)
are rejected with [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113).
## Byte layout
BJData follows the structure of UBJSON: a one-byte, human-readable ASCII *type marker* followed by the payload of that type, with strings prefixed by a length that is itself an integer value. In contrast to UBJSON, numbers are stored in little-endian byte order, and additional markers cover unsigned 16/32/64-bit integers, half-precision floats, and bytes. Containers can be written unoptimized (opened with `[` or `{`, closed with `]` or `}`) or optimized with a count (`#`), optionally preceded by a declared element type (`$`). BJData extends the optimized form: the count can be a list of dimensions, which stores an N-dimensional packed array as a flat sequence of values without any per-element markers.
Consider the following JSON value (shown formatted for readability):
```json
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_bjdata` encodes it to 51 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.
<div class="byte-layout" markdown="0"><table>
<thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody>
<tr><td><code>{</code></td><td><span class="bl bl-t">7B</span></td><td>object start (<code>{</code>)</td></tr>
<tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">49</span> <span class="bl bl-d">2C 01</span></td><td>key: length 2 as int8 (<code>i</code>), then UTF-8 bytes; value: int16 (<code>I</code>), little-endian</td></tr>
<tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">53</span> <span class="bl bl-l">69 04</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: length 4; value: string (<code>S</code>), length 4 as int8, UTF-8 bytes</td></tr>
<tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">44</span> <span class="bl bl-d">1F 85 EB 51 B8 1E 09 40</span></td><td>key: length 2; value: float64 (<code>D</code>), little-endian</td></tr>
<tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">5B</span></td><td>key: length 4; value: array start (<code>[</code>)</td></tr>
<tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">61</span></td><td>string, length 1</td></tr>
<tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">62</span></td><td>string, length 1</td></tr>
<tr><td><code>]</code></td><td><span class="bl bl-t">5D</span></td><td>array end (<code>]</code>)</td></tr>
<tr><td><code>}</code></td><td><span class="bl bl-t">7D</span></td><td>object end (<code>}</code>)</td></tr>
</tbody></table></div>
## Serialization
The library uses the following mapping from JSON values types to BJData types according to the BJData specification:
| JSON value type | value/range | BJData type | marker |
|-----------------|-------------------------------------------|----------------|--------|
| null | `null` | null | `Z` |
| boolean | `true` | true | `T` |
| boolean | `false` | false | `F` |
| number_integer | -9223372036854775808..-2147483649 | int64 | `L` |
| number_integer | -2147483648..-32769 | int32 | `l` |
| number_integer | -32768..-129 | int16 | `I` |
| number_integer | -128..127 | int8 | `i` |
| number_integer | 128..255 | uint8 | `U` |
| number_integer | 256..32767 | int16 | `I` |
| number_integer | 32768..65535 | uint16 | `u` |
| number_integer | 65536..2147483647 | int32 | `l` |
| number_integer | 2147483648..4294967295 | uint32 | `m` |
| number_integer | 4294967296..9223372036854775807 | int64 | `L` |
| number_integer | 9223372036854775808..18446744073709551615 | uint64 | `M` |
| number_unsigned | 0..127 | int8 | `i` |
| number_unsigned | 128..255 | uint8 | `U` |
| number_unsigned | 256..32767 | int16 | `I` |
| number_unsigned | 32768..65535 | uint16 | `u` |
| number_unsigned | 65536..2147483647 | int32 | `l` |
| number_unsigned | 2147483648..4294967295 | uint32 | `m` |
| number_unsigned | 4294967296..9223372036854775807 | int64 | `L` |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | `M` |
| number_float | *any value* | float64 | `D` |
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format/ND-array* | array | `[` |
| object | *see notes on optimized format* | map | `{` |
| binary | *see notes on binary values* | array | `[$B` |
| JSON value type | value/range | BJData type | marker |
|-----------------|-------------------------------------------|-------------|-------------------------------------------------------------------------|
| null | `null` | null | `Z` |
| boolean | `true` | true | `T` |
| boolean | `false` | false | `F` |
| number_integer | -9223372036854775808..-2147483649 | int64 | `L` |
| number_integer | -2147483648..-32769 | int32 | `l` |
| number_integer | -32768..-129 | int16 | `I` |
| number_integer | -128..127 | int8 | `i` |
| number_integer | 128..255 | uint8 | `U` |
| number_integer | 256..32767 | int16 | `I` |
| number_integer | 32768..65535 | uint16 | `u` |
| number_integer | 65536..2147483647 | int32 | `l` |
| number_integer | 2147483648..4294967295 | uint32 | `m` |
| number_integer | 4294967296..9223372036854775807 | int64 | `L` |
| number_integer | 9223372036854775808..18446744073709551615 | uint64 | `M` |
| number_unsigned | 0..127 | int8 | `i` |
| number_unsigned | 128..255 | uint8 | `U` |
| number_unsigned | 256..32767 | int16 | `I` |
| number_unsigned | 32768..65535 | uint16 | `u` |
| number_unsigned | 65536..2147483647 | int32 | `l` |
| number_unsigned | 2147483648..4294967295 | uint32 | `m` |
| number_unsigned | 4294967296..9223372036854775807 | int64 | `L` |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | `M` |
| number_float | *any value* | float64 | `D` |
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format/ND-array* | array | `[` |
| object | *see notes on optimized format* | map | `{` |
| binary | *see notes on binary values* | array | `[$B` (Draft 3 with `use_size` and `use_type`), `[U`, or `[B` otherwise |
!!! success "Complete mapping"
@@ -74,10 +126,16 @@ The library uses the following mapping from JSON values types to BJData types ac
!!! info "Unused BJData markers"
The following markers are not used in the conversion:
The following markers are not written:
- `Z`: no-op values are not created.
- `C`: single-byte strings are serialized with `S` markers.
- `N`: no-op values are not created (they are skipped when reading).
- `C`: single-character strings are serialized with `S` markers (`C` is read, and written for the elements of
ND-arrays with `"_ArrayType_": "char"`).
- `h`: floating-point numbers are written as `D`, not as float16 (`h` is read).
- `B`: only written in Draft 3 mode.
- `d` is only written for `single` ND-arrays or if `number_float_t` is `float`; `H` is only written for integers
wider than 64 bits (custom `number_integer_t`/`number_unsigned_t`). An unsigned integer above 9223372036854775807
is written as `M`.
!!! info "NaN/infinity handling"
@@ -109,7 +167,7 @@ The library uses the following mapping from JSON values types to BJData types ac
BJData extends UBJSON's optimized array **size** marker to support ND-arrays of uniform numerical data types
(referred to as *packed arrays*). For example, the 2-D `uint8` integer array `[[1,2],[3,4],[5,6]]`, stored as nested
optimized array in UBJSON `[ [$U#i2 1 2 [$U#i2 3 4 [$U#i2 5 6 ]`, can be further compressed in BJData to
`[$U#[$i#i2 2 3 1 2 3 4 5 6` or `[$U#[i2 i3] 1 2 3 4 5 6`.
`[$U#[$i#i2 3 2 1 2 3 4 5 6` or `[$U#[i3 i2] 1 2 3 4 5 6`.
To maintain type and size information, ND-arrays are converted to JSON objects following the **annotated array
format** (defined in the [JData specification (Draft 3)][JDataAAFmt]), when parsed using
@@ -119,11 +177,14 @@ The library uses the following mapping from JSON values types to BJData types ac
```json
{
"_ArrayType_": "uint8",
"_ArraySize_": [2,3],
"_ArraySize_": [3,2],
"_ArrayData_": [1,2,3,4,5,6]
}
```
ND-arrays with the element type `h` (float16) are returned with `"_ArrayType_": "half"`. `to_bjdata` does not pack
`"half"` objects; they are written as regular objects, which read back as an equal value.
Likewise, when a JSON object in the above form is serialized using
[`to_bjdata`](../../api/basic_json/to_bjdata.md), it is automatically converted into a compact BJData ND-array.
@@ -134,34 +195,40 @@ The library uses the following mapping from JSON values types to BJData types ac
object; otherwise it is serialized as a regular JSON object, so the annotation is never lost in a round trip. This requires
all of the following:
- the object has exactly the three keys `"_ArrayType_"`, `"_ArraySize_"`, and `"_ArrayData_"` (any other key, e.g.,
`"_ArrayOrder_"`, makes it a regular object),
- `"_ArrayType_"` is one of `uint8`, `int8`, `uint16`, `int16`, `uint32`, `int32`, `uint64`, `int64`, `single`,
`double`, `char`, or `byte`,
`double`, `char`, or (only with `version = bjdata_version_t::draft3`, since the `B` marker is not part of Draft 2)
`byte`; the names are case-sensitive,
- `"_ArraySize_"` is an array, since the dimensions are written as the ND-array header's length,
- `"_ArraySize_"` has at least two entries and is not a 1×N row vector (first entry 1), since other shapes are
parsed back as a regular array,
- `"_ArraySize_"` has at least two entries and is not a two-entry row vector (`[1,N]`), since other shapes are
parsed back as a regular array (`[1,2,2]` is packed, `[1,4]` is not),
- every entry of `"_ArraySize_"` is a positive integer, and their product is representable as a `std::size_t`,
- `"_ArrayData_"` is an array holding exactly that many elements, and
- every element of `"_ArrayData_"` is a number of the kind named by `"_ArrayType_"`: for the integer types, a
value that fits the named width; for `double`, any value; for `single`, a value that survives narrowing to
`float` and back without change (for instance, `0.1` does not, since it is not exactly representable as
`float`).
- every element of `"_ArrayData_"` is a number of the kind named by `"_ArrayType_"`: for the integer types, an
integer (a floating-point number such as `1.0` does not qualify) that fits the named width (for `char`, 0..127);
for `double`, a floating-point number (an integer such as `1` does not qualify); for `single`, a floating-point
number that survives narrowing to `float` and back without change (for instance, `0.1` does not, since it is not
exactly representable as `float`).
An annotated object is always read back with its keys in the order shown above, `"_ArrayType_"`, `"_ArraySize_"`,
`"_ArrayData_"`, regardless of the order the ND-array's header stores them in on the wire. This matters for
`ordered_json`, whose comparison takes key order into account.
With `ordered_json`, an annotated object read from BJData has its keys in the order `"_ArrayType_"`,
`"_ArraySize_"`, `"_ArrayData_"`; with `json`, they are sorted like those of any other object.
The current version of this library does not yet support automatic detection of and conversion from a nested JSON
array input to a BJData ND-array.
[JDataAAFmt]: https://github.com/NeuroJSON/jdata/blob/master/JData_specification.md#annotated-storage-of-n-d-arrays
[JDataAAFmt]: https://github.com/NeuroJSON/jdata/blob/Draft-3/JData_specification.md#annotated-storage-of-n-d-arrays
!!! info "Restrictions in optimized data types for arrays and objects"
Due to diminished space saving, hampered readability, and increased security risks, in BJData, the allowed data
types following the `$` marker in an optimized array and object container are restricted to
Due to diminished space saving, hampered readability, and increased security risks, in BJData, the
allowed data types following the `$` marker in an optimized array and object container are restricted to
**non-zero-fixed-length** data types. Therefore, the valid optimized type markers can only be one of
`UiuImlMLhdDCB`. This also means other variable (`[{SH`) or zero-length types (`TFN`) can not be used in an
optimized array or object in BJData.
`UiuImlMLhdDC`, plus `B` since Draft 3 (the reader accepts `B` in both drafts). This also means other variable (`[{SH`) or zero-length types (`TFZN`) can not be used in an
optimized array or object in BJData. The library enforces this restriction for both drafts. A `C` (char) element above 127 is rejected on read with
[`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113), in 1-D and in N-dimensional arrays. With `use_type`, the
elements must have the same marker (`[1,200,3]` is not optimized, since it mixes `i` and `U`); booleans, null,
strings, arrays, and objects are never optimized with `$`.
!!! info "Binary values"
@@ -169,14 +236,16 @@ The library uses the following mapping from JSON values types to BJData types ac
in optimized arrays to designate binary data. This means that, unlike UBJSON, binary data can be both serialized and
deserialized.
To preserve compatibility with BJData Draft 2, the Draft 3 optimized binary array must be explicitly enabled using
the `version` parameter of [`to_bjdata`](../../api/basic_json/to_bjdata.md).
To preserve compatibility with BJData Draft 2, the Draft 3 optimized binary array must be explicitly enabled: pass
`version = bjdata_version_t::draft3` together with `use_size = true` and `use_type = true` to
[`to_bjdata`](../../api/basic_json/to_bjdata.md). With Draft 3 and only `use_size` (or neither), a binary value is
written as an array of `B` elements.
In Draft2 mode (default), if the JSON data contains the binary type, the value stored as a list of integers, as
In Draft 2 mode (default), if the JSON data contains the binary type, the value is stored as a list of integers, as
suggested by the BJData documentation. In particular, this means that the serialization and the deserialization of
JSON containing binary values into BJData and back will result in a different JSON object.
[BJDataBinArr]: https://github.com/NeuroJSON/bjdata/blob/master/Binary_JData_Specification.md#optimized-binary-array
[BJDataBinArr]: https://github.com/NeuroJSON/bjdata/blob/Draft-3/Binary_JData_Specification.md#optimized-binary-array
??? example "Example: serialize JSON values to BJData, with and without size/type optimization"
@@ -185,7 +254,7 @@ The library uses the following mapping from JSON values types to BJData types ac
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
using namespace nlohmann::literals;
// function to print BJData's diagnostic format
@@ -193,11 +262,11 @@ The library uses the following mapping from JSON values types to BJData types ac
{
if (32 < byte and byte < 128)
{
std::cout << (char)byte;
std::cout << static_cast<char>(byte);
}
else
{
std::cout << (int)byte;
std::cout << static_cast<int>(byte);
}
}
@@ -214,7 +283,7 @@ The library uses the following mapping from JSON values types to BJData types ac
{
print_byte(byte);
}
std::cout << std::endl;
std::cout << '\n';
// create an array of numbers
json array = {1, 2, 3, 4, 5, 6, 7, 8};
@@ -231,19 +300,19 @@ The library uses the following mapping from JSON values types to BJData types ac
{
print_byte(byte);
}
std::cout << std::endl;
std::cout << '\n';
for (auto& byte : v_array_size)
{
print_byte(byte);
}
std::cout << std::endl;
std::cout << '\n';
for (auto& byte : v_array_size_and_type)
{
print_byte(byte);
}
std::cout << std::endl;
std::cout << '\n';
}
```
@@ -258,73 +327,50 @@ The library uses the following mapping from JSON values types to BJData types ac
```
### Byte layout
Consider the following JSON value (shown formatted for readability):
```json
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_bjdata` encodes it to 51 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.
<div class="byte-layout" markdown="0"><table>
<thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody>
<tr><td><code>{</code></td><td><span class="bl bl-t">7B</span></td><td>object start (<code>{</code>)</td></tr>
<tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">49</span> <span class="bl bl-d">2C 01</span></td><td>key: length 2 as int8 (<code>i</code>), then UTF-8 bytes; value: int16 (<code>I</code>), little-endian</td></tr>
<tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">53</span> <span class="bl bl-l">69 04</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: length 4; value: string (<code>S</code>), length 4 as int8, UTF-8 bytes</td></tr>
<tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">44</span> <span class="bl bl-d">1F 85 EB 51 B8 1E 09 40</span></td><td>key: length 2; value: float64 (<code>D</code>), little-endian</td></tr>
<tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">5B</span></td><td>key: length 4; value: array start (<code>[</code>)</td></tr>
<tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">61</span></td><td>string, length 1</td></tr>
<tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">62</span></td><td>string, length 1</td></tr>
<tr><td><code>]</code></td><td><span class="bl bl-t">5D</span></td><td>array end (<code>]</code>)</td></tr>
<tr><td><code>}</code></td><td><span class="bl bl-t">7D</span></td><td>object end (<code>}</code>)</td></tr>
</tbody></table></div>
## Deserialization
The library maps BJData types to JSON value types as follows:
| BJData type | JSON value type | marker |
|-------------|------------------------------------------|----------|
| no-op | *no value, next value is read* | `N` |
| null | `null` | `Z` |
| false | `false` | `F` |
| true | `true` | `T` |
| float16 | number_float | `h` |
| float32 | number_float | `d` |
| float64 | number_float | `D` |
| uint8 | number_unsigned | `U` |
| int8 | number_integer | `i` |
| uint16 | number_unsigned | `u` |
| int16 | number_integer | `I` |
| uint32 | number_unsigned | `m` |
| int32 | number_integer | `l` |
| uint64 | number_unsigned | `M` |
| int64 | number_integer | `L` |
| byte | number_unsigned | `B` |
| string | string | `S` |
| char | string | `C` |
| array | array (optimized values are supported) | `[` |
| ND-array | object (in JData annotated array format) | `[$.#[.` |
| object | object (optimized values are supported) | `{` |
| binary | binary (strongly-typed byte array) | `[$B` |
| BJData type | JSON value type | marker |
|-----------------------|--------------------------------------------------|----------|
| no-op | *no value, next value is read* | `N` |
| null | `null` | `Z` |
| false | `false` | `F` |
| true | `true` | `T` |
| float16 | number_float | `h` |
| float32 | number_float | `d` |
| float64 | number_float | `D` |
| high-precision number | number_integer, number_unsigned, or number_float | `H` |
| uint8 | number_unsigned | `U` |
| int8 | number_integer | `i` |
| uint16 | number_unsigned | `u` |
| int16 | number_integer | `I` |
| uint32 | number_unsigned | `m` |
| int32 | number_integer | `l` |
| uint64 | number_unsigned | `M` |
| int64 | number_integer | `L` |
| byte | number_unsigned | `B` |
| string | string | `S` |
| char | string | `C` |
| array | array (optimized values are supported) | `[` |
| ND-array | object (in JData annotated array format) | `[$.#[.` |
| object | object (optimized values are supported) | `{` |
| binary | binary (strongly-typed byte array) | `[$B` |
!!! success "Complete mapping"
The mapping is **complete** in the sense that any BJData value can be converted to a JSON value.
The mapping is **complete** for the BJData types of Draft 2 and Draft 3 with one exception: column-major
N-dimensional arrays are rejected with [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113).
The Draft 4 constructs (structure-of-arrays containers and the extension type `E`) are not supported either.
The digits of an `H` value are parsed like a JSON number: integers that fit 64 bits become `number_integer` or
`number_unsigned`, anything else a `number_float`; invalid text throws
[`parse_error.115`](../../home/exceptions.md#jsonexceptionparse_error115).
!!! info "Duplicate object keys"
If an object contains the same key more than once, `from_bjdata` accepts it and keeps the last value; with
`ordered_json`, the key keeps the position of its first occurrence.
!!! warning "Ill-formed UTF-8 in string values and object keys"
@@ -356,7 +402,7 @@ The library maps BJData types to JSON value types as follows:
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -370,7 +416,7 @@ The library maps BJData types to JSON value types as follows:
json j = json::from_bjdata(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/bjdata/index.html
+23 -23
View File
@@ -1,13 +1,22 @@
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/bjdata/" rel="canonical"><link href=../ rel=prev><link href=../bon8/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>BJData - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#bjdata class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> BJData </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div class=md-searcLine truncated
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/bjdata/" rel="canonical"><link href=../ rel=prev><link href=../bon8/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>BJData - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#bjdata class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> BJData </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div class=md-searcLine truncated
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_bjdata</code> encodes it to 51 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">7B</span></td><td>object start (<code>{</code>)</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">49</span> <span class="bl bl-d">2C 01</span></td><td>key: length 2 as int8 (<code>i</code>), then UTF-8 bytes; value: int16 (<code>I</code>), little-endian</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">53</span> <span class="bl bl-l">69 04</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: length 4; value: string (<code>S</code>), length 4 as int8, UTF-8 bytes</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">44</span> <span class="bl bl-d">1F 85 EB 51 B8 1E 09 40</span></td><td>key: length 2; value: float64 (<code>D</code>), little-endian</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">5B</span></td><td>key: length 4; value: array start (<code>[</code>)</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">61</span></td><td>string, length 1</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">62</span></td><td>string, length 1</td></tr> <tr><td><code>]</code></td><td><span class="bl bl-t">5D</span></td><td>array end (<code>]</code>)</td></tr> <tr><td><code>}</code></td><td><span class="bl bl-t">7D</span></td><td>object end (<code>}</code>)</td></tr> </tbody></table></div> <h2 id=serialization>Serialization<a class=headerlink href=#serialization title="Permanent link">&para;</a></h2> <p>The library uses the following mapping from <abbr title="JavaScript Object Notation">JSON</abbr> values types to <abbr title="Binary JData">BJData</abbr> types according to the <abbr title="Binary JData">BJData</abbr> specification:</p> <table> <thead> <tr> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>value/range</th> <th><abbr title="Binary JData">BJData</abbr> type</th> <th>marker</th> </tr> </thead> <tbody> <tr> <td>null</td> <td><code>null</code></td> <td>null</td> <td><code>Z</code></td> </tr> <tr> <td>boolean</td> <td><code>true</code></td> <td>true</td> <td><code>T</code></td> </tr> <tr> <td>boolean</td> <td><code>false</code></td> <td>false</td> <td><code>F</code></td> </tr> <tr> <td>number_integer</td> <td>-9223372036854775808..-2147483649</td> <td>int64</td> <td><code>L</code></td> </tr> <tr> <td>number_integer</td> <td>-2147483648..-32769</td> <td>int32</td> <td><code>l</code></td> </tr> <tr> <td>number_integer</td> <td>-32768..-129</td> <td>int16</td> <td><code>I</code></td> </tr> <tr> <td>number_integer</td> <td>-128..127</td> <td>int8</td> <td><code>i</code></td> </tr> <tr> <td>number_integer</td> <td>128..255</td> <td>uint8</td> <td><code>U</code></td> </tr> <tr> <td>number_integer</td> <td>256..32767</td> <td>int16</td> <td><code>I</code></td> </tr> <tr> <td>number_integer</td> <td>32768..65535</td> <td>uint16</td> <td><code>u</code></td> </tr> <tr> <td>number_integer</td> <td>65536..2147483647</td> <td>int32</td> <td><code>l</code></td> </tr> <tr> <td>number_integer</td> <td>2147483648..4294967295</td> <td>uint32</td> <td><code>m</code></td> </tr> <tr> <td>number_integer</td> <td>4294967296..9223372036854775807</td> <td>int64</td> <td><code>L</code></td> </tr> <tr> <td>number_integer</td> <td>9223372036854775808..18446744073709551615</td> <td>uint64</td> <td><code>M</code></td> </tr> <tr> <td>number_unsigned</td> <td>0..127</td> <td>int8</td> <td><code>i</code></td> </tr> <tr> <td>number_unsigned</td> <td>128..255</td> <td>uint8</td> <td><code>U</code></td> </tr> <tr> <td>number_unsigned</td> <td>256..32767</td> <td>int16</td> <td><code>I</code></td> </tr> <tr> <td>number_unsigned</td> <td>32768..65535</td> <td>uint16</td> <td><code>u</code></td> </tr> <tr> <td>number_unsigned</td> <td>65536..2147483647</td> <td>int32</td> <td><code>l</code></td> </tr> <tr> <td>number_unsigned</td> <td>2147483648..4294967295</td> <td>uint32</td> <td><code>m</code></td> </tr> <tr> <td>number_unsigned</td> <td>4294967296..9223372036854775807</td> <td>int64</td> <td><code>L</code></td> </tr> <tr> <td>number_unsigned</td> <td>Line truncated
<span class=w> </span><span class=nt>"_ArrayType_"</span><span class=p>:</span><span class=w> </span><span class=s2>"uint8"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"_ArraySize_"</span><span class=p>:</span><span class=w> </span><span class=p>[</span><span class=mi>2</span><span class=p>,</span><span class=mi>3</span><span class=p>],</span>
<span class=w> </span><span class=nt>"_ArraySize_"</span><span class=p>:</span><span class=w> </span><span class=p>[</span><span class=mi>3</span><span class=p>,</span><span class=mi>2</span><span class=p>],</span>
<span class=w> </span><span class=nt>"_ArrayData_"</span><span class=p>:</span><span class=w> </span><span class=p>[</span><span class=mi>1</span><span class=p>,</span><span class=mi>2</span><span class=p>,</span><span class=mi>3</span><span class=p>,</span><span class=mi>4</span><span class=p>,</span><span class=mi>5</span><span class=p>,</span><span class=mi>6</span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p>Likewise, when a <abbr title="JavaScript Object Notation">JSON</abbr> object in the above form is serialized using <a href=../../../api/basic_json/to_bjdata/ ><code>to_bjdata</code></a>, it is automatically converted into a compact <abbr title="Binary JData">BJData</abbr> ND-array.</p> <p>When parsing, an ND-array whose dimension vector is empty, contains a single integer, contains two integers with the first being 1, or contains a 0 is returned as a regular (possibly empty) array rather than an annotated object.</p> <p>An object is only converted if the annotation describes a packed array that is parsed back into the same annotated object; otherwise it is serialized as a regular <abbr title="JavaScript Object Notation">JSON</abbr> object, so the annotation is never lost in a round trip. This requires all of the following:</p> <ul> <li><code>"_ArrayType_"</code> is one of <code>uint8</code>, <code>int8</code>, <code>uint16</code>, <code>int16</code>, <code>uint32</code>, <code>int32</code>, <code>uint64</code>, <code>int64</code>, <code>single</code>, <code>double</code>, <code>char</code>, or <code>byte</code>,</li> <li><code>"_ArraySize_"</code> is an array, since the dimensions are written as the ND-array header's length,</li> <li><code>"_ArraySize_"</code> has at least two entries and is not a 1×N row vector (first entry 1), since other shapes are parsed back as a regular array,</li> <li>every entry of <code>"_ArraySize_"</code> is a positive integer, and their product is representable as a <code>std::size_t</code>,</li> <li><code>"_ArrayData_"</code> is an array holding exactly that many elements, and</li> <li>every element of <code>"_ArrayData_"</code> is a number of the kind named by <code>"_ArrayType_"</code>: for the integer types, a value that fits the named width; for <code>double</code>, any value; for <code>single</code>, a value that survives narrowing to <code>float</code> and back without change (for instance, <code>0.1</code> does not, since it is not exactly representable as <code>float</code>).</li> </ul> <p>An annotated object is always read back with its keys in the order shown above, <code>"_ArrayType_"</code>, <code>"_ArraySize_"</code>, <code>"_ArrayData_"</code>, regardless of the order the ND-array's header stores them in on the wire. This matters for <code>ordered_json</code>, whose comparison takes key order into account.</p> <p>The current version of this library does not yet support automatic detection of and conversion from a nested <abbr title="JavaScript Object Notation">JSON</abbr> array input to a <abbr title="Binary JData">BJData</abbr> ND-array.</p> </div> <div class="admonition info"> <p class=admonition-title>Restrictions in optimized data types for arrays and objects</p> <p>Due to diminished space saving, hampered readability, and increased security risks, in <abbr title="Binary JData">BJData</abbr>, the allowed data types following the <code>$</code> marker in an optimized array and object container are restricted to <strong>non-zero-fixed-length</strong> data types. Therefore, the valid optimized type markers can only be one of <code>UiuImlMLhdDCB</code>. This also means other variable (<code>[{SH</code>) or zero-length types (<code>TFN</code>) can not be used in an optimized array or object in <abbr title="Binary JData">BJData</abbr>.</p> </div> <div class="admonition info"> <p class=admonition-title>Binary values</p> <p><abbr title="Binary JData">BJData</abbr> provides a dedicated <code>B</code> marker (defined in the <a href="https://github.com/NeuroJSON/bjdata/blob/master/Binary_JData_Specification.md#optimized-binary-array"><abbr title="Binary JData">BJData</abbr> specification (Draft 3)</a>) that is used in optimized arrays to designate binary data. This means that, unlike <abbr title="Universal Binary JSON">UBJSON</abbr>, binary data can be both serialized and deserialized.</p> <p>To preserve compatibility with <abbr title="Binary JData">BJData</abbr> Draft 2, the Draft 3 optimized binary array must be explicitly enabled using the <code>version</code> parameter of <a href=../../../api/basic_json/to_bjdata/ ><code>to_bjdata</code></a>.</p> <p>In Draft2 mode (default), if the <abbr title="JavaScript Object Notation">JSON</abbr> data contains the binary type, the value stored as a list of integers, as suggested by the <abbr title="Binary JData">BJData</abbr> documentation. In particular, this means that the serialization and the deserialization of <abbr title="JavaScript Object Notation">JSON</abbr> containing binary values into <abbr title="Binary JData">BJData</abbr> and back will result in a different <abbr title="JavaScript Object Notation">JSON</abbr> object.</p> </div> <details class=example> <summary>Example: serialize <abbr title="JavaScript Object Notation">JSON</abbr> values to <abbr title="Binary JData">BJData</abbr>, with and without size/type optimization</summary> <div class=highlight><pre><span></span><code><span class=cp>Line truncated
</code></pre></div> <p>ND-arrays with the element type <code>h</code> (float16) are returned with <code>"_ArrayType_": "half"</code>. <code>to_bjdata</code> does not pack <code>"half"</code> objects; they are written as regular objects, which read back as an equal value.</p> <p>Likewise, when a <abbr title="JavaScript Object Notation">JSON</abbr> object in the above form is serialized using <a href=../../../api/basic_json/to_bjdata/ ><code>to_bjdata</code></a>, it is automatically converted into a compact <abbr title="Binary JData">BJData</abbr> ND-array.</p> <p>When parsing, an ND-array whose dimension vector is empty, contains a single integer, contains two integers with the first being 1, or contains a 0 is returned as a regular (possibly empty) array rather than an annotated object.</p> <p>An object is only converted if the annotation describes a packed array that is parsed back into the same annotated object; otherwise it is serialized as a regular <abbr title="JavaScript Object Notation">JSON</abbr> object, so the annotation is never lost in a round trip. This requires all of the following:</p> <ul> <li>the object has exactly the three keys <code>"_ArrayType_"</code>, <code>"_ArraySize_"</code>, and <code>"_ArrayData_"</code> (any other key, e.g., <code>"_ArrayOrder_"</code>, makes it a regular object),</li> <li><code>"_ArrayType_"</code> is one of <code>uint8</code>, <code>int8</code>, <code>uint16</code>, <code>int16</code>, <code>uint32</code>, <code>int32</code>, <code>uint64</code>, <code>int64</code>, <code>single</code>, <code>double</code>, <code>char</code>, or (only with <code>version = bjdata_version_t::draft3</code>, since the <code>B</code> marker is not part of Draft 2) <code>byte</code>; the names are case-sensitive,</li> <li><code>"_ArraySize_"</code> is an array, since the dimensions are written as the ND-array header's length,</li> <li><code>"_ArraySize_"</code> has at least two entries and is not a two-entry row vector (<code>[1,N]</code>), since other shapes are parsed back as a regular array (<code>[1,2,2]</code> is packed, <code>[1,4]</code> is not),</li> <li>every entry of <code>"_ArraySize_"</code> is a positive integer, and their product is representable as a <code>std::size_t</code>,</li> <li><code>"_ArrayData_"</code> is an array holding exactly that many elements, and</li> <li>every element of <code>"_ArrayData_"</code> is a number of the kind named by <code>"_ArrayType_"</code>: for the integer types, an integer (a floating-point number such as <code>1.0</code> does not qualify) that fits the named width (for <code>char</code>, 0..127); for <code>double</code>, a floating-point number (an integer such as <code>1</code> does not qualify); for <code>single</code>, a floating-point number that survives narrowing to <code>float</code> and back without change (for instance, <code>0.1</code> does not, since it is not exactly representable as <code>float</code>).</li> </ul> <p>With <code>ordered_json</code>, an annotated object read from <abbr title="Binary JData">BJData</abbr> has its keys in the order <code>"_ArrayType_"</code>, <code>"_ArraySize_"</code>, <code>"_ArrayData_"</code>; with <code>json</code>, they are sorted like those of any other object.</p> <p>The current version of this library does not yet support automatic detection of and conversion from a nested <abbr title="JavaScript Object Notation">JSON</abbr> array input to a <abbr title="Binary JData">BJData</abbr> ND-array.</p> </div> <div class="admonition info"> <p class=admonition-title>Restrictions in optimized data types for arrays and objects</p> <p>Due to diminished space saving, hampered readability, and increased security risks, in <abbr title="Binary JData">BJData</abbr>, the allowed data types following the <code>$</code> marker in an optimized array and object container are restricted to <strong>non-zero-fixed-length</strong> data types. Therefore, the valid optimized type markers can only be one of <code>UiuImlMLhdDC</code>, plus <code>B</code> since Draft 3 (the reader accepts <code>B</code> in both drafts). This also means other variable (<code>[{SH</code>) or zero-length types (<code>TFZN</code>) can not be used in an optimized array or object in <abbr title="Binary JData">BJData</abbr>. The library enforces this restriction for both drafts. A <code>C</code> (char) element above 127 is rejected on read with <a href=../../../home/exceptions/#jsonexceptionparse_error113><code>parse_error.113</code></a>, in 1-D and in N-dimensional arrays. With <code>use_type</code>, the elements must have the same marker (<code>[1,200,3]</code> is not optimized, since it mixes <code>i</code> and <code>U</code>); booleans, null, strings, arrays, and objects are never optimized with <code>$</code>.</p> </div> <div class="admonition info"> <p class=admonition-title>Binary values</p> <p><abbr title="Binary JData">BJData</abbr> provides a dedicated <code>B</code> marker (defined in the <a href="https://github.com/NeuroJSOLine truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=k>namespace</span><span class=w> </span><span class=nn>nlohmann</span><span class=o>::</span><span class=nn>literals</span><span class=p>;</span>
<span class=c1>// function to print BJData's diagnostic format</span>
@@ -15,11 +24,11 @@
<span class=p>{</span>
<span class=w> </span><span class=k>if</span><span class=w> </span><span class=p>(</span><span class=mi>32</span><span class=w> </span><span class=o>&lt;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=k>and</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>&lt;</span><span class=w> </span><span class=mi>128</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=p>(</span><span class=kt>char</span><span class=p>)</span><span class=kt>byte</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=k>static_cast</span><span class=o>&lt;</span><span class=kt>char</span><span class=o>&gt;</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=k>else</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=p>(</span><span class=kt>int</span><span class=p>)</span><span class=kt>byte</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=k>static_cast</span><span class=o>&lt;</span><span class=kt>int</span><span class=o>&gt;</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=p>}</span>
@@ -36,7 +45,7 @@
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>print_byte</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=w> </span><span class=c1>// create an array of numbers</span>
<span class=w> </span><span class=n>json</span><span class=w> </span><span class=n>array</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=p>{</span><span class=mi>1</span><span class=p>,</span><span class=w> </span><span class=mi>2</span><span class=p>,</span><span class=w> </span><span class=mi>3</span><span class=p>,</span><span class=w> </span><span class=mi>4</span><span class=p>,</span><span class=w> </span><span class=mi>5</span><span class=p>,</span><span class=w> </span><span class=mi>6</span><span class=p>,</span><span class=w> </span><span class=mi>7</span><span class=p>,</span><span class=w> </span><span class=mi>8</span><span class=p>};</span>
@@ -53,38 +62,29 @@
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>print_byte</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=w> </span><span class=k>for</span><span class=w> </span><span class=p>(</span><span class=k>auto</span><span class=o>&amp;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>:</span><span class=w> </span><span class=n>v_array_size</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>print_byte</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=w> </span><span class=k>for</span><span class=w> </span><span class=p>(</span><span class=k>auto</span><span class=o>&amp;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>:</span><span class=w> </span><span class=n>v_array_size_and_type</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>print_byte</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=p>{</span><span class=n>i7compactTi6schemaF</span><span class=p>}</span>
<span class=p>[</span><span class=n>i1i2i3i4i5i6i7i8</span><span class=p>]</span>
<span class=p>[</span><span class=err>#</span><span class=n>i8i1i2i3i4i5i6i7i8</span>
<span class=p>[</span><span class=n>$i</span><span class=err>#</span><span class=n>i812345678</span>
</code></pre></div> </details> <h3 id=byte-layout>Byte layout<a class=headerlink href=#byte-layout title="Permanent link">&para;</a></h3> <p>Consider the following <abbr title="JavaScript Object Notation">JSON</abbr> value (shown formatted for readability):</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_bjdata</code> encodes it to 51 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">7B</span></td><td>object start (<code>{</code>)</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">49</span> <span class="bl bl-d">2C 01</span></td><td>key: length 2 as int8 (<code>i</code>), then UTF-8 bytes; value: int16 (<code>I</code>), little-endian</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">53</span> <span class="bl bl-l">69 04</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: length 4; value: string (<code>S</code>), length 4 as int8, UTF-8 bytes</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">44</span> <span class="bl bl-d">1F 85 EB 51 B8 1E 09 40</span></td><td>key: length 2; value: float64 (<code>D</code>), little-endian</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">5B</span></td><td>key: length 4; value: array start (<code>[</code>)</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">61</span></td><td>string, length 1</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">62</span></td><td>string, length 1</td></tr> <tr><td><code>]</code></td><td><span class="bl bl-t">5D</span></td><td>array end (<code>]</code>)</td></tr> <tr><td><code>}</code></td><td><span class="bl bl-t">7D</span></td><td>object end (<code>}</code>)</td></tr> </tbody></table></div> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Binary JData">BJData</abbr> types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Binary JData">BJData</abbr> type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>marker</th> </tr> </thead> <tbody> <tr> <td>no-op</td> <td><em>no value, next value is read</em></td> <td><code>N</code></td> </tr> <tr> <td>null</td> <td><code>null</code></td> <td><code>Z</code></td> </tr> <tr> <td>false</td> <td><code>false</code></td> <td><code>F</code></td> </tr> <tr> <td>true</td> <td><code>true</code></td> <td><code>T</code></td> </tr> <tr> <td>float16</td> <td>number_float</td> <td><code>h</code></td> </tr> <tr> <td>float32</td> <td>number_float</td> <td><code>d</code></td> </tr> <tr> <td>float64</td> <td>number_float</td> <td><code>D</code></td> </tr> <tr> <td>uint8</td> <td>number_unsigned</td> <td><code>U</code></td> </tr> <tr> <td>int8</td> <td>number_integer</td> <td><code>i</code></td> </tr> <tr> <td>uint16</td> <td>number_unsigned</td> <td><code>u</code></td> </tr> <tr> <td>int16</td> <td>number_integer</td> <td><code>I</code></td> </tr> <tr> <td>uint32</td> <td>number_unsigned</td> <td><code>m</code></td> </tr> <tr> <td>int32</td> <td>number_integer</td> <td><code>l</code></td> </tr> <tr> <td>uint64</td> <td>number_unsigned</td> <td><code>M</code></td> </tr> <tr> <td>int64</td> <td>number_integer</td> <td><code>L</code></td> </tr> <tr> <td>byte</td> <td>number_unsigned</td> <td><code>B</code></td> </tr> <tr> <td>string</td> <td>string</td> <td><code>S</code></td> </tr> <tr> <td>char</td> <td>string</td> <td><code>C</code></td> </tr> <tr> <td>array</td> <td>array (optimized values are supported)</td> <td><code>[</code></td> </tr> <tr> <td>ND-array</td> <td>object (in JData annotated array format)</td> <td><code>[$.#[.</code></td> </tr> <tr> <td>object</td> <td>object (optimized values are supported)</td> <td><code>{</code></td> </tr> <tr> <td>binary</td> <td>binary (strongly-typed byte array)</td> <td><code>[$B</code></td> </tr> </tbody> </table> <div class="admonition success"> <p class=admonition-title>Complete mapping</p> <p>The mapping is <strong>complete</strong> in the sense that any <abbr title="Binary JData">BJData</abbr> value can be converted to a <abbr title="JavaScript Object Notation">JSON</abbr> value.</p> </div> <div class="admonition warning"> <p class=admonition-title>Ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8 in string values and object keys</p> <p><abbr title="Binary JData">BJDaLine truncated
</code></pre></div> </details> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Binary JData">BJData</abbr> types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Binary JData">BJData</abbr> type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>marker</th> </tr> </thead> <tbody> <tr> <td>no-op</td> <td><em>no value, next value is read</em></td> <td><code>N</code></td> </tr> <tr> <td>null</td> <td><code>null</code></td> <td><code>Z</code></td> </tr> <tr> <td>false</td> <td><code>false</code></td> <td><code>F</code></td> </tr> <tr> <td>true</td> <td><code>true</code></td> <td><code>T</code></td> </tr> <tr> <td>float16</td> <td>number_float</td> <td><code>h</code></td> </tr> <tr> <td>float32</td> <td>number_float</td> <td><code>d</code></td> </tr> <tr> <td>float64</td> <td>number_float</td> <td><code>D</code></td> </tr> <tr> <td>high-precision number</td> <td>number_integer, number_unsigned, or number_float</td> <td><code>H</code></td> </tr> <tr> <td>uint8</td> <td>number_unsigned</td> <td><code>U</code></td> </tr> <tr> <td>int8</td> <td>number_integer</td> <td><code>i</code></td> </tr> <tr> <td>uint16</td> <td>number_unsigned</td> <td><code>u</code></td> </tr> <tr> <td>int16</td> <td>number_integer</td> <td><code>I</code></td> </tr> <tr> <td>uint32</td> <td>number_unsigned</td> <td><code>m</code></td> </tr> <tr> <td>int32</td> <td>number_integer</td> <td><code>l</code></td> </tr> <tr> <td>uint64</td> <td>number_unsigned</td> <td><code>M</code></td> </tr> <tr> <td>int64</td> <td>number_integer</td> <td><code>L</code></td> </tr> <tr> <td>byte</td> <td>number_unsigned</td> <td><code>B</code></td> </tr> <tr> <td>string</td> <td>string</td> <td><code>S</code></td> </tr> <tr> <td>char</td> <td>string</td> <td><code>C</code></td> </tr> <tr> <td>array</td> <td>array (optimized values are supported)</td> <td><code>[</code></td> </tr> <tr> <td>ND-array</td> <td>object (in JData annotated array format)</td> <td><code>[$.#[.</code></td> </tr> <tr> <td>object</td> <td>object (optimized values are supported)</td> <td><code>{</code></td> </tr> <tr> <td>binary</td> <td>binary (strongly-typed byte array)</td> <td><code>[$B</code></td> </tr> </tbody> </table> <div class="admonition success"> <p class=admonition-title>Complete mapping</p> <p>The mapping is <strong>complete</strong> for the <abbr title="Binary JData">BJData</abbr> types of Draft 2 and Draft 3 with one exception: column-major N-dimensional arrays are rejected with <a href=../../../home/exceptions/#jsonexceptionparse_error113><code>parse_error.113</code></a>. The Draft 4 constructs (structure-of-arrays containers and the extension type <code>E</code>) are not supported either.</p> <p>The digits of an <code>H</code> value are parsed like a <abbr title="JavaScript Object Notation">JSON</abbr> number: integers that fit 64 bits become <code>number_integer</code> or <code>number_unsigned</code>, anything else a <code>number_float</code>; invalid text throws <a href=../../../home/exceptions/#jsonexceptionparse_error115><code>parse_error.115</code></a>.</p> </div> <div class="admonition info"> <p class=admonition-title>Duplicate object keys</p> <p>If an object contains the same key more than once, <code>from_bjdata</code> accepts it and keeps the last value; with <code>ordered_json</code>, the key keeps the position of its first occurrence.</p> </div> <div class="admonition warning"> <p class=admonition-title>Ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8 in string values and object keys</p> <p><abbr title="Binary JData">BJData</abbr> strings must use <abbr title="Unicode Transformation Format">UTF</abbr>-8 encoding, but checking it on read is opt-in: with the <a href=../../../api/basic_json/from_bjdata/ ><code>error_handler</code></a> parameter left at <code>keep</code> (the default), <code>from_bjdata()</code> accepts a string value or object key whose bytes are not valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 and hands them back unchanged. Passing <code>error_handler_t::strict</code> makes <code>from_bjdata()</code> check and throw <a href=../../../home/exceptions/#jsonexceptionparse_error113><code>parse_error.113</code></a> for ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8, and <code>replace</code>/<code>ignore</code> sanitize the string instead of keeping it. However, <a href=../../../api/basic_json/dump/ ><code>dump()</code></a> still requires valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 and throws <a href=../../../home/exceptions/#jsonexceptiontype_error316><code>type_error.316</code></a> for a value read with the default <code>keep</code> handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. <code>to_bjdata()</code>'s own <code>error_handler</cLine truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
<span class=p>{</span>
@@ -98,10 +98,10 @@
<span class=w> </span><span class=n>json</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>json</span><span class=o>::</span><span class=n>from_bjdata</span><span class=p>(</span><span class=n>v</span><span class=p>);</span>
<span class=w> </span><span class=c1>// print the deserialized JSON value</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"compact"</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=p>,</span>
<span class=w> </span><span class=nt>"schema"</span><span class=p>:</span><span class=w> </span><span class=mi>0</span>
<span class=p>}</span>
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 24, 2026 15:57:44 UTC">October 24, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 26, 2026 17:25:46 UTC">October 26, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
features/binary_formats/bjdata/index.md
+245 -219
View File
@@ -1,209 +1,27 @@
# BJData
The [BJData format](https://neurojson.org) was derived from and improved upon [Universal Binary JSON(UBJSON)](https://ubjson.org) specification (Draft 12). Specifically, it introduces an optimized array container for efficient storage of N-dimensional packed arrays (**ND-arrays**); it also adds 5 new type markers - `[u] - uint16`, `[m] - uint32`, `[M] - uint64`, `[h] - float16` and `[B] - byte` - to unambiguously map common binary numeric types; furthermore, it uses little-endian (LE) to store all numerics instead of big-endian (BE) as in UBJSON to avoid unnecessary conversions on commonly available platforms.
The [BJData format](https://neurojson.org) was derived from and improved upon [Universal Binary JSON(UBJSON)](https://ubjson.org) specification (Draft 12). Specifically, it introduces an optimized array container for efficient storage of N-dimensional packed arrays (**ND-arrays**); it also adds four new type markers - `[u] - uint16`, `[m] - uint32`, `[M] - uint64`, and `[h] - float16` - to unambiguously map common binary numeric types (Draft 3 adds `[B] - byte`); furthermore, it uses little-endian (LE) to store all numerics instead of big-endian (BE) as in UBJSON to avoid unnecessary conversions on commonly available platforms.
Compared to other binary JSON-like formats such as MessagePack and CBOR, both BJData and UBJSON demonstrate a rare combination of being both binary and **quasi-human-readable**. This is because all semantic elements in BJData and UBJSON, including the data-type markers and name/string types, are directly human-readable. Data stored in the BJData/UBJSON format is not only compact in size, fast to read/write, but also can be directly searched or read using simple processing.
References
- [BJData Specification](https://neurojson.org/bjdata/draft2)
- [BJData Specification (Draft 3)](https://neurojson.org/bjdata/draft3)
- [BJData Specification (Draft 2)](https://neurojson.org/bjdata/draft2)
## Serialization
Supported drafts
The library uses the following mapping from JSON values types to BJData types according to the BJData specification:
The library supports BJData Draft 2 (the default) and Draft 3, except for column-major N-dimensional arrays (see below). The draft used for writing is selected with the [`version` parameter](https://json.nlohmann.me/api/basic_json/bjdata_version_t/index.md) of [`to_bjdata`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md); [`from_bjdata`](https://json.nlohmann.me/api/basic_json/from_bjdata/index.md) has no such parameter and reads the markers of both drafts.
| JSON value type | value/range | BJData type | marker |
| --------------- | ----------------------------------------- | ----------- | ------ |
| null | `null` | null | `Z` |
| boolean | `true` | true | `T` |
| boolean | `false` | false | `F` |
| number_integer | -9223372036854775808..-2147483649 | int64 | `L` |
| number_integer | -2147483648..-32769 | int32 | `l` |
| number_integer | -32768..-129 | int16 | `I` |
| number_integer | -128..127 | int8 | `i` |
| number_integer | 128..255 | uint8 | `U` |
| number_integer | 256..32767 | int16 | `I` |
| number_integer | 32768..65535 | uint16 | `u` |
| number_integer | 65536..2147483647 | int32 | `l` |
| number_integer | 2147483648..4294967295 | uint32 | `m` |
| number_integer | 4294967296..9223372036854775807 | int64 | `L` |
| number_integer | 9223372036854775808..18446744073709551615 | uint64 | `M` |
| number_unsigned | 0..127 | int8 | `i` |
| number_unsigned | 128..255 | uint8 | `U` |
| number_unsigned | 256..32767 | int16 | `I` |
| number_unsigned | 32768..65535 | uint16 | `u` |
| number_unsigned | 65536..2147483647 | int32 | `l` |
| number_unsigned | 2147483648..4294967295 | uint32 | `m` |
| number_unsigned | 4294967296..9223372036854775807 | int64 | `L` |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | `M` |
| number_float | *any value* | float64 | `D` |
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format/ND-array* | array | `[` |
| object | *see notes on optimized format* | map | `{` |
| binary | *see notes on binary values* | array | `[$B` |
Draft 4 and column-major ND-arrays are not supported
Complete mapping
BJData Draft 4 (April 2026) adds structure-of-arrays (SoA) containers (`[$` or `{$` followed by a schema object) and the extension type `E`. `from_bjdata` does not implement them and throws [`parse_error.112`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error112) for such input; `to_bjdata` never writes them.
The mapping is **complete** in the sense that any JSON value type can be converted to a BJData value.
Likewise, column-major ND-arrays of Draft 3 (the `#` marker followed by an array that wraps the dimension vector) are rejected with [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113).
Any BJData output created by `to_bjdata` can be successfully parsed by `from_bjdata`.
## Byte layout
Size constraints
The following values can **not** be converted to a BJData value:
- strings with more than 18446744073709551615 bytes, i.e., 264-1 bytes (theoretical)
UTF-8 validation of string values and object keys
BJData strings must use UTF-8 encoding. By default (the [`error_handler`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md) parameter left at `keep`), `to_bjdata()` writes the bytes of string values and object keys unchanged, even if they are not valid UTF-8. With `error_handler_t::strict`, it throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for ill-formed UTF-8 instead; `replace`/`ignore` sanitize the string. [`JSON_STRICT_BINARY_UTF8`](https://json.nlohmann.me/api/macros/json_strict_binary_utf8/index.md) makes `strict` the default.
Unused BJData markers
The following markers are not used in the conversion:
- `Z`: no-op values are not created.
- `C`: single-byte strings are serialized with `S` markers.
NaN/infinity handling
If NaN or Infinity are stored inside a JSON number, they are serialized properly. This behavior differs from the [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md) function which serializes NaN or Infinity to `null`.
Endianness
A breaking difference between BJData and UBJSON is the endianness of numerical values. In BJData, all numerical data types (integers `UiuImlML` and floating-point values `hdD`) are stored in the little-endian (LE) byte order as opposed to big-endian as used by UBJSON. Adopting LE to store numeric records avoids unnecessary byte swapping on most modern computers where LE is used as the default byte order.
Optimized formats
Optimized formats for containers are supported via two parameters of [`to_bjdata`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md):
- Parameter `use_size` adds size information to the beginning of a container and removes the closing marker.
- Parameter `use_type` further checks whether all elements of a container have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be used together with `use_size = true`.
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the receiving side is immediately informed of the number of elements in the container.
ND-array optimized format
BJData extends UBJSON's optimized array **size** marker to support ND-arrays of uniform numerical data types (referred to as *packed arrays*). For example, the 2-D `uint8` integer array `[[1,2],[3,4],[5,6]]`, stored as nested optimized array in UBJSON `[ [$U#i2 1 2 [$U#i2 3 4 [$U#i2 5 6 ]`, can be further compressed in BJData to `[$U#[$i#i2 2 3 1 2 3 4 5 6` or `[$U#[i2 i3] 1 2 3 4 5 6`.
To maintain type and size information, ND-arrays are converted to JSON objects following the **annotated array format** (defined in the [JData specification (Draft 3)](https://github.com/NeuroJSON/jdata/blob/master/JData_specification.md#annotated-storage-of-n-d-arrays)), when parsed using [`from_bjdata`](https://json.nlohmann.me/api/basic_json/from_bjdata/index.md). For example, the above 2-D `uint8` array can be parsed and accessed as
```
{
"_ArrayType_": "uint8",
"_ArraySize_": [2,3],
"_ArrayData_": [1,2,3,4,5,6]
}
```
Likewise, when a JSON object in the above form is serialized using [`to_bjdata`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md), it is automatically converted into a compact BJData ND-array.
When parsing, an ND-array whose dimension vector is empty, contains a single integer, contains two integers with the first being 1, or contains a 0 is returned as a regular (possibly empty) array rather than an annotated object.
An object is only converted if the annotation describes a packed array that is parsed back into the same annotated object; otherwise it is serialized as a regular JSON object, so the annotation is never lost in a round trip. This requires all of the following:
- `"_ArrayType_"` is one of `uint8`, `int8`, `uint16`, `int16`, `uint32`, `int32`, `uint64`, `int64`, `single`, `double`, `char`, or `byte`,
- `"_ArraySize_"` is an array, since the dimensions are written as the ND-array header's length,
- `"_ArraySize_"` has at least two entries and is not a 1×N row vector (first entry 1), since other shapes are parsed back as a regular array,
- every entry of `"_ArraySize_"` is a positive integer, and their product is representable as a `std::size_t`,
- `"_ArrayData_"` is an array holding exactly that many elements, and
- every element of `"_ArrayData_"` is a number of the kind named by `"_ArrayType_"`: for the integer types, a value that fits the named width; for `double`, any value; for `single`, a value that survives narrowing to `float` and back without change (for instance, `0.1` does not, since it is not exactly representable as `float`).
An annotated object is always read back with its keys in the order shown above, `"_ArrayType_"`, `"_ArraySize_"`, `"_ArrayData_"`, regardless of the order the ND-array's header stores them in on the wire. This matters for `ordered_json`, whose comparison takes key order into account.
The current version of this library does not yet support automatic detection of and conversion from a nested JSON array input to a BJData ND-array.
Restrictions in optimized data types for arrays and objects
Due to diminished space saving, hampered readability, and increased security risks, in BJData, the allowed data types following the `$` marker in an optimized array and object container are restricted to **non-zero-fixed-length** data types. Therefore, the valid optimized type markers can only be one of `UiuImlMLhdDCB`. This also means other variable (`[{SH`) or zero-length types (`TFN`) can not be used in an optimized array or object in BJData.
Binary values
BJData provides a dedicated `B` marker (defined in the [BJData specification (Draft 3)](https://github.com/NeuroJSON/bjdata/blob/master/Binary_JData_Specification.md#optimized-binary-array)) that is used in optimized arrays to designate binary data. This means that, unlike UBJSON, binary data can be both serialized and deserialized.
To preserve compatibility with BJData Draft 2, the Draft 3 optimized binary array must be explicitly enabled using the `version` parameter of [`to_bjdata`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md).
In Draft2 mode (default), if the JSON data contains the binary type, the value stored as a list of integers, as suggested by the BJData documentation. In particular, this means that the serialization and the deserialization of JSON containing binary values into BJData and back will result in a different JSON object.
Example: serialize JSON values to BJData, with and without size/type optimization
```
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using namespace nlohmann::literals;
// function to print BJData's diagnostic format
void print_byte(uint8_t byte)
{
if (32 < byte and byte < 128)
{
std::cout << (char)byte;
}
else
{
std::cout << (int)byte;
}
}
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": false})"_json;
// serialize it to BJData
std::vector<std::uint8_t> v = json::to_bjdata(j);
// print the vector content
for (auto& byte : v)
{
print_byte(byte);
}
std::cout << std::endl;
// create an array of numbers
json array = {1, 2, 3, 4, 5, 6, 7, 8};
// serialize it to BJData using default representation
std::vector<std::uint8_t> v_array = json::to_bjdata(array);
// serialize it to BJData using size optimization
std::vector<std::uint8_t> v_array_size = json::to_bjdata(array, true);
// serialize it to BJData using type optimization
std::vector<std::uint8_t> v_array_size_and_type = json::to_bjdata(array, true, true);
// print the vector contents
for (auto& byte : v_array)
{
print_byte(byte);
}
std::cout << std::endl;
for (auto& byte : v_array_size)
{
print_byte(byte);
}
std::cout << std::endl;
for (auto& byte : v_array_size_and_type)
{
print_byte(byte);
}
std::cout << std::endl;
}
```
Output:
```
{i7compactTi6schemaF}
[i1i2i3i4i5i6i7i8]
[#i8i1i2i3i4i5i6i7i8
[$i#i812345678
```
### Byte layout
BJData follows the structure of UBJSON: a one-byte, human-readable ASCII *type marker* followed by the payload of that type, with strings prefixed by a length that is itself an integer value. In contrast to UBJSON, numbers are stored in little-endian byte order, and additional markers cover unsigned 16/32/64-bit integers, half-precision floats, and bytes. Containers can be written unoptimized (opened with `[` or `{`, closed with `]` or `}`) or optimized with a count (`#`), optionally preceded by a declared element type (`$`). BJData extends the optimized form: the count can be a list of dimensions, which stores an N-dimensional packed array as a flat sequence of values without any per-element markers.
Consider the following JSON value (shown formatted for readability):
@@ -233,38 +51,246 @@ Consider the following JSON value (shown formatted for readability):
| `]` | 5D | array end (`]`) |
| `}` | 7D | object end (`}`) |
## Serialization
The library uses the following mapping from JSON values types to BJData types according to the BJData specification:
| JSON value type | value/range | BJData type | marker |
| --------------- | ----------------------------------------- | ----------- | ----------------------------------------------------------------------- |
| null | `null` | null | `Z` |
| boolean | `true` | true | `T` |
| boolean | `false` | false | `F` |
| number_integer | -9223372036854775808..-2147483649 | int64 | `L` |
| number_integer | -2147483648..-32769 | int32 | `l` |
| number_integer | -32768..-129 | int16 | `I` |
| number_integer | -128..127 | int8 | `i` |
| number_integer | 128..255 | uint8 | `U` |
| number_integer | 256..32767 | int16 | `I` |
| number_integer | 32768..65535 | uint16 | `u` |
| number_integer | 65536..2147483647 | int32 | `l` |
| number_integer | 2147483648..4294967295 | uint32 | `m` |
| number_integer | 4294967296..9223372036854775807 | int64 | `L` |
| number_integer | 9223372036854775808..18446744073709551615 | uint64 | `M` |
| number_unsigned | 0..127 | int8 | `i` |
| number_unsigned | 128..255 | uint8 | `U` |
| number_unsigned | 256..32767 | int16 | `I` |
| number_unsigned | 32768..65535 | uint16 | `u` |
| number_unsigned | 65536..2147483647 | int32 | `l` |
| number_unsigned | 2147483648..4294967295 | uint32 | `m` |
| number_unsigned | 4294967296..9223372036854775807 | int64 | `L` |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | `M` |
| number_float | *any value* | float64 | `D` |
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format/ND-array* | array | `[` |
| object | *see notes on optimized format* | map | `{` |
| binary | *see notes on binary values* | array | `[$B` (Draft 3 with `use_size` and `use_type`), `[U`, or `[B` otherwise |
Complete mapping
The mapping is **complete** in the sense that any JSON value type can be converted to a BJData value.
Any BJData output created by `to_bjdata` can be successfully parsed by `from_bjdata`.
Size constraints
The following values can **not** be converted to a BJData value:
- strings with more than 18446744073709551615 bytes, i.e., 264-1 bytes (theoretical)
UTF-8 validation of string values and object keys
BJData strings must use UTF-8 encoding. By default (the [`error_handler`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md) parameter left at `keep`), `to_bjdata()` writes the bytes of string values and object keys unchanged, even if they are not valid UTF-8. With `error_handler_t::strict`, it throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for ill-formed UTF-8 instead; `replace`/`ignore` sanitize the string. [`JSON_STRICT_BINARY_UTF8`](https://json.nlohmann.me/api/macros/json_strict_binary_utf8/index.md) makes `strict` the default.
Unused BJData markers
The following markers are not written:
- `N`: no-op values are not created (they are skipped when reading).
- `C`: single-character strings are serialized with `S` markers (`C` is read, and written for the elements of ND-arrays with `"_ArrayType_": "char"`).
- `h`: floating-point numbers are written as `D`, not as float16 (`h` is read).
- `B`: only written in Draft 3 mode.
- `d` is only written for `single` ND-arrays or if `number_float_t` is `float`; `H` is only written for integers wider than 64 bits (custom `number_integer_t`/`number_unsigned_t`). An unsigned integer above 9223372036854775807 is written as `M`.
NaN/infinity handling
If NaN or Infinity are stored inside a JSON number, they are serialized properly. This behavior differs from the [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md) function which serializes NaN or Infinity to `null`.
Endianness
A breaking difference between BJData and UBJSON is the endianness of numerical values. In BJData, all numerical data types (integers `UiuImlML` and floating-point values `hdD`) are stored in the little-endian (LE) byte order as opposed to big-endian as used by UBJSON. Adopting LE to store numeric records avoids unnecessary byte swapping on most modern computers where LE is used as the default byte order.
Optimized formats
Optimized formats for containers are supported via two parameters of [`to_bjdata`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md):
- Parameter `use_size` adds size information to the beginning of a container and removes the closing marker.
- Parameter `use_type` further checks whether all elements of a container have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be used together with `use_size = true`.
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the receiving side is immediately informed of the number of elements in the container.
ND-array optimized format
BJData extends UBJSON's optimized array **size** marker to support ND-arrays of uniform numerical data types (referred to as *packed arrays*). For example, the 2-D `uint8` integer array `[[1,2],[3,4],[5,6]]`, stored as nested optimized array in UBJSON `[ [$U#i2 1 2 [$U#i2 3 4 [$U#i2 5 6 ]`, can be further compressed in BJData to `[$U#[$i#i2 3 2 1 2 3 4 5 6` or `[$U#[i3 i2] 1 2 3 4 5 6`.
To maintain type and size information, ND-arrays are converted to JSON objects following the **annotated array format** (defined in the [JData specification (Draft 3)](https://github.com/NeuroJSON/jdata/blob/Draft-3/JData_specification.md#annotated-storage-of-n-d-arrays)), when parsed using [`from_bjdata`](https://json.nlohmann.me/api/basic_json/from_bjdata/index.md). For example, the above 2-D `uint8` array can be parsed and accessed as
```
{
"_ArrayType_": "uint8",
"_ArraySize_": [3,2],
"_ArrayData_": [1,2,3,4,5,6]
}
```
ND-arrays with the element type `h` (float16) are returned with `"_ArrayType_": "half"`. `to_bjdata` does not pack `"half"` objects; they are written as regular objects, which read back as an equal value.
Likewise, when a JSON object in the above form is serialized using [`to_bjdata`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md), it is automatically converted into a compact BJData ND-array.
When parsing, an ND-array whose dimension vector is empty, contains a single integer, contains two integers with the first being 1, or contains a 0 is returned as a regular (possibly empty) array rather than an annotated object.
An object is only converted if the annotation describes a packed array that is parsed back into the same annotated object; otherwise it is serialized as a regular JSON object, so the annotation is never lost in a round trip. This requires all of the following:
- the object has exactly the three keys `"_ArrayType_"`, `"_ArraySize_"`, and `"_ArrayData_"` (any other key, e.g., `"_ArrayOrder_"`, makes it a regular object),
- `"_ArrayType_"` is one of `uint8`, `int8`, `uint16`, `int16`, `uint32`, `int32`, `uint64`, `int64`, `single`, `double`, `char`, or (only with `version = bjdata_version_t::draft3`, since the `B` marker is not part of Draft 2) `byte`; the names are case-sensitive,
- `"_ArraySize_"` is an array, since the dimensions are written as the ND-array header's length,
- `"_ArraySize_"` has at least two entries and is not a two-entry row vector (`[1,N]`), since other shapes are parsed back as a regular array (`[1,2,2]` is packed, `[1,4]` is not),
- every entry of `"_ArraySize_"` is a positive integer, and their product is representable as a `std::size_t`,
- `"_ArrayData_"` is an array holding exactly that many elements, and
- every element of `"_ArrayData_"` is a number of the kind named by `"_ArrayType_"`: for the integer types, an integer (a floating-point number such as `1.0` does not qualify) that fits the named width (for `char`, 0..127); for `double`, a floating-point number (an integer such as `1` does not qualify); for `single`, a floating-point number that survives narrowing to `float` and back without change (for instance, `0.1` does not, since it is not exactly representable as `float`).
With `ordered_json`, an annotated object read from BJData has its keys in the order `"_ArrayType_"`, `"_ArraySize_"`, `"_ArrayData_"`; with `json`, they are sorted like those of any other object.
The current version of this library does not yet support automatic detection of and conversion from a nested JSON array input to a BJData ND-array.
Restrictions in optimized data types for arrays and objects
Due to diminished space saving, hampered readability, and increased security risks, in BJData, the allowed data types following the `$` marker in an optimized array and object container are restricted to **non-zero-fixed-length** data types. Therefore, the valid optimized type markers can only be one of `UiuImlMLhdDC`, plus `B` since Draft 3 (the reader accepts `B` in both drafts). This also means other variable (`[{SH`) or zero-length types (`TFZN`) can not be used in an optimized array or object in BJData. The library enforces this restriction for both drafts. A `C` (char) element above 127 is rejected on read with [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113), in 1-D and in N-dimensional arrays. With `use_type`, the elements must have the same marker (`[1,200,3]` is not optimized, since it mixes `i` and `U`); booleans, null, strings, arrays, and objects are never optimized with `$`.
Binary values
BJData provides a dedicated `B` marker (defined in the [BJData specification (Draft 3)](https://github.com/NeuroJSON/bjdata/blob/Draft-3/Binary_JData_Specification.md#optimized-binary-array)) that is used in optimized arrays to designate binary data. This means that, unlike UBJSON, binary data can be both serialized and deserialized.
To preserve compatibility with BJData Draft 2, the Draft 3 optimized binary array must be explicitly enabled: pass `version = bjdata_version_t::draft3` together with `use_size = true` and `use_type = true` to [`to_bjdata`](https://json.nlohmann.me/api/basic_json/to_bjdata/index.md). With Draft 3 and only `use_size` (or neither), a binary value is written as an array of `B` elements.
In Draft 2 mode (default), if the JSON data contains the binary type, the value is stored as a list of integers, as suggested by the BJData documentation. In particular, this means that the serialization and the deserialization of JSON containing binary values into BJData and back will result in a different JSON object.
Example: serialize JSON values to BJData, with and without size/type optimization
```
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using nlohmann::json;
using namespace nlohmann::literals;
// function to print BJData's diagnostic format
void print_byte(uint8_t byte)
{
if (32 < byte and byte < 128)
{
std::cout << static_cast<char>(byte);
}
else
{
std::cout << static_cast<int>(byte);
}
}
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": false})"_json;
// serialize it to BJData
std::vector<std::uint8_t> v = json::to_bjdata(j);
// print the vector content
for (auto& byte : v)
{
print_byte(byte);
}
std::cout << '\n';
// create an array of numbers
json array = {1, 2, 3, 4, 5, 6, 7, 8};
// serialize it to BJData using default representation
std::vector<std::uint8_t> v_array = json::to_bjdata(array);
// serialize it to BJData using size optimization
std::vector<std::uint8_t> v_array_size = json::to_bjdata(array, true);
// serialize it to BJData using type optimization
std::vector<std::uint8_t> v_array_size_and_type = json::to_bjdata(array, true, true);
// print the vector contents
for (auto& byte : v_array)
{
print_byte(byte);
}
std::cout << '\n';
for (auto& byte : v_array_size)
{
print_byte(byte);
}
std::cout << '\n';
for (auto& byte : v_array_size_and_type)
{
print_byte(byte);
}
std::cout << '\n';
}
```
Output:
```
{i7compactTi6schemaF}
[i1i2i3i4i5i6i7i8]
[#i8i1i2i3i4i5i6i7i8
[$i#i812345678
```
## Deserialization
The library maps BJData types to JSON value types as follows:
| BJData type | JSON value type | marker |
| ----------- | ---------------------------------------- | -------- |
| no-op | *no value, next value is read* | `N` |
| null | `null` | `Z` |
| false | `false` | `F` |
| true | `true` | `T` |
| float16 | number_float | `h` |
| float32 | number_float | `d` |
| float64 | number_float | `D` |
| uint8 | number_unsigned | `U` |
| int8 | number_integer | `i` |
| uint16 | number_unsigned | `u` |
| int16 | number_integer | `I` |
| uint32 | number_unsigned | `m` |
| int32 | number_integer | `l` |
| uint64 | number_unsigned | `M` |
| int64 | number_integer | `L` |
| byte | number_unsigned | `B` |
| string | string | `S` |
| char | string | `C` |
| array | array (optimized values are supported) | `[` |
| ND-array | object (in JData annotated array format) | `[$.#[.` |
| object | object (optimized values are supported) | `{` |
| binary | binary (strongly-typed byte array) | `[$B` |
| BJData type | JSON value type | marker |
| --------------------- | ------------------------------------------------ | -------- |
| no-op | *no value, next value is read* | `N` |
| null | `null` | `Z` |
| false | `false` | `F` |
| true | `true` | `T` |
| float16 | number_float | `h` |
| float32 | number_float | `d` |
| float64 | number_float | `D` |
| high-precision number | number_integer, number_unsigned, or number_float | `H` |
| uint8 | number_unsigned | `U` |
| int8 | number_integer | `i` |
| uint16 | number_unsigned | `u` |
| int16 | number_integer | `I` |
| uint32 | number_unsigned | `m` |
| int32 | number_integer | `l` |
| uint64 | number_unsigned | `M` |
| int64 | number_integer | `L` |
| byte | number_unsigned | `B` |
| string | string | `S` |
| char | string | `C` |
| array | array (optimized values are supported) | `[` |
| ND-array | object (in JData annotated array format) | `[$.#[.` |
| object | object (optimized values are supported) | `{` |
| binary | binary (strongly-typed byte array) | `[$B` |
Complete mapping
The mapping is **complete** in the sense that any BJData value can be converted to a JSON value.
The mapping is **complete** for the BJData types of Draft 2 and Draft 3 with one exception: column-major N-dimensional arrays are rejected with [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113). The Draft 4 constructs (structure-of-arrays containers and the extension type `E`) are not supported either.
The digits of an `H` value are parsed like a JSON number: integers that fit 64 bits become `number_integer` or `number_unsigned`, anything else a `number_float`; invalid text throws [`parse_error.115`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error115).
Duplicate object keys
If an object contains the same key more than once, `from_bjdata` accepts it and keeps the last value; with `ordered_json`, the key keeps the position of its first occurrence.
Ill-formed UTF-8 in string values and object keys
@@ -281,7 +307,7 @@ Example: deserialize a JSON value from BJData
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -295,7 +321,7 @@ int main()
json j = json::from_bjdata(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/bon8.md
+52 -44
View File
@@ -1,6 +1,6 @@
# BON8
BON8 (Binary Object Notation 8) is a compact binary serialization format for JSON values. It uses the byte values that
BON8 (Binary Object Notation 8) is a compact binary serialization format for JSON values. It uses byte values that
cannot begin a UTF-8 character as type markers, so strings are stored as plain UTF-8 without a length prefix: a string
ends at the first byte that cannot continue it. Integers from -10 to 39, `true`, `false`, `null`, and the floating-point
values -1.0, 0.0, and 1.0 take a single byte, and arrays and objects with up to four elements need no terminator.
@@ -10,6 +10,39 @@ values -1.0, 0.0, and 1.0 take a single byte, and arrays and objects with up to
- [BON8 specification](https://github.com/hikoworks/hikogui/blob/main/docs/BON8.md)
- [Reference implementation](https://github.com/hikoworks/hikogui/blob/main/src/hikogui/codec/BON8.hpp) in HikoGUI
## Byte layout
BON8 exploits the fact that some byte values can never start a UTF-8 character. These bytes serve as *type markers*, so strings can be stored as plain UTF-8 without a length prefix: a string ends at the first byte that cannot continue it: a marker byte, or a lead byte 0xC2..0xF7 followed by a byte outside 0x80..0xBF. The ending byte is often the marker of the next value. Small integers, `true`, `false`, `null`, and a few common floats are a single byte. Arrays and objects are introduced by a marker; those with up to four elements encode their size in the marker and need no terminator, while larger ones are closed by the end marker `0xFE`.
Consider the following JSON value (shown formatted for readability):
```json
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_bon8` encodes it to 35 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.
<div class="byte-layout" markdown="0"><table>
<thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody>
<tr><td><code>{</code></td><td><span class="bl bl-t">8A</span></td><td>object with 4 members</td></tr>
<tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-d">69 64</span> <span class="bl bl-t">C4</span> <span class="bl bl-d">04</span></td><td>key: UTF-8 bytes; the string ends at 0xC4, because the byte after it (0x04) is not a UTF-8 continuation byte, so 0xC4 starts an integer; value: 2-byte positive integer (lead byte, then a byte below 0x80)</td></tr>
<tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-l">FF</span> <span class="bl bl-d">6A 73 6F 6E</span> <span class="bl bl-l">FF</span></td><td>key: UTF-8 bytes; 0xFF terminates the string as another string follows; value: UTF-8 bytes; terminated as a string key follows</td></tr>
<tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-d">70 69</span> <span class="bl bl-t">8F</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: UTF-8 bytes; value: binary64 (IEEE 754, big-endian); its marker also ends <code>&quot;pi&quot;</code></td></tr>
<tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">82</span></td><td>key: UTF-8 bytes; value: array with 2 elements; its marker also ends <code>&quot;tags&quot;</code></td></tr>
<tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-d">61</span> <span class="bl bl-l">FF</span></td><td>UTF-8 bytes; terminated as a string follows</td></tr>
<tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-d">62</span> <span class="bl bl-l">FF</span></td><td>UTF-8 bytes; terminated as nothing follows</td></tr>
<tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr>
<tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr>
</tbody></table></div>
## Serialization
The library uses the following mapping from JSON values types to BON8 types according to the BON8 specification:
@@ -67,7 +100,8 @@ serialized as 0x81 0x65 0xFF, but `[1,2,3,4,"e"]` as 0x85 0x91 0x92 0x93 0x94 0x
The following values can **not** be converted to a BON8 value:
- unsigned integers above 9223372036854775807, because BON8 has no unsigned 64-bit integer type
- integers outside the int64 range (unsigned integers above 9223372036854775807 and, for number types wider than
64 bits, any larger or smaller integer), because BON8 integers are limited to int64
([out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407))
- strings that are not valid UTF-8, because the end of a string is determined from its encoding
([type_error.316](../../home/exceptions.md#jsonexceptiontype_error316))
@@ -100,7 +134,7 @@ serialized as 0x81 0x65 0xFF, but `[1,2,3,4,"e"]` as 0x85 0x91 0x92 0x93 0x94 0x
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
using namespace nlohmann::literals;
int main()
@@ -114,9 +148,9 @@ serialized as 0x81 0x65 0xFF, but `[1,2,3,4,"e"]` as 0x85 0x91 0x92 0x93 0x94 0x
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << std::endl;
std::cout << '\n';
}
```
@@ -128,41 +162,6 @@ serialized as 0x81 0x65 0xFF, but `[1,2,3,4,"e"]` as 0x85 0x91 0x92 0x93 0x94 0x
```
### Byte layout
Consider the following JSON value (shown formatted for readability):
```json
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_bon8` encodes it to 35 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.
<div class="byte-layout" markdown="0"><table>
<thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody>
<tr><td><code>{</code></td><td><span class="bl bl-t">8A</span></td><td>object with 4 members</td></tr>
<tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-d">69 64</span> <span class="bl bl-t">C4</span> <span class="bl bl-d">04</span></td><td>key: UTF-8 bytes; the string ends at the next byte, which cannot continue a UTF-8 character; value: 2-byte positive integer (lead byte, then a byte below 0x80)</td></tr>
<tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-l">FF</span> <span class="bl bl-d">6A 73 6F 6E</span> <span class="bl bl-l">FF</span></td><td>key: UTF-8 bytes; 0xFF terminates the string as another string follows; value: UTF-8 bytes; terminated as a string key follows</td></tr>
<tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-d">70 69</span> <span class="bl bl-t">8F</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: UTF-8 bytes; value: binary64 (IEEE 754, big-endian); its marker also ends <code>&quot;pi&quot;</code></td></tr>
<tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">82</span></td><td>key: UTF-8 bytes; value: array with 2 elements; its marker also ends <code>&quot;tags&quot;</code></td></tr>
<tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-d">61</span> <span class="bl bl-l">FF</span></td><td>UTF-8 bytes; terminated as a string follows</td></tr>
<tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-d">62</span> <span class="bl bl-l">FF</span></td><td>UTF-8 bytes; terminated as nothing follows</td></tr>
<tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr>
<tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr>
</tbody></table></div>
## Deserialization
The library maps BON8 types to JSON value types as follows:
@@ -193,7 +192,13 @@ The library maps BON8 types to JSON value types as follows:
Non-negative integers are read as number_unsigned, negative integers as number_integer.
!!! info
A byte 0xF5..0xF7 followed by 0x80..0xBF is neither a string nor an integer and is rejected with
[parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) (invalid UTF-8 byte). The same error is
reported for any other invalid byte (such as a stray 0xFE), for ill-formed UTF-8, and for an object key that is not a
string. Input that ends prematurely, trailing data after the value, and a missing final 0xFF after a string at the end
of the message are reported as [parse_error.110](../../home/exceptions.md#jsonexceptionparse_error110).
!!! info "Non-canonical input"
Values that do not use the canonical representation, such as integers with a longer encoding than necessary,
arrays and objects with up to four elements that are terminated by 0xFE, unsorted object keys, or a 0xFF after a
@@ -201,7 +206,10 @@ Non-negative integers are read as number_unsigned, negative integers as number_i
Strings must be valid UTF-8, and a string at the very end of a message must be terminated by 0xFF.
!!! info
Duplicate object keys are accepted; the last value wins (`ordered_json` keeps the position of the first occurrence).
The specification does not say how to treat them; the HikoGUI reference decoder keeps the first value.
!!! info "Round trips"
Any BON8 output created by `to_bon8` can be successfully parsed by `from_bon8`.
@@ -212,7 +220,7 @@ Non-negative integers are read as number_unsigned, negative integers as number_i
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -227,7 +235,7 @@ Non-negative integers are read as number_unsigned, negative integers as number_i
json j = json::from_bon8(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/bon8/index.html
+17 -17
View File
@@ -1,8 +1,17 @@
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/bon8/" rel="canonical"><link href=../bjdata/ rel=prev><link href=../bson/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>BON8 - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#bon8 class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> BON8 </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div class=md-searchLine truncated
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/bon8/" rel="canonical"><link href=../bjdata/ rel=prev><link href=../bson/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>BON8 - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#bon8 class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> BON8 </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div class=md-searchLine truncated
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_bon8</code> encodes it to 35 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">8A</span></td><td>object with 4 members</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-d">69 64</span> <span class="bl bl-t">C4</span> <span class="bl bl-d">04</span></td><td>key: UTF-8 bytes; the string ends at 0xC4, because the byte after it (0x04) is not a UTF-8 continuation byte, so 0xC4 starts an integer; value: 2-byte positive integer (lead byte, then a byte below 0x80)</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-l">FF</span> <span class="bl bl-d">6A 73 6F 6E</span> <span class="bl bl-l">FF</span></td><td>key: UTF-8 bytes; 0xFF terminates the string as another string follows; value: UTF-8 bytes; terminated as a string key follows</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-d">70 69</span> <span class="bl bl-t">8F</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: UTF-8 bytes; value: binary64 (IEEE 754, big-endian); its marker also ends <code>&quot;pi&quot;</code></td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">82</span></td><td>key: UTF-8 bytes; value: array with 2 elements; its marker also ends <code>&quot;tags&quot;</code></td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-d">61</span> <span class="bl bl-l">FF</span></td><td>UTF-8 bytes; terminated as a string follows</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-d">62</span> <span class="bl bl-l">FF</span></td><td>UTF-8 bytes; terminated as nothing follows</td></tr> <tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr> <tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr> </tbody></table></div> <h2 id=serialization>Serialization<a class=headerlink href=#serialization title="Permanent link">&para;</a></h2> <p>The library uses the following mapping from <abbr title="JavaScript Object Notation">JSON</abbr> values types to <abbr title="Binary Object Notation 8">BON8</abbr> types according to the <abbr title="Binary Object Notation 8">BON8</abbr> specification:</p> <table> <thead> <tr> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>value/range</th> <th><abbr title="Binary Object Notation 8">BON8</abbr> type</th> <th>first byte</th> </tr> </thead> <tbody> <tr> <td>null</td> <td><code>null</code></td> <td>null</td> <td>0xFA</td> </tr> <tr> <td>boolean</td> <td><code>true</code></td> <td>true</td> <td>0xF9</td> </tr> <tr> <td>boolean</td> <td><code>false</code></td> <td>false</td> <td>0xF8</td> </tr> <tr> <td>number_integer</td> <td>-9223372036854775808..-2147483649</td> <td>int64</td> <td>0x8D</td> </tr> <tr> <td>number_integer</td> <td>-2147483648..-33818507</td> <td>int32</td> <td>0x8C</td> </tr> <tr> <td>number_integer</td> <td>-33818506..-264075</td> <td>4-byte negative integer</td> <td>0xF0..0xF7</td> </tr> <tr> <td>number_integer</td> <td>-264074..-1931</td> <td>3-byte negative integer</td> <td>0xE0..0xEF</td> </tr> <tr> <td>number_integer</td> <td>-1930..-11</td> <td>2-byte negative integer</td> <td>0xC2..0xDF</td> </tr> <tr> <td>number_integer</td> <td>-10..-1</td> <td>1-byte negative integer</td> <td>0xB8..0xC1</td> </tr> <tr> <td>number_integer</td> <td>0..39</td> <td>1-byte positive integer</td> <td>0x90..0xB7</td> </tr> <tr> <td>number_integer</td> <td>40..3879</td> <td>2-byte positive integer</td> <td>0xC2..0xDF</td> </tr> <tr> <td>number_integer</td> <td>3880..528167</td> <td>3-byte positive integer</td> <td>0xE0..0xEF</td> </tr> <tr> <td>number_integer</td> <td>528168..67637031</td> <td>4-byte positive integer</td> <td>0xF0..0xF7</td> </tr> <tr> <td>number_integer</td> <td>67637032..2147483647</td> <td>int32</td> <td>0x8C</td> </tr> <tr> <td>number_integer</td> <td>2147483648..9223372036854775807</td> <td>int64</td> <td>0x8D</td> </tr> <tr> <td>number_unsigned</td> <td>0..39</td> <td>1-byte positive integer</td> <td>0x90..0xB7</td> </tr> <tr> <td>number_unsigned</td> <td>40..3879</td> <td>2-byte positive integer</td> <td>0xC2..0xDF</td> </tr> <tr> <td>number_unsigned</td> <td>3880..528167</td> <td>3-byte positive integer</td> <td>0xE0..0xEF</td> </tr> <tr> <td>number_unsigned</td> <td>528168..67637031</td> <td>4-byte positive integer</td> <td>0xF0..0xF7</td> </tr> <tr> <td>number_unsigned</td> <td>67637032..2147483647</td> <td>int32</td> <td>0x8C</td> </tr> <tr> <tLine truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=k>namespace</span><span class=w> </span><span class=nn>nlohmann</span><span class=o>::</span><span class=nn>literals</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
@@ -16,25 +25,16 @@
<span class=w> </span><span class=c1>// print the vector content</span>
<span class=w> </span><span class=k>for</span><span class=w> </span><span class=p>(</span><span class=k>auto</span><span class=o>&amp;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>:</span><span class=w> </span><span class=n>v</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>"0x"</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>hex</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setfill</span><span class=p>(</span><span class=sc>'0'</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=p>(</span><span class=kt>int</span><span class=p>)</span><span class=kt>byte</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>" "</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>"0x"</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>hex</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setfill</span><span class=p>(</span><span class=sc>'0'</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=k>static_cast</span><span class=o>&lt;</span><span class=kt>int</span><span class=o>&gt;</span><span class=p>(</span><span class=kt>byte</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>" "</span><span class=p>;</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=mh>0x89</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x6f</span><span class=w> </span><span class=mh>0x6d</span><span class=w> </span><span class=mh>0x70</span><span class=w> </span><span class=mh>0x61</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x74</span><span class=w> </span><span class=mh>0xf9</span><span class=w> </span><span class=mh>0x66</span><span class=w> </span><span class=mh>0x6f</span><span class=w> </span><span class=mh>0x72</span><span class=w> </span><span class=mh>0x6d</span><span class=w> </span><span class=mh>0x61</span><span class=w> </span><span class=mh>0x74</span><span class=w> </span><span class=mh>0xff</span><span class=w> </span><span class=mh>0x42</span><span class=w> </span><span class=mh>0x4f</span><span class=w> </span><span class=mh>0x4e</span><span class=w> </span><span class=mh>0x38</span><span class=w> </span><span class=mh>0xff</span><span class=w> </span><span class=mh>0x73</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x68</span><span class=w> </span><span class=mh>0x65</span><span class=w> </span><span class=mh>0x6d</span><span class=w> </span><span class=mh>0x61</span><span class=w> </span><span class=mh>0x90</span><span class=w> </span>
</code></pre></div> </details> <h3 id=byte-layout>Byte layout<a class=headerlink href=#byte-layout title="Permanent link">&para;</a></h3> <p>Consider the following <abbr title="JavaScript Object Notation">JSON</abbr> value (shown formatted for readability):</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_bon8</code> encodes it to 35 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">8A</span></td><td>object with 4 members</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-d">69 64</span> <span class="bl bl-t">C4</span> <span class="bl bl-d">04</span></td><td>key: UTF-8 bytes; the string ends at the next byte, which cannot continue a UTF-8 character; value: 2-byte positive integer (lead byte, then a byte below 0x80)</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-l">FF</span> <span class="bl bl-d">6A 73 6F 6E</span> <span class="bl bl-l">FF</span></td><td>key: UTF-8 bytes; 0xFF terminates the string as another string follows; value: UTF-8 bytes; terminated as a string key follows</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-d">70 69</span> <span class="bl bl-t">8F</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: UTF-8 bytes; value: binary64 (IEEE 754, big-endian); its marker also ends <code>&quot;pi&quot;</code></td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">82</span></td><td>key: UTF-8 bytes; value: array with 2 elements; its marker also ends <code>&quot;tags&quot;</code></td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-d">61</span> <span class="bl bl-l">FF</span></td><td>UTF-8 bytes; terminated as a string follows</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-d">62</span> <span class="bl bl-l">FF</span></td><td>UTF-8 bytes; terminated as nothing follows</td></tr> <tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr> <tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr> </tbody></table></div> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Binary Object Notation 8">BON8</abbr> types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Binary Object Notation 8">BON8</abbr> type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>first byte</th> </tr> </thead> <tbody> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x00..0x7F</td> </tr> <tr> <td>array with count</td> <td>array</td> <td>0x80..0x84</td> </tr> <tr> <td>array (terminated by 0xFE)</td> <td>array</td> <td>0x85</td> </tr> <tr> <td>object with count</td> <td>object</td> <td>0x86..0x8A</td> </tr> <tr> <td>object (terminated by 0xFE)</td> <td>object</td> <td>0x8B</td> </tr> <tr> <td>int32</td> <td>number_unsigned or number_integer</td> <td>0x8C</td> </tr> <tr> <td>int64</td> <td>number_unsigned or number_integer</td> <td>0x8D</td> </tr> <tr> <td>binary32</td> <td>number_float</td> <td>0x8E</td> </tr> <tr> <td>binary64</td> <td>number_float</td> <td>0x8F</td> </tr> <tr> <td>1-byte positive integer</td> <td>number_unsigned</td> <td>0x90..0xB7</td> </tr> <tr> <td>1-byte negative integer</td> <td>number_integer</td> <td>0xB8..0xC1</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0xC2..0xF4, followed by 0x80..0xBF</td> </tr> <tr> <td>2- to 4-byte positive integer</td> <td>number_unsigned</td> <td>0xC2..0xF7, followed by 0x00..0x7F</td> </tr> <tr> <td>2- to 4-byte negative integer</td> <td>number_integer</td> <td>0xC2..0xF7, followed by 0xC0..0xFF</td> </tr> <tr> <td>false</td> <td><code>false</code></td> <td>0xF8</td> </tr> <tr> <td>true</td> <td><code>true</code></td> <td>0xF9</td> </tr> <tr> <td>null</td> <td><code>null</code></td> <td>0xFA</td> </tr> <tr> <td>-1.0</td> <td>number_float</td> <td>0xFB</td> </tr> <tr> <td>0.0</td> <td>number_float</td> <td>0xFC</td> </tr> <tr> <td>1.0</td> <td>number_float</td> <td>0xFD</td> </tr> <tr> <td>empty string</td> <td>string</td> <td>0xFF</td> </tr> </tbody> </table> <p>Non-negative integers are read as number_unsigned, negative integers as number_integer.</p> <div class="admonition info"> <p class=admonition-title>Info</p> <p>Values that do not use the canonical representation, such as integers with a longer encoding than necessary, arrays and objects with up to four elements that are terminated by 0xFE, unsorted object keys, or a 0xFF after a string that would also end without it, are accepted. A second 0xFF is not a terminator but an empty Line truncated
</code></pre></div> </details> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Binary Object Notation 8">BON8</abbr> types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Binary Object Notation 8">BON8</abbr> type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>first byte</th> </tr> </thead> <tbody> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x00..0x7F</td> </tr> <tr> <td>array with count</td> <td>array</td> <td>0x80..0x84</td> </tr> <tr> <td>array (terminated by 0xFE)</td> <td>array</td> <td>0x85</td> </tr> <tr> <td>object with count</td> <td>object</td> <td>0x86..0x8A</td> </tr> <tr> <td>object (terminated by 0xFE)</td> <td>object</td> <td>0x8B</td> </tr> <tr> <td>int32</td> <td>number_unsigned or number_integer</td> <td>0x8C</td> </tr> <tr> <td>int64</td> <td>number_unsigned or number_integer</td> <td>0x8D</td> </tr> <tr> <td>binary32</td> <td>number_float</td> <td>0x8E</td> </tr> <tr> <td>binary64</td> <td>number_float</td> <td>0x8F</td> </tr> <tr> <td>1-byte positive integer</td> <td>number_unsigned</td> <td>0x90..0xB7</td> </tr> <tr> <td>1-byte negative integer</td> <td>number_integer</td> <td>0xB8..0xC1</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0xC2..0xF4, followed by 0x80..0xBF</td> </tr> <tr> <td>2- to 4-byte positive integer</td> <td>number_unsigned</td> <td>0xC2..0xF7, followed by 0x00..0x7F</td> </tr> <tr> <td>2- to 4-byte negative integer</td> <td>number_integer</td> <td>0xC2..0xF7, followed by 0xC0..0xFF</td> </tr> <tr> <td>false</td> <td><code>false</code></td> <td>0xF8</td> </tr> <tr> <td>true</td> <td><code>true</code></td> <td>0xF9</td> </tr> <tr> <td>null</td> <td><code>null</code></td> <td>0xFA</td> </tr> <tr> <td>-1.0</td> <td>number_float</td> <td>0xFB</td> </tr> <tr> <td>0.0</td> <td>number_float</td> <td>0xFC</td> </tr> <tr> <td>1.0</td> <td>number_float</td> <td>0xFD</td> </tr> <tr> <td>empty string</td> <td>string</td> <td>0xFF</td> </tr> </tbody> </table> <p>Non-negative integers are read as number_unsigned, negative integers as number_integer.</p> <p>A byte 0xF5..0xF7 followed by 0x80..0xBF is neither a string nor an integer and is rejected with <a href=../../../home/exceptions/#jsonexceptionparse_error112>parse_error.112</a> (invalid <abbr title="Unicode Transformation Format">UTF</abbr>-8 byte). The same error is reported for any other invalid byte (such as a stray 0xFE), for ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8, and for an object key that is not a string. Input that ends prematurely, trailing data after the value, and a missing final 0xFF after a string at the end of the message are reported as <a href=../../../home/exceptions/#jsonexceptionparse_error110>parse_error.110</a>.</p> <div class="admonition info"> <p class=admonition-title>Non-canonical input</p> <p>Values that do not use the canonical representation, such as integers with a longer encoding than necessary, arrays and objects with up to four elements that are terminated by 0xFE, unsorted object keys, or a 0xFF after a string that would also end without it, are accepted. A second 0xFF is not a terminator but an empty string.</p> <p>Strings must be valid <abbr title="Unicode Transformation Format">UTF</abbr>-8, and a string at the very end of a message must be terminated by 0xFF.</p> <p>Duplicate object keys are accepted; the last value wins (<code>ordered_json</code> keeps the position of the first occurrence). The specification does not say how to treat them; the HikoGUI reference decoder keeps the first value.</p> </div> <div class="admonition info"> <p class=admonition-title>Round trips</p> <p>Any <abbr title="Binary Object Notation 8">BON8</abbr> output created by <code>to_bon8</code> can be successfully parsed by <code>from_bon8</code>.</p> </div> <details class=example> <summary>Example: deserialize a <abbr title="JavaScript Object Notation">JSON</abbr> value from <abbr title="Binary Object Notation 8">BON8</abbr></summary> <div class=highlight><pre><span></span><code><span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iostream&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
<span class=p>{</span>
@@ -49,11 +49,11 @@
<span class=w> </span><span class=n>json</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>json</span><span class=o>::</span><span class=n>from_bon8</span><span class=p>(</span><span class=n>v</span><span class=p>);</span>
<span class=w> </span><span class=c1>// print the deserialized JSON value</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"compact"</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=p>,</span>
<span class=w> </span><span class=nt>"format"</span><span class=p>:</span><span class=w> </span><span class=s2>"BON8"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"schema"</span><span class=p>:</span><span class=w> </span><span class=mi>0</span>
<span class=p>}</span>
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 24, 2026 15:57:44 UTC">October 24, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 26, 2026 17:25:46 UTC">October 26, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
features/binary_formats/bon8/index.md
+45 -39
View File
@@ -1,12 +1,44 @@
# BON8
BON8 (Binary Object Notation 8) is a compact binary serialization format for JSON values. It uses the byte values that cannot begin a UTF-8 character as type markers, so strings are stored as plain UTF-8 without a length prefix: a string ends at the first byte that cannot continue it. Integers from -10 to 39, `true`, `false`, `null`, and the floating-point values -1.0, 0.0, and 1.0 take a single byte, and arrays and objects with up to four elements need no terminator.
BON8 (Binary Object Notation 8) is a compact binary serialization format for JSON values. It uses byte values that cannot begin a UTF-8 character as type markers, so strings are stored as plain UTF-8 without a length prefix: a string ends at the first byte that cannot continue it. Integers from -10 to 39, `true`, `false`, `null`, and the floating-point values -1.0, 0.0, and 1.0 take a single byte, and arrays and objects with up to four elements need no terminator.
References
- [BON8 specification](https://github.com/hikoworks/hikogui/blob/main/docs/BON8.md)
- [Reference implementation](https://github.com/hikoworks/hikogui/blob/main/src/hikogui/codec/BON8.hpp) in HikoGUI
## Byte layout
BON8 exploits the fact that some byte values can never start a UTF-8 character. These bytes serve as *type markers*, so strings can be stored as plain UTF-8 without a length prefix: a string ends at the first byte that cannot continue it: a marker byte, or a lead byte 0xC2..0xF7 followed by a byte outside 0x80..0xBF. The ending byte is often the marker of the next value. Small integers, `true`, `false`, `null`, and a few common floats are a single byte. Arrays and objects are introduced by a marker; those with up to four elements encode their size in the marker and need no terminator, while larger ones are closed by the end marker `0xFE`.
Consider the following JSON value (shown formatted for readability):
```
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_bon8` encodes it to 35 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: type/marker length/count/size data.
| JSON | Bytes | Meaning |
| ----------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `{` | 8A | object with 4 members |
| `"id": 300,` | 69 64 C4 04 | key: UTF-8 bytes; the string ends at 0xC4, because the byte after it (0x04) is not a UTF-8 continuation byte, so 0xC4 starts an integer; value: 2-byte positive integer (lead byte, then a byte below 0x80) |
| `"name": "json",` | 6E 61 6D 65 FF 6A 73 6F 6E FF | key: UTF-8 bytes; 0xFF terminates the string as another string follows; value: UTF-8 bytes; terminated as a string key follows |
| `"pi": 3.14,` | 70 69 8F 40 09 1E B8 51 EB 85 1F | key: UTF-8 bytes; value: binary64 (IEEE 754, big-endian); its marker also ends `"pi"` |
| `"tags": [` | 74 61 67 73 82 | key: UTF-8 bytes; value: array with 2 elements; its marker also ends `"tags"` |
| `"a",` | 61 FF | UTF-8 bytes; terminated as a string follows |
| `"b"` | 62 FF | UTF-8 bytes; terminated as nothing follows |
| `]` | | no end marker: the element count was given up front |
| `}` | | no end marker: the member count was given up front |
## Serialization
The library uses the following mapping from JSON values types to BON8 types according to the BON8 specification:
@@ -60,7 +92,7 @@ Unsupported values
The following values can **not** be converted to a BON8 value:
- unsigned integers above 9223372036854775807, because BON8 has no unsigned 64-bit integer type ([out_of_range.407](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range407))
- integers outside the int64 range (unsigned integers above 9223372036854775807 and, for number types wider than 64 bits, any larger or smaller integer), because BON8 integers are limited to int64 ([out_of_range.407](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range407))
- strings that are not valid UTF-8, because the end of a string is determined from its encoding ([type_error.316](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316))
NaN/infinity handling
@@ -85,7 +117,7 @@ Example: serialize a JSON value to BON8
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
using namespace nlohmann::literals;
int main()
@@ -99,9 +131,9 @@ int main()
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << std::endl;
std::cout << '\n';
}
```
@@ -111,36 +143,6 @@ Output:
0x89 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0xf9 0x66 0x6f 0x72 0x6d 0x61 0x74 0xff 0x42 0x4f 0x4e 0x38 0xff 0x73 0x63 0x68 0x65 0x6d 0x61 0x90
```
### Byte layout
Consider the following JSON value (shown formatted for readability):
```
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_bon8` encodes it to 35 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: type/marker length/count/size data.
| JSON | Bytes | Meaning |
| ----------------- | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `{` | 8A | object with 4 members |
| `"id": 300,` | 69 64 C4 04 | key: UTF-8 bytes; the string ends at the next byte, which cannot continue a UTF-8 character; value: 2-byte positive integer (lead byte, then a byte below 0x80) |
| `"name": "json",` | 6E 61 6D 65 FF 6A 73 6F 6E FF | key: UTF-8 bytes; 0xFF terminates the string as another string follows; value: UTF-8 bytes; terminated as a string key follows |
| `"pi": 3.14,` | 70 69 8F 40 09 1E B8 51 EB 85 1F | key: UTF-8 bytes; value: binary64 (IEEE 754, big-endian); its marker also ends `"pi"` |
| `"tags": [` | 74 61 67 73 82 | key: UTF-8 bytes; value: array with 2 elements; its marker also ends `"tags"` |
| `"a",` | 61 FF | UTF-8 bytes; terminated as a string follows |
| `"b"` | 62 FF | UTF-8 bytes; terminated as nothing follows |
| `]` | | no end marker: the element count was given up front |
| `}` | | no end marker: the member count was given up front |
## Deserialization
The library maps BON8 types to JSON value types as follows:
@@ -171,13 +173,17 @@ The library maps BON8 types to JSON value types as follows:
Non-negative integers are read as number_unsigned, negative integers as number_integer.
Info
A byte 0xF5..0xF7 followed by 0x80..0xBF is neither a string nor an integer and is rejected with [parse_error.112](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error112) (invalid UTF-8 byte). The same error is reported for any other invalid byte (such as a stray 0xFE), for ill-formed UTF-8, and for an object key that is not a string. Input that ends prematurely, trailing data after the value, and a missing final 0xFF after a string at the end of the message are reported as [parse_error.110](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error110).
Non-canonical input
Values that do not use the canonical representation, such as integers with a longer encoding than necessary, arrays and objects with up to four elements that are terminated by 0xFE, unsorted object keys, or a 0xFF after a string that would also end without it, are accepted. A second 0xFF is not a terminator but an empty string.
Strings must be valid UTF-8, and a string at the very end of a message must be terminated by 0xFF.
Info
Duplicate object keys are accepted; the last value wins (`ordered_json` keeps the position of the first occurrence). The specification does not say how to treat them; the HikoGUI reference decoder keeps the first value.
Round trips
Any BON8 output created by `to_bon8` can be successfully parsed by `from_bon8`.
@@ -188,7 +194,7 @@ Example: deserialize a JSON value from BON8
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -203,7 +209,7 @@ int main()
json j = json::from_bon8(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/bson.md
+117 -84
View File
@@ -10,85 +10,9 @@ representation of data types that are not part of the JSON spec. For example, BS
- [BSON Specification](http://bsonspec.org/spec.html) - the specification
## Serialization
## Byte layout
The library uses the following mapping from JSON values types to BSON types:
| JSON value type | value/range | BSON type | marker |
|-----------------|-------------------------------------------|-----------|--------|
| null | `null` | null | 0x0A |
| boolean | `true`, `false` | boolean | 0x08 |
| number_integer | -9223372036854775808..-2147483649 | int64 | 0x12 |
| number_integer | -2147483648..2147483647 | int32 | 0x10 |
| number_integer | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 0..2147483647 | int32 | 0x10 |
| number_unsigned | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | 0x11 |
| number_float | *any value* | double | 0x01 |
| string | *any value* | string | 0x02 |
| array | *any value* | document | 0x04 |
| object | *any value* | document | 0x03 |
| binary | *any value* | binary | 0x05 |
!!! warning "Incomplete mapping"
The mapping is **incomplete**, since only JSON-objects (and things contained therein) can be serialized to BSON.
Also, keys may not contain U+0000, since they are serialized a zero-terminated c-strings.
!!! warning "BSON type 0x11 interoperability"
The BSON specification defines type `0x11` as a Timestamp. This library uses marker `0x11` when serializing
`number_unsigned` values in the range `9223372036854775808..18446744073709551615`. Other BSON implementations may
therefore interpret these values as Timestamps instead of unsigned integers.
!!! info "Binary values without a subtype"
BSON requires every binary value to have a subtype. If a binary value has no subtype, this library serializes it
with the generic subtype `0x00`. After deserialization, `has_subtype()` returns `true` and `subtype()` returns `0`.
As a result, serializing and deserializing a JSON object containing such a value produces a different JSON object,
even though the binary data is unchanged.
??? example "Example: serialize a JSON value to BSON"
```cpp
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using namespace nlohmann::literals;
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": 0})"_json;
// serialize it to BSON
std::vector<std::uint8_t> v = json::to_bson(j);
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
}
std::cout << std::endl;
}
```
Output:
```c
0x1b 0x00 0x00 0x00 0x08 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0x00 0x01 0x10 0x73 0x63 0x68 0x65 0x6d 0x61 0x00 0x00 0x00 0x00 0x00 0x00
```
### Byte layout
A BSON document is a length-prefixed sequence of *elements*. The first four bytes hold the total size of the document (little-endian int32), followed by the elements, and a final `0x00` byte ends the document. Each element starts with a one-byte type, followed by its key as a NUL-terminated string and the value, whose encoding depends on the type. Arrays are documents whose keys are the decimal indices `"0"`, `"1"`, ...; nesting is expressed by embedding documents. Because every document announces its size, a reader can skip over it without parsing its content. The top level of a BSON value is always a document, which is why only JSON objects can be serialized.
Consider the following JSON value (shown formatted for readability):
@@ -119,6 +43,97 @@ Consider the following JSON value (shown formatted for readability):
<tr><td><code>}</code></td><td><span class="bl bl-t">00</span></td><td>end of the document</td></tr>
</tbody></table></div>
## Serialization
The library uses the following mapping from JSON values types to BSON types:
| JSON value type | value/range | BSON type | marker |
|-----------------|-------------------------------------------|-----------|--------|
| null | `null` | null | 0x0A |
| boolean | `true`, `false` | boolean | 0x08 |
| number_integer | -9223372036854775808..-2147483649 | int64 | 0x12 |
| number_integer | -2147483648..2147483647 | int32 | 0x10 |
| number_integer | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 0..2147483647 | int32 | 0x10 |
| number_unsigned | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | 0x11 |
| number_float | *any value* | double | 0x01 |
| string | *any value* | string | 0x02 |
| array | *any value* | document | 0x04 |
| object | *any value* | document | 0x03 |
| binary | *any value* | binary | 0x05 |
!!! warning "Incomplete mapping"
The mapping is **incomplete**, since only JSON-objects (and things contained therein) can be serialized to BSON.
Also, keys may not contain U+0000, since they are serialized as zero-terminated C strings.
!!! info "Order of object members"
The members of an object are written in the order in which iteration visits them: sorted by key for
[`json`](../../api/json.md), in insertion order for [`ordered_json`](../../api/ordered_json.md).
!!! warning "BSON type 0x11 interoperability"
The BSON specification defines type `0x11` as a Timestamp. This library uses marker `0x11` when serializing
`number_unsigned` values in the range `9223372036854775808..18446744073709551615`. Other BSON implementations may
therefore interpret these values as Timestamps instead of unsigned integers.
!!! info "Binary values without a subtype"
BSON requires every binary value to have a subtype. If a binary value has no subtype, this library serializes it
with the generic subtype `0x00`. After deserialization, `has_subtype()` returns `true` and `subtype()` returns `0`.
As a result, serializing and deserializing a JSON object containing such a value produces a different JSON object,
even though the binary data is unchanged.
!!! info "Binary subtype 0x02 (old binary)"
The BSON specification defines the payload of the "binary (old)" subtype `0x02` as an int32 followed by that many
bytes, so the value is preceded by two lengths: the length of the binary element and, as the first four bytes of
the payload, the number of bytes that follow. A binary value with subtype `2` is therefore written with the outer
length 4 bytes larger than its size and the size as inner length: `binary({1, 2, 3}, 2)` becomes
`07 00 00 00 02 03 00 00 00 01 02 03`. When reading, the inner length is checked and removed, so the value
contains only the bytes that follow it. `from_bson()` throws
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) if the outer length is less than 4 or the
inner length is not the outer length minus 4. Versions before 3.13.0 did not write the inner length and returned
it as part of the payload; BSON data written by those versions with subtype 2 is not readable any more and must be
written again.
??? example "Example: serialize a JSON value to BSON"
```cpp
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using nlohmann::json;
using namespace nlohmann::literals;
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": 0})"_json;
// serialize it to BSON
std::vector<std::uint8_t> v = json::to_bson(j);
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << '\n';
}
```
Output:
```c
0x1b 0x00 0x00 0x00 0x08 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0x00 0x01 0x10 0x73 0x63 0x68 0x65 0x6d 0x61 0x00 0x00 0x00 0x00 0x00 0x00
```
## Deserialization
The library maps BSON record types to JSON value types as follows:
@@ -161,12 +176,29 @@ The library maps BSON record types to JSON value types as follows:
The BSON reader is lenient in a few areas where the BSON specification is more restrictive:
- array element keys are not checked against the required decimal sequence (`0`, `1`, `2`, ...),
- any non-zero byte is accepted as `true` for the boolean type, and
- the payload for binary subtype `0x02` is returned as-is, including its inner length prefix.
- any non-zero byte is accepted as `true` for the boolean type,
- the payloads of the binary subtypes `0x03` (UUID, old), `0x04` (UUID), and `0x05` (MD5) are not checked to have
16 bytes,
- duplicate keys within a document are accepted; the last value wins (`{"a": 1, "a": 2}` is read as `{"a": 2}`),
while a SAX handler receives every occurrence,
- a `string` value may contain embedded `0x00` bytes, and
- with `strict` set to `false`, bytes after the end of the document are ignored.
If BSON input must be validated for strict specification compliance, validate it separately before passing it to
`from_bson()`.
!!! note "Document size"
Every document starts with its size. `from_bson()` rejects a document with
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) if the size is less than 5 (the size prefix and the terminator), if an embedded document does not end before the
end of the document that contains it, if the elements do not fit into the declared size, or if the size does not
match the number of bytes read when the terminating `0x00` byte is reached. A size that is too small is detected
before an element is read whose type byte and name no longer fit; an element that starts inside the declared size
but extends past it is read, and reported to a SAX handler, before the mismatch is found at the terminator. A size
that is too large is likewise found only at the terminator. Passing `strict` as `false` skips only the check for bytes after
the end of the document ([`parse_error.110`](../../home/exceptions.md#jsonexceptionparse_error110)), not the
document size checks.
!!! warning "Ill-formed UTF-8 in string values"
The BSON specification requires `string` values (type `0x02`) to be valid UTF-8, but this is not required of a
@@ -180,8 +212,9 @@ The library maps BSON record types to JSON value types as follows:
ill-formed bytes. `to_bson()`'s own `error_handler` parameter defaults to `keep`, so such a string value or element
(key) name is written unchanged; with `strict` (the default if
[`JSON_STRICT_BINARY_UTF8`](../../api/macros/json_strict_binary_utf8.md) is enabled), it throws the same exception
instead. Element (key) names are never validated on read, since they are read byte-by-byte as a C string. `binary`
values (type `0x05`) are unaffected, since they are not required to hold text.
instead. Element (key) names are treated like string values: with `keep` they are read unchanged, with `strict`,
`replace`, or `ignore` they are checked and handled in the same way. `binary` values (type `0x05`) are unaffected,
since they are not required to hold text.
??? example "Example: deserialize a JSON value from BSON"
@@ -190,7 +223,7 @@ The library maps BSON record types to JSON value types as follows:
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -205,7 +238,7 @@ The library maps BSON record types to JSON value types as follows:
json j = json::from_bson(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/bson/index.html
+17 -17
View File
@@ -1,8 +1,17 @@
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/bson/" rel="canonical"><link href=../bon8/ rel=prev><link href=../cbor/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>BSON - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#bson class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> BSON </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div class=md-search dLine truncated
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/bson/" rel="canonical"><link href=../bon8/ rel=prev><link href=../cbor/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>BSON - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#bson class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> BSON </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div class=md-search dLine truncated
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_bson</code> encodes it to 69 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-l">45 00 00 00</span></td><td>document size: 69 bytes (int32, little-endian)</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">10</span> <span class="bl bl-d">69 64 00</span> <span class="bl bl-d">2C 01 00 00</span></td><td>int32 element; key as NUL-terminated string; value (little-endian)</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">02</span> <span class="bl bl-d">6E 61 6D 65 00</span> <span class="bl bl-l">05 00 00 00</span> <span class="bl bl-d">6A 73 6F 6E 00</span></td><td>string element; key; length incl. NUL (int32); NUL-terminated UTF-8 bytes</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">01</span> <span class="bl bl-d">70 69 00</span> <span class="bl bl-d">1F 85 EB 51 B8 1E 09 40</span></td><td>double element; key; value (little-endian)</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">04</span> <span class="bl bl-d">74 61 67 73 00</span> <span class="bl bl-l">17 00 00 00</span></td><td>array element; key; embedded document of 23 bytes (keys are the indices)</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">02</span> <span class="bl bl-d">30 00</span> <span class="bl bl-l">02 00 00 00</span> <span class="bl bl-d">61 00</span></td><td>string element; key <code>&quot;0&quot;</code>; length 2; <code>a</code> + NUL</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">02</span> <span class="bl bl-d">31 00</span> <span class="bl bl-l">02 00 00 00</span> <span class="bl bl-d">62 00</span></td><td>string element; key <code>&quot;1&quot;</code>; length 2; <code>b</code> + NUL</td></tr> <tr><td><code>]</code></td><td><span class="bl bl-t">00</span></td><td>end of the array document</td></tr> <tr><td><code>}</code></td><td><span class="bl bl-t">00</span></td><td>end of the document</td></tr> </tbody></table></div> <h2 id=serialization>Serialization<a class=headerlink href=#serialization title="Permanent link">&para;</a></h2> <p>The library uses the following mapping from <abbr title="JavaScript Object Notation">JSON</abbr> values types to <abbr title="Binary JSON">BSON</abbr> types:</p> <table> <thead> <tr> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>value/range</th> <th><abbr title="Binary JSON">BSON</abbr> type</th> <th>marker</th> </tr> </thead> <tbody> <tr> <td>null</td> <td><code>null</code></td> <td>null</td> <td>0x0A</td> </tr> <tr> <td>boolean</td> <td><code>true</code>, <code>false</code></td> <td>boolean</td> <td>0x08</td> </tr> <tr> <td>number_integer</td> <td>-9223372036854775808..-2147483649</td> <td>int64</td> <td>0x12</td> </tr> <tr> <td>number_integer</td> <td>-2147483648..2147483647</td> <td>int32</td> <td>0x10</td> </tr> <tr> <td>number_integer</td> <td>2147483648..9223372036854775807</td> <td>int64</td> <td>0x12</td> </tr> <tr> <td>number_unsigned</td> <td>0..2147483647</td> <td>int32</td> <td>0x10</td> </tr> <tr> <td>number_unsigned</td> <td>2147483648..9223372036854775807</td> <td>int64</td> <td>0x12</td> </tr> <tr> <td>number_unsigned</td> <td>9223372036854775808..18446744073709551615</td> <td>uint64</td> <td>0x11</td> </tr> <tr> <td>number_float</td> <td><em>any value</em></td> <td>double</td> <td>0x01</td> </tr> <tr> <td>string</td> <td><em>any value</em></td> <td>string</td> <td>0x02</td> </tr> <tr> <td>array</td> <td><em>any value</em></td> <td>document</td> <td>0x04</td> </tr> <tr> <td>object</td> <td><em>any value</em></td> <td>document</td> <td>0x03</td> </tr> <tr> <td>binary</td> <td><em>any value</em></td> <td>binary</td> <td>0x05</td> </tr> </tbody> </table> <div class="admonition warning"> <p class=admonition-title>Incomplete mapping</p> <p>The mapping is <strong>incomplete</strong>, since only <abbr title="JavaScript Object Notation">JSON</abbr>-objects (and things contained therein) can be serialized to <abbr title="Binary JSON">BSON</abbr>. Also, keys may not contain U+0000, since they are serialized as zero-terminated C strings.</p> </div> <div class="admonition info"> <p class=admonition-title>Order of object members</p> <p>The members of an object are written in the order in which iteration visits them: sorted by key for <a href=../../../api/json/ ><code>json</code></a>, in insertion order for <a href=../../../api/ordered_json/ ><code>ordered_json</code></a>.</p> </div> <div class="admonition warning"> <p class=admonition-title><abbr title="Binary JSON">BSON</abbr> type 0x11 interoperabilitLine truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=k>namespace</span><span class=w> </span><span class=nn>nlohmann</span><span class=o>::</span><span class=nn>literals</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
@@ -16,25 +25,16 @@
<span class=w> </span><span class=c1>// print the vector content</span>
<span class=w> </span><span class=k>for</span><span class=w> </span><span class=p>(</span><span class=k>auto</span><span class=o>&amp;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>:</span><span class=w> </span><span class=n>v</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>"0x"</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>hex</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setfill</span><span class=p>(</span><span class=sc>'0'</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=p>(</span><span class=kt>int</span><span class=p>)</span><span class=kt>byte</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>" "</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>"0x"</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>hex</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setfill</span><span class=p>(</span><span class=sc>'0'</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=k>static_cast</span><span class=o>&lt;</span><span class=kt>int</span><span class=o>&gt;</span><span class=p>(</span><span class=kt>byte</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>" "</span><span class=p>;</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=mh>0x1b</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span><span class=mh>0x08</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x6f</span><span class=w> </span><span class=mh>0x6d</span><span class=w> </span><span class=mh>0x70</span><span class=w> </span><span class=mh>0x61</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x74</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span><span class=mh>0x01</span><span class=w> </span><span class=mh>0x10</span><span class=w> </span><span class=mh>0x73</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x68</span><span class=w> </span><span class=mh>0x65</span><span class=w> </span><span class=mh>0x6d</span><span class=w> </span><span class=mh>0x61</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span>
</code></pre></div> </details> <h3 id=byte-layout>Byte layout<a class=headerlink href=#byte-layout title="Permanent link">&para;</a></h3> <p>Consider the following <abbr title="JavaScript Object Notation">JSON</abbr> value (shown formatted for readability):</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_bson</code> encodes it to 69 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-l">45 00 00 00</span></td><td>document size: 69 bytes (int32, little-endian)</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">10</span> <span class="bl bl-d">69 64 00</span> <span class="bl bl-d">2C 01 00 00</span></td><td>int32 element; key as NUL-terminated string; value (little-endian)</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">02</span> <span class="bl bl-d">6E 61 6D 65 00</span> <span class="bl bl-l">05 00 00 00</span> <span class="bl bl-d">6A 73 6F 6E 00</span></td><td>string element; key; length incl. NUL (int32); NUL-terminated UTF-8 bytes</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">01</span> <span class="bl bl-d">70 69 00</span> <span class="bl bl-d">1F 85 EB 51 B8 1E 09 40</span></td><td>double element; key; value (little-endian)</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">04</span> <span class="bl bl-d">74 61 67 73 00</span> <span class="bl bl-l">17 00 00 00</span></td><td>array element; key; embedded document of 23 bytes (keys are the indices)</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">02</span> <span class="bl bl-d">30 00</span> <span class="bl bl-l">02 00 00 00</span> <span class="bl bl-d">61 00</span></td><td>string element; key <code>&quot;0&quot;</code>; length 2; <code>a</code> + NUL</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">02</span> <span class="bl bl-d">31 00</span> <span class="bl bl-l">02 00 00 00</span> <span class="bl bl-d">62 00</span></td><td>string element; key <code>&quot;1&quot;</code>; length 2; <code>b</code> + NUL</td></tr> <tr><td><code>]</code></td><td><span class="bl bl-t">00</span></td><td>end of the array document</td></tr> <tr><td><code>}</code></td><td><span class="bl bl-t">00</span></td><td>end of the document</td></tr> </tbody></table></div> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Binary JSON">BSON</abbr> record types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Binary JSON">BSON</abbr> type</th> <th><abbr title="Binary JSON">BSON</abbr> marker byte</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> </tr> </thead> <tbody> <tr> <td>double</td> <td>0x01</td> <td>number_float</td> </tr> <tr> <td>string</td> <td>0x02</td> <td>string</td> </tr> <tr> <td>document</td> <td>0x03</td> <td>object</td> </tr> <tr> <td>array</td> <td>0x04</td> <td>array</td> </tr> <tr> <td>binary</td> <td>0x05</td> <td>binary</td> </tr> <tr> <td>undefined</td> <td>0x06</td> <td><em>unsupported</em></td> </tr> <tr> <td>ObjectId</td> <td>0x07</td> <td><em>unsupported</em></td> </tr> <tr> <td>boolean</td> <td>0x08</td> <td>boolean</td> </tr> <tr> <td>UTC Date-Time</td> <td>0x09</td> <td><em>unsupported</em></td> </tr> <tr> <td>null</td> <td>0x0A</td> <td>null</td> </tr> <tr> <td>Regular Expr.</td> <td>0x0B</td> <td><em>unsupported</em></td> </tr> <tr> <td>DB Pointer</td> <td>0x0C</td> <td><em>unsupported</em></td> </tr> <tr> <td>JavaScript Code</td> <td>0x0D</td> <td><em>unsupported</em></td> </tr> <tr> <td>Symbol</td> <td>0x0E</td> <td><em>unsupported</em></td> </tr> <tr> <td>JavaScript Code w/ scope</td> <td>0x0F</td> <td><em>unsupported</em></td> </tr> <tr> <td>int32</td> <td>0x10</td> <td>number_integer</td> </tr> <tr> <td>uint64(Timestamp)</td> <td>0x11</td> <td>number_unsigned</td> </tr> <tr> <td>int64</td> <td>0x12</td> <td>number_integer</td> </tr> <tr> <td>128-bit decimal float</td> <td>0x13</td> <td><em>unsupported</em></td> </tr> <tr> <td>Max Key</td> <td>0x7F</td> <td><em>unsupported</em></td> </tr> <tr> <td>Min Key</td> <td>0xFF</td> <td><em>unsupported</em></td> </tr> </tbody> </table> <div class="admonition warning"> <p class=admonition-title>Incomplete mapping</p> <p>The mapping is <strong>incomplete</strong>. The unsupported mappings are indicated in the table above.</p> </div> <div class="admonition note"> <p class=admonition-title>Handling of <abbr title="Binary JSON">BSON</abbr> type 0x11</p> <p>This library deserializes <abbr title="Binary JSON">BSON</abbr> type <code>0x11</code> (Timestamp) as a <code>number_unsigned</code> value. The 64-bit value is preserved, but the Timestamp type information is not.</p> </div> <div class="admonition warning"> <p class=admonition-title>Lenient <abbr tLine truncated
</code></pre></div> </details> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Binary JSON">BSON</abbr> record types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Binary JSON">BSON</abbr> type</th> <th><abbr title="Binary JSON">BSON</abbr> marker byte</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> </tr> </thead> <tbody> <tr> <td>double</td> <td>0x01</td> <td>number_float</td> </tr> <tr> <td>string</td> <td>0x02</td> <td>string</td> </tr> <tr> <td>document</td> <td>0x03</td> <td>object</td> </tr> <tr> <td>array</td> <td>0x04</td> <td>array</td> </tr> <tr> <td>binary</td> <td>0x05</td> <td>binary</td> </tr> <tr> <td>undefined</td> <td>0x06</td> <td><em>unsupported</em></td> </tr> <tr> <td>ObjectId</td> <td>0x07</td> <td><em>unsupported</em></td> </tr> <tr> <td>boolean</td> <td>0x08</td> <td>boolean</td> </tr> <tr> <td>UTC Date-Time</td> <td>0x09</td> <td><em>unsupported</em></td> </tr> <tr> <td>null</td> <td>0x0A</td> <td>null</td> </tr> <tr> <td>Regular Expr.</td> <td>0x0B</td> <td><em>unsupported</em></td> </tr> <tr> <td>DB Pointer</td> <td>0x0C</td> <td><em>unsupported</em></td> </tr> <tr> <td>JavaScript Code</td> <td>0x0D</td> <td><em>unsupported</em></td> </tr> <tr> <td>Symbol</td> <td>0x0E</td> <td><em>unsupported</em></td> </tr> <tr> <td>JavaScript Code w/ scope</td> <td>0x0F</td> <td><em>unsupported</em></td> </tr> <tr> <td>int32</td> <td>0x10</td> <td>number_integer</td> </tr> <tr> <td>uint64(Timestamp)</td> <td>0x11</td> <td>number_unsigned</td> </tr> <tr> <td>int64</td> <td>0x12</td> <td>number_integer</td> </tr> <tr> <td>128-bit decimal float</td> <td>0x13</td> <td><em>unsupported</em></td> </tr> <tr> <td>Max Key</td> <td>0x7F</td> <td><em>unsupported</em></td> </tr> <tr> <td>Min Key</td> <td>0xFF</td> <td><em>unsupported</em></td> </tr> </tbody> </table> <div class="admonition warning"> <p class=admonition-title>Incomplete mapping</p> <p>The mapping is <strong>incomplete</strong>. The unsupported mappings are indicated in the table above.</p> </div> <div class="admonition note"> <p class=admonition-title>Handling of <abbr title="Binary JSON">BSON</abbr> type 0x11</p> <p>This library deserializes <abbr title="Binary JSON">BSON</abbr> type <code>0x11</code> (Timestamp) as a <code>number_unsigned</code> value. The 64-bit value is preserved, but the Timestamp type information is not.</p> </div> <div class="admonition warning"> <p class=admonition-title>Lenient <abbr title="Binary JSON">BSON</abbr> input handling</p> <p>The <abbr title="Binary JSON">BSON</abbr> reader is lenient in a few areas where the <abbr title="Binary JSON">BSON</abbr> specification is more restrictive:</p> <ul> <li>array element keys are not checked against the required decimal sequence (<code>0</code>, <code>1</code>, <code>2</code>, ...),</li> <li>any non-zero byte is accepted as <code>true</code> for the boolean type,</li> <li>the payloads of the binary subtypes <code>0x03</code> (UUID, old), <code>0x04</code> (UUID), and <code>0x05</code> (MD5) are not checked to have 16 bytes,</li> <li>duplicate keys within a document are accepted; the last value wins (<code>{"a": 1, "a": 2}</code> is read as <code>{"a": 2}</code>), while a <abbr title="Simple API for XML">SAX</abbr> handler receives every occurrence,</li> <li>a <code>string</code> value may contain embedded <code>0x00</code> bytes, and</li> <li>with <code>strict</code> set to <code>false</code>, bytes after the end of the document are ignored.</li> </ul> <p>If <abbr title="Binary JSON">BSON</abbr> input must be validated for strict specification compliance, validate it separately before passing it to <code>from_bson()</code>.</p> </div> <div class="admonition note"> <p class=admonition-title>Document size</p> <p>Every document starts with its size. <code>from_bson()</code> rejects a document with <a href=../../../home/exceptions/#jsonexceptionparse_error112><code>parse_error.112</code></a> if the size is less than 5 (the size prefix and the terminator), if an embedded document does not end before the end of the document that contains it, if the elements do not fit into the declared size, or if the size does not match the number of bytes read when the terminating <code>0x00</code> byte is reached. A size that is too small is detected before an element is read whose type byte and name no longer fit; an element that starts inside the declared size but extends past it is read, and reported to a <abbr title="Simple API for XML">SAX</abbr> handler, before the mismatch is found at the terminator. A size that is too large is likewise found only at the terminator. Passing <code>strict</code> as <code>false</code> skips only the check for bytes after the end of the document (<a href=../../../home/exceptions/#jsonexceptionparse_error110><code>parse_error.110</code></a>), not Line truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
<span class=p>{</span>
@@ -49,10 +49,10 @@
<span class=w> </span><span class=n>json</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>json</span><span class=o>::</span><span class=n>from_bson</span><span class=p>(</span><span class=n>v</span><span class=p>);</span>
<span class=w> </span><span class=c1>// print the deserialized JSON value</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"compact"</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=p>,</span>
<span class=w> </span><span class=nt>"schema"</span><span class=p>:</span><span class=w> </span><span class=mi>0</span>
<span class=p>}</span>
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 24, 2026 15:57:44 UTC">October 24, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 26, 2026 17:25:46 UTC">October 26, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
features/binary_formats/bson/index.md
+87 -70
View File
@@ -7,72 +7,9 @@ References
- [BSON Website](http://bsonspec.org) - the main source on BSON
- [BSON Specification](http://bsonspec.org/spec.html) - the specification
## Serialization
## Byte layout
The library uses the following mapping from JSON values types to BSON types:
| JSON value type | value/range | BSON type | marker |
| --------------- | ----------------------------------------- | --------- | ------ |
| null | `null` | null | 0x0A |
| boolean | `true`, `false` | boolean | 0x08 |
| number_integer | -9223372036854775808..-2147483649 | int64 | 0x12 |
| number_integer | -2147483648..2147483647 | int32 | 0x10 |
| number_integer | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 0..2147483647 | int32 | 0x10 |
| number_unsigned | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | 0x11 |
| number_float | *any value* | double | 0x01 |
| string | *any value* | string | 0x02 |
| array | *any value* | document | 0x04 |
| object | *any value* | document | 0x03 |
| binary | *any value* | binary | 0x05 |
Incomplete mapping
The mapping is **incomplete**, since only JSON-objects (and things contained therein) can be serialized to BSON. Also, keys may not contain U+0000, since they are serialized a zero-terminated c-strings.
BSON type 0x11 interoperability
The BSON specification defines type `0x11` as a Timestamp. This library uses marker `0x11` when serializing `number_unsigned` values in the range `9223372036854775808..18446744073709551615`. Other BSON implementations may therefore interpret these values as Timestamps instead of unsigned integers.
Binary values without a subtype
BSON requires every binary value to have a subtype. If a binary value has no subtype, this library serializes it with the generic subtype `0x00`. After deserialization, `has_subtype()` returns `true` and `subtype()` returns `0`. As a result, serializing and deserializing a JSON object containing such a value produces a different JSON object, even though the binary data is unchanged.
Example: serialize a JSON value to BSON
```
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using namespace nlohmann::literals;
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": 0})"_json;
// serialize it to BSON
std::vector<std::uint8_t> v = json::to_bson(j);
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
}
std::cout << std::endl;
}
```
Output:
```
0x1b 0x00 0x00 0x00 0x08 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0x00 0x01 0x10 0x73 0x63 0x68 0x65 0x6d 0x61 0x00 0x00 0x00 0x00 0x00 0x00
```
### Byte layout
A BSON document is a length-prefixed sequence of *elements*. The first four bytes hold the total size of the document (little-endian int32), followed by the elements, and a final `0x00` byte ends the document. Each element starts with a one-byte type, followed by its key as a NUL-terminated string and the value, whose encoding depends on the type. Arrays are documents whose keys are the decimal indices `"0"`, `"1"`, ...; nesting is expressed by embedding documents. Because every document announces its size, a reader can skip over it without parsing its content. The top level of a BSON value is always a document, which is why only JSON objects can be serialized.
Consider the following JSON value (shown formatted for readability):
@@ -102,6 +39,79 @@ Consider the following JSON value (shown formatted for readability):
| `]` | 00 | end of the array document |
| `}` | 00 | end of the document |
## Serialization
The library uses the following mapping from JSON values types to BSON types:
| JSON value type | value/range | BSON type | marker |
| --------------- | ----------------------------------------- | --------- | ------ |
| null | `null` | null | 0x0A |
| boolean | `true`, `false` | boolean | 0x08 |
| number_integer | -9223372036854775808..-2147483649 | int64 | 0x12 |
| number_integer | -2147483648..2147483647 | int32 | 0x10 |
| number_integer | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 0..2147483647 | int32 | 0x10 |
| number_unsigned | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | 0x11 |
| number_float | *any value* | double | 0x01 |
| string | *any value* | string | 0x02 |
| array | *any value* | document | 0x04 |
| object | *any value* | document | 0x03 |
| binary | *any value* | binary | 0x05 |
Incomplete mapping
The mapping is **incomplete**, since only JSON-objects (and things contained therein) can be serialized to BSON. Also, keys may not contain U+0000, since they are serialized as zero-terminated C strings.
Order of object members
The members of an object are written in the order in which iteration visits them: sorted by key for [`json`](https://json.nlohmann.me/api/json/index.md), in insertion order for [`ordered_json`](https://json.nlohmann.me/api/ordered_json/index.md).
BSON type 0x11 interoperability
The BSON specification defines type `0x11` as a Timestamp. This library uses marker `0x11` when serializing `number_unsigned` values in the range `9223372036854775808..18446744073709551615`. Other BSON implementations may therefore interpret these values as Timestamps instead of unsigned integers.
Binary values without a subtype
BSON requires every binary value to have a subtype. If a binary value has no subtype, this library serializes it with the generic subtype `0x00`. After deserialization, `has_subtype()` returns `true` and `subtype()` returns `0`. As a result, serializing and deserializing a JSON object containing such a value produces a different JSON object, even though the binary data is unchanged.
Binary subtype 0x02 (old binary)
The BSON specification defines the payload of the "binary (old)" subtype `0x02` as an int32 followed by that many bytes, so the value is preceded by two lengths: the length of the binary element and, as the first four bytes of the payload, the number of bytes that follow. A binary value with subtype `2` is therefore written with the outer length 4 bytes larger than its size and the size as inner length: `binary({1, 2, 3}, 2)` becomes `07 00 00 00 02 03 00 00 00 01 02 03`. When reading, the inner length is checked and removed, so the value contains only the bytes that follow it. `from_bson()` throws [`parse_error.112`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error112) if the outer length is less than 4 or the inner length is not the outer length minus 4. Versions before 3.13.0 did not write the inner length and returned it as part of the payload; BSON data written by those versions with subtype 2 is not readable any more and must be written again.
Example: serialize a JSON value to BSON
```
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using nlohmann::json;
using namespace nlohmann::literals;
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": 0})"_json;
// serialize it to BSON
std::vector<std::uint8_t> v = json::to_bson(j);
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << '\n';
}
```
Output:
```
0x1b 0x00 0x00 0x00 0x08 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0x00 0x01 0x10 0x73 0x63 0x68 0x65 0x6d 0x61 0x00 0x00 0x00 0x00 0x00 0x00
```
## Deserialization
The library maps BSON record types to JSON value types as follows:
@@ -143,14 +153,21 @@ Lenient BSON input handling
The BSON reader is lenient in a few areas where the BSON specification is more restrictive:
- array element keys are not checked against the required decimal sequence (`0`, `1`, `2`, ...),
- any non-zero byte is accepted as `true` for the boolean type, and
- the payload for binary subtype `0x02` is returned as-is, including its inner length prefix.
- any non-zero byte is accepted as `true` for the boolean type,
- the payloads of the binary subtypes `0x03` (UUID, old), `0x04` (UUID), and `0x05` (MD5) are not checked to have 16 bytes,
- duplicate keys within a document are accepted; the last value wins (`{"a": 1, "a": 2}` is read as `{"a": 2}`), while a SAX handler receives every occurrence,
- a `string` value may contain embedded `0x00` bytes, and
- with `strict` set to `false`, bytes after the end of the document are ignored.
If BSON input must be validated for strict specification compliance, validate it separately before passing it to `from_bson()`.
Document size
Every document starts with its size. `from_bson()` rejects a document with [`parse_error.112`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error112) if the size is less than 5 (the size prefix and the terminator), if an embedded document does not end before the end of the document that contains it, if the elements do not fit into the declared size, or if the size does not match the number of bytes read when the terminating `0x00` byte is reached. A size that is too small is detected before an element is read whose type byte and name no longer fit; an element that starts inside the declared size but extends past it is read, and reported to a SAX handler, before the mismatch is found at the terminator. A size that is too large is likewise found only at the terminator. Passing `strict` as `false` skips only the check for bytes after the end of the document ([`parse_error.110`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error110)), not the document size checks.
Ill-formed UTF-8 in string values
The BSON specification requires `string` values (type `0x02`) to be valid UTF-8, but this is not required of a decoder, so checking is opt-in: with the [`error_handler`](https://json.nlohmann.me/api/basic_json/from_bson/index.md) parameter left at `keep` (the default), `from_bson()` accepts a `string` value whose bytes are not valid UTF-8 and hands them back unchanged. Passing `error_handler_t::strict` makes `from_bson()` check and throw [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113) for ill-formed UTF-8, and `replace`/`ignore` sanitize the string instead of keeping it. However, [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md) still requires valid UTF-8 and throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for a value read with the default `keep` handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. `to_bson()`'s own `error_handler` parameter defaults to `keep`, so such a string value or element (key) name is written unchanged; with `strict` (the default if [`JSON_STRICT_BINARY_UTF8`](https://json.nlohmann.me/api/macros/json_strict_binary_utf8/index.md) is enabled), it throws the same exception instead. Element (key) names are never validated on read, since they are read byte-by-byte as a C string. `binary` values (type `0x05`) are unaffected, since they are not required to hold text.
The BSON specification requires `string` values (type `0x02`) to be valid UTF-8, but this is not required of a decoder, so checking is opt-in: with the [`error_handler`](https://json.nlohmann.me/api/basic_json/from_bson/index.md) parameter left at `keep` (the default), `from_bson()` accepts a `string` value whose bytes are not valid UTF-8 and hands them back unchanged. Passing `error_handler_t::strict` makes `from_bson()` check and throw [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113) for ill-formed UTF-8, and `replace`/`ignore` sanitize the string instead of keeping it. However, [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md) still requires valid UTF-8 and throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for a value read with the default `keep` handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. `to_bson()`'s own `error_handler` parameter defaults to `keep`, so such a string value or element (key) name is written unchanged; with `strict` (the default if [`JSON_STRICT_BINARY_UTF8`](https://json.nlohmann.me/api/macros/json_strict_binary_utf8/index.md) is enabled), it throws the same exception instead. Element (key) names are treated like string values: with `keep` they are read unchanged, with `strict`, `replace`, or `ignore` they are checked and handled in the same way. `binary` values (type `0x05`) are unaffected, since they are not required to hold text.
Example: deserialize a JSON value from BSON
@@ -159,7 +176,7 @@ Example: deserialize a JSON value from BSON
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -174,7 +191,7 @@ int main()
json j = json::from_bson(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/cbor.md
+80 -52
View File
@@ -9,6 +9,39 @@ extremely small code sizes, fairly small message size, and extensibility without
- [CBOR Playground](http://cbor.me) - an interactive webpage to translate between JSON and CBOR
- [RFC 8949](https://www.rfc-editor.org/rfc/rfc8949.html) - the CBOR specification
## Byte layout
CBOR encodes every data item as a *head* followed by optional content. The first byte of the head holds the *major type* in its upper three bits (unsigned integer, negative integer, byte string, text string, array, map, tag, or simple/float) and an *additional information* value in its lower five bits. Additional information below 24 is the argument itself (for example, a small integer or a short length); the values 24 to 27 announce that the argument follows in 1, 2, 4, or 8 big-endian bytes. Strings, arrays, and maps are prefixed with their length (or count of elements/pairs), and content follows directly. A reader of definite-length items knows how many items to expect without a terminator; indefinite-length items (additional information 31) are instead closed by the "break" byte 0xFF, which `from_cbor` reads but `to_cbor` never writes.
Consider the following JSON value (shown formatted for readability):
```json
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_cbor` encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-d">data</span>.
<div class="byte-layout" markdown="0"><table>
<thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody>
<tr><td><code>{</code></td><td><span class="bl bl-t">A4</span></td><td>map with 4 key/value pairs</td></tr>
<tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">62</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">19</span> <span class="bl bl-d">01 2C</span></td><td>key: text string of length 2, then its UTF-8 bytes; value: unsigned integer, 2 bytes follow (big-endian)</td></tr>
<tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">64</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">64</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: text string of length 4; value: text string of length 4</td></tr>
<tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">62</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">FB</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: text string of length 2; value: double-precision float (IEEE 754, big-endian)</td></tr>
<tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">64</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">82</span></td><td>key: text string of length 4; value: array with 2 elements</td></tr>
<tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">61</span> <span class="bl bl-d">61</span></td><td>text string of length 1</td></tr>
<tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">61</span> <span class="bl bl-d">62</span></td><td>text string of length 1</td></tr>
<tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr>
<tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr>
</tbody></table></div>
## Serialization
The library uses the following mapping from JSON values types to CBOR types according to the CBOR specification
@@ -19,16 +52,16 @@ The library uses the following mapping from JSON values types to CBOR types acco
| null | `null` | Null | 0xF6 |
| boolean | `true` | True | 0xF5 |
| boolean | `false` | False | 0xF4 |
| number_integer | -9223372036854775808..-2147483649 | Negative integer (8 bytes follow) | 0x3B |
| number_integer | -2147483648..-32769 | Negative integer (4 bytes follow) | 0x3A |
| number_integer | -32768..-129 | Negative integer (2 bytes follow) | 0x39 |
| number_integer | -128..-25 | Negative integer (1 byte follow) | 0x38 |
| number_integer | -9223372036854775808..-4294967297 | Negative integer (8 bytes follow) | 0x3B |
| number_integer | -4294967296..-65537 | Negative integer (4 bytes follow) | 0x3A |
| number_integer | -65536..-257 | Negative integer (2 bytes follow) | 0x39 |
| number_integer | -256..-25 | Negative integer (1 byte follow) | 0x38 |
| number_integer | -24..-1 | Negative integer | 0x20..0x37 |
| number_integer | 0..23 | Integer | 0x00..0x17 |
| number_integer | 24..255 | Unsigned integer (1 byte follow) | 0x18 |
| number_integer | 256..65535 | Unsigned integer (2 bytes follow) | 0x19 |
| number_integer | 65536..4294967295 | Unsigned integer (4 bytes follow) | 0x1A |
| number_integer | 4294967296..18446744073709551615 | Unsigned integer (8 bytes follow) | 0x1B |
| number_integer | 4294967296..9223372036854775807 | Unsigned integer (8 bytes follow) | 0x1B |
| number_unsigned | 0..23 | Integer | 0x00..0x17 |
| number_unsigned | 24..255 | Unsigned integer (1 byte follow) | 0x18 |
| number_unsigned | 256..65535 | Unsigned integer (2 bytes follow) | 0x19 |
@@ -62,7 +95,24 @@ see "binary" cells in the table above.
!!! success "Complete mapping"
The mapping is **complete** in the sense that any JSON value type can be converted to a CBOR value.
The mapping is **complete** in the sense that any JSON value type can be converted to a CBOR value, with one
exception: an integer outside the range of CBOR integers (only possible with a number type wider than 64 bits)
throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407).
!!! info "Not a deterministic encoding"
`to_cbor` does not produce the deterministic encoding of
[RFC 8949, Section 4.2.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-4.2.1), and not always the preferred
serialization of Section 4.1:
- floating-point numbers are written as single precision (0xFA) when that is exact and as double precision (0xFB)
otherwise; half precision (0xF9) is used only for `NaN` and `Infinity`/`-Infinity` (`1.0` is `fa 3f 80 00 00`);
- a binary subtype 0..23 is written as `d8 xx` instead of the one-byte tag 0xC0..0xD7 (`d8 01 41 01`);
- object members are written in the order of `object_t` (for the default `std::map`: sorted by the key string, not
by the encoded key bytes).
Integers and lengths use the shortest form, and indefinite-length items are never written. Decoders must accept all
of this; the RFC does not require the preferred serialization.
!!! info "NaN/infinity handling"
@@ -72,9 +122,8 @@ see "binary" cells in the table above.
!!! note
Prior to version 3.13.0, NaN and Infinity were instead serialized as a CBOR double-precision float (type 0xFB,
9 bytes total), because the check used to select a smaller encoding compared magnitudes with NaN, which is
always `false` and caused the intended half-precision path to be skipped.
Prior to version 3.8.0, NaN and Infinity were instead serialized as a CBOR double-precision float (type 0xFB,
9 bytes total).
!!! info "Unused CBOR types"
@@ -91,7 +140,6 @@ see "binary" cells in the table above.
- expected conversions (0xD5..0xD7)
- simple values (0xE0..0xF3, 0xF8)
- undefined (0xF7)
- half-precision floats (0xF9)
- break (0xFF)
!!! info "Tagged items"
@@ -105,7 +153,7 @@ see "binary" cells in the table above.
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
using namespace nlohmann::literals;
int main()
@@ -119,9 +167,9 @@ see "binary" cells in the table above.
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << std::endl;
std::cout << '\n';
}
```
@@ -133,41 +181,6 @@ see "binary" cells in the table above.
```
### Byte layout
Consider the following JSON value (shown formatted for readability):
```json
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_cbor` encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.
<div class="byte-layout" markdown="0"><table>
<thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody>
<tr><td><code>{</code></td><td><span class="bl bl-t">A4</span></td><td>map with 4 key/value pairs</td></tr>
<tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">62</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">19</span> <span class="bl bl-d">01 2C</span></td><td>key: text string of length 2, then its UTF-8 bytes; value: unsigned integer, 2 bytes follow (big-endian)</td></tr>
<tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">64</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">64</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: text string of length 4; value: text string of length 4</td></tr>
<tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">62</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">FB</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: text string of length 2; value: double-precision float (IEEE 754, big-endian)</td></tr>
<tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">64</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">82</span></td><td>key: text string of length 4; value: array with 2 elements</td></tr>
<tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">61</span> <span class="bl bl-d">61</span></td><td>text string of length 1</td></tr>
<tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">61</span> <span class="bl bl-d">62</span></td><td>text string of length 1</td></tr>
<tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr>
<tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr>
</tbody></table></div>
## Deserialization
The library maps CBOR types to JSON value types as follows:
@@ -251,10 +264,20 @@ string of the same major type, as required by [RFC 8949, Section 3.2.3](https://
([RFC 8392](https://www.rfc-editor.org/rfc/rfc8392.html)), cannot be read with this library and need a
general-purpose CBOR library instead.
A tag in front of a text string key is handled like a tag in front of a value: it is skipped with
`cbor_tag_handler_t::ignore` or `cbor_tag_handler_t::store` and rejected with the default `error`. A tag does not
turn a key of another type into a text string.
A map with duplicate keys is well-formed, but invalid under
[RFC 8949, Section 5.3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-5.3.1). `from_cbor` accepts it and the
last value wins; the [SAX interface](../parsing/sax_interface.md) reports every occurrence.
!!! warning "Ill-formed UTF-8 in text strings"
[RFC 8949, Section 3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-3.1) requires CBOR text strings (major
type 3) to be valid UTF-8, but leaves it up to the decoder whether to enforce this, so checking is opt-in: with the
type 3) to be valid UTF-8 (a string with ill-formed UTF-8 is "well-formed but invalid"), and
[Section 5.3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-5.3.1) leaves it up to the decoder whether to
verify this, so checking is opt-in: with the
[`error_handler`](../../api/basic_json/from_cbor.md) parameter left at `keep` (the default), `from_cbor()` accepts a
text string (object keys included) whose bytes are not valid UTF-8 and hands them back unchanged. Passing
`error_handler_t::strict` makes `from_cbor()` check and throw
@@ -267,9 +290,14 @@ string of the same major type, as required by [RFC 8949, Section 3.2.3](https://
[`JSON_STRICT_BINARY_UTF8`](../../api/macros/json_strict_binary_utf8.md) is enabled), it throws the same exception
instead. Byte strings (major type 2) are unaffected, since they are not required to hold text.
The chunks of an indefinite-length text string are checked one by one, as required by
[RFC 8949, Section 3.2.3](https://www.rfc-editor.org/rfc/rfc8949.html#section-3.2.3): a code point cannot be spread
over two chunks. With `error_handler_t::strict` such a string throws `parse_error.113`; `replace` and `ignore`
sanitize each chunk on its own.
!!! warning "Tagged items"
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing `cbor_tag_handler_t::store` to function `from_cbor` stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the innermost one is stored. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing `cbor_tag_handler_t::store` to function `from_cbor` stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the tag directly in front of it is stored, and only when it has the form 0xD8..0xDB (a tag 0..23 written as 0xC0..0xD7 is skipped); for example, `D8 05 C1 41 AA` yields subtype `null`, whereas `C1 D8 05 41 AA` yields 5. Tags in front of a text string map key are skipped as well. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
??? example "Example: deserialize a JSON value from CBOR"
@@ -278,7 +306,7 @@ string of the same major type, as required by [RFC 8949, Section 3.2.3](https://
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -292,7 +320,7 @@ string of the same major type, as required by [RFC 8949, Section 3.2.3](https://
json j = json::from_cbor(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/cbor/index.html
+18 -18
View File
@@ -1,8 +1,17 @@
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/cbor/" rel="canonical"><link href=../bson/ rel=prev><link href=../messagepack/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>CBOR - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#cbor class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> CBOR </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div class=md-sLine truncated
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/cbor/" rel="canonical"><link href=../bson/ rel=prev><link href=../messagepack/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>CBOR - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#cbor class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> CBOR </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div class=md-sLine truncated
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_cbor</code> encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">A4</span></td><td>map with 4 key/value pairs</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">62</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">19</span> <span class="bl bl-d">01 2C</span></td><td>key: text string of length 2, then its UTF-8 bytes; value: unsigned integer, 2 bytes follow (big-endian)</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">64</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">64</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: text string of length 4; value: text string of length 4</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">62</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">FB</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: text string of length 2; value: double-precision float (IEEE 754, big-endian)</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">64</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">82</span></td><td>key: text string of length 4; value: array with 2 elements</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">61</span> <span class="bl bl-d">61</span></td><td>text string of length 1</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">61</span> <span class="bl bl-d">62</span></td><td>text string of length 1</td></tr> <tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr> <tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr> </tbody></table></div> <h2 id=serialization>Serialization<a class=headerlink href=#serialization title="Permanent link">&para;</a></h2> <p>The library uses the following mapping from <abbr title="JavaScript Object Notation">JSON</abbr> values types to <abbr title="Concise Binary Object Representation">CBOR</abbr> types according to the <abbr title="Concise Binary Object Representation">CBOR</abbr> specification (<a href="https://www.rfc-editor.org/rfc/rfc8949.html"><abbr title="Request for Comments">RFC</abbr> 8949</a>):</p> <table> <thead> <tr> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>value/range</th> <th><abbr title="Concise Binary Object Representation">CBOR</abbr> type</th> <th>first byte</th> </tr> </thead> <tbody> <tr> <td>null</td> <td><code>null</code></td> <td>Null</td> <td>0xF6</td> </tr> <tr> <td>boolean</td> <td><code>true</code></td> <td>True</td> <td>0xF5</td> </tr> <tr> <td>boolean</td> <td><code>false</code></td> <td>False</td> <td>0xF4</td> </tr> <tr> <td>number_integer</td> <td>-9223372036854775808..-4294967297</td> <td>Negative integer (8 bytes follow)</td> <td>0x3B</td> </tr> <tr> <td>number_integer</td> <td>-4294967296..-65537</td> <td>Negative integer (4 bytes follow)</td> <td>0x3A</td> </tr> <tr> <td>number_integer</td> <td>-65536..-257</td> <td>Negative integer (2 bytes follow)</td> <td>0x39</td> </tr> <tr> <td>number_integer</td> <td>-256..-25</td> <td>Negative integer (1 byte follow)</td> <td>0x38</td> </tr> <tr> <td>number_integer</td> <td>-24..-1</td> <td>Negative integer</td> <td>0x20..0x37</td> </tr> <tr> <td>number_integer</td> <td>0..23</td> <td>Integer</td> <td>0x00..0x17</td> </tr> <tr> <td>number_integer</td> <td>24..255</td> <td>Unsigned integer (1 byte follow)</td> <td>0x18</td> </tr> <tr> <td>number_integer</td> <td>256..65535</td> <td>Unsigned integer (2 bytes follow)</td> <td>0x19</td> </tr> <tr> <td>number_integer</td> <td>65536..4294967295</td> <td>Unsigned integer (4 bytes follow)</td> <td>0x1A</td> </tr> <tr> <td>number_integer</td> <td>4294967296..9223372036854775807</td> <td>Unsigned integer (8 bytes follow)</td> <td>0x1B</td> </tr> <tr> <td>number_unsigned</td> <td>0..23</td> <td>Integer</td> <td>0x00..0x17</td> </tr> <tr> <td>number_unsigned</td> <td>24..255</td> <td>Unsigned integer (1 byte follow)</td> <td>0x18</td> </tr> <tr> <td>number_unsigned</td> <td>256..65535</td> <td>Unsigned integer (2 bytes follow)</td> <td>0x19</td> </tr> <tr> <td>number_unsigned</td> <td>65536..4294967295</td> <td>Unsigned integer (4 bytes follow)</td> <td>0x1A</td> </tr> <tr> <td>number_unsigned</td> <td>4294967296..18446744073709551615</td> <td>Unsigned integer (8 bytes follow)</td> <td>0x1B</td> </tr> <tr> <td>number_float</td> <td><em>any value representable by a float</em></td> <td>Single-Precision Float</td> <td>0xFA</td> </tr> <tr> <td>number_float</td> <td><em>any value Line truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=k>namespace</span><span class=w> </span><span class=nn>nlohmann</span><span class=o>::</span><span class=nn>literals</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
@@ -16,26 +25,17 @@
<span class=w> </span><span class=c1>// print the vector content</span>
<span class=w> </span><span class=k>for</span><span class=w> </span><span class=p>(</span><span class=k>auto</span><span class=o>&amp;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>:</span><span class=w> </span><span class=n>v</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>"0x"</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>hex</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setfill</span><span class=p>(</span><span class=sc>'0'</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=p>(</span><span class=kt>int</span><span class=p>)</span><span class=kt>byte</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>" "</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>"0x"</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>hex</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setfill</span><span class=p>(</span><span class=sc>'0'</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=k>static_cast</span><span class=o>&lt;</span><span class=kt>int</span><span class=o>&gt;</span><span class=p>(</span><span class=kt>byte</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>" "</span><span class=p>;</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=mh>0xa2</span><span class=w> </span><span class=mh>0x67</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x6f</span><span class=w> </span><span class=mh>0x6d</span><span class=w> </span><span class=mh>0x70</span><span class=w> </span><span class=mh>0x61</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x74</span><span class=w> </span><span class=mh>0xf5</span><span class=w> </span><span class=mh>0x66</span><span class=w> </span><span class=mh>0x73</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x68</span><span class=w> </span><span class=mh>0x65</span><span class=w> </span><span class=mh>0x6d</span><span class=w> </span><span class=mh>0x61</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span>
</code></pre></div> </details> <h3 id=byte-layout>Byte layout<a class=headerlink href=#byte-layout title="Permanent link">&para;</a></h3> <p>Consider the following <abbr title="JavaScript Object Notation">JSON</abbr> value (shown formatted for readability):</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_cbor</code> encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">A4</span></td><td>map with 4 key/value pairs</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">62</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">19</span> <span class="bl bl-d">01 2C</span></td><td>key: text string of length 2, then its UTF-8 bytes; value: unsigned integer, 2 bytes follow (big-endian)</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">64</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">64</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: text string of length 4; value: text string of length 4</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">62</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">FB</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: text string of length 2; value: double-precision float (IEEE 754, big-endian)</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">64</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">82</span></td><td>key: text string of length 4; value: array with 2 elements</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">61</span> <span class="bl bl-d">61</span></td><td>text string of length 1</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">61</span> <span class="bl bl-d">62</span></td><td>text string of length 1</td></tr> <tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr> <tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr> </tbody></table></div> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Concise Binary Object Representation">CBOR</abbr> types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Concise Binary Object Representation">CBOR</abbr> type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>first byte</th> </tr> </thead> <tbody> <tr> <td>Integer</td> <td>number_unsigned</td> <td>0x00..0x17</td> </tr> <tr> <td>Unsigned integer</td> <td>number_unsigned</td> <td>0x18</td> </tr> <tr> <td>Unsigned integer</td> <td>number_unsigned</td> <td>0x19</td> </tr> <tr> <td>Unsigned integer</td> <td>number_unsigned</td> <td>0x1A</td> </tr> <tr> <td>Unsigned integer</td> <td>number_unsigned</td> <td>0x1B</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x20..0x37</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x38</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x39</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x3A</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x3B</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x40..0x57</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x58</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x59</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x5A</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x5B</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x5F</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x60..0x77</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x78</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x79</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x7A</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x7B</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x7F</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x80..0x97</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x98</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x99</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x9A</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x9B</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x9F</td> </tr> <tr> <td>map</td> <td>object</td> <td>0xA0..0xB7</td> </tr> <tr> <td>map</td> <td>object</td> <td>0xB8</td> </tr> <tr> <td>map</td> <td>object</td> <td>0xBLine truncated
</code></pre></div> <p>This applies to the <a href=../../parsing/sax_interface/ ><abbr title="Simple API for XML">SAX</abbr> interface</a> as well, as the key is read before it is passed on. This is a deliberate restriction of the library's <abbr title="JavaScript Object Notation">JSON</abbr> value model, not an oversight: formats built on <abbr title="Concise Binary Object Representation">CBOR</abbr> maps with integer keys, such as COSE (<a href="https://www.rfc-editor.org/rfc/rfc9052.html"><abbr title="Request for Comments">RFC</abbr> 9052</a>) or CWT (<a href="https://www.rfc-editor.org/rfc/rfc8392.html"><abbr title="Request for Comments">RFC</abbr> 8392</a>), cannot be read with this library and need a general-purpose <abbr title="Concise Binary Object Representation">CBOR</abbr> library instead.</p> </div> <div class="admonition warning"> <p class=admonition-title>Ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8 in text strings</p> <p><a href="https://www.rfc-editor.org/rfc/rfc8949.html#section-3.1"><abbr title="Request for Comments">RFC</abbr> 8949, Section 3.1</a> requires <abbr title="Concise Binary Object Representation">CBOR</abbr> text strings (major type 3) to be valid <abbr title="Unicode Transformation Format">UTF</abbr>-8, but leaves it up to the decoder whether to enforce this, so checking is opt-in: with the <a href=../../../api/basic_json/from_cbor/ ><code>error_handler</code></a> parameter left at <code>keep</code> (the default), <code>from_cbor()</code> accepts a text string (object keys included) whose bytes are not valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 and hands them back unchanged. Passing <code>error_handler_t::strict</code> makes <code>from_cbor()</code> check and throw <a href=../../../home/exceptions/#jsonexceptionparse_error113><code>parse_error.113</code></a> for ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8, and <code>replace</code>/<code>ignore</code> sanitize the string instead of keeping it. However, <a href=../../../api/basic_json/dump/ ><code>dump()</code></a> still requires valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 and throws <a href=../../../home/exceptions/#jsonexceptiontype_error316><code>type_error.316</code></a> for a value read with the default <code>keep</code> handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. <code>to_cbor()</code>'s own <a href=../../../api/basic_json/to_cbor/ ><code>error_handler</code></a> parameter defaults to <code>keep</code>, so such a value is written back unchanged; with <code>strict</code> (the default if <a href=../../../api/macros/json_strict_binary_utf8/ ><code>JSON_STRICT_BINARY_UTF8</code></a> is enabled), it throws the same exception instead. Byte strings (major type 2) are unaffected, since they are not required to hold text.</p> </div> <div class="admonition warning"> <p class=admonition-title>Tagged items</p> <p>Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing <code>cbor_tag_handler_t::ignore</code> to function <code>from_cbor</code>, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing <code>cbor_tag_handler_t::store</code> to function <code>from_cbor</code> stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the innermost one is stored. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.</p> </div> <details class=example> <summary>Example: deserialize a <abbr title="JavaScript Object Notation">JSON</abbr> value from <abbr title="Concise Binary Object Representation">CBOR</abbr></summary> <div class=highlight><pre><span></span><code><span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iostream&gt;</span>
</code></pre></div> </details> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Concise Binary Object Representation">CBOR</abbr> types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Concise Binary Object Representation">CBOR</abbr> type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>first byte</th> </tr> </thead> <tbody> <tr> <td>Integer</td> <td>number_unsigned</td> <td>0x00..0x17</td> </tr> <tr> <td>Unsigned integer</td> <td>number_unsigned</td> <td>0x18</td> </tr> <tr> <td>Unsigned integer</td> <td>number_unsigned</td> <td>0x19</td> </tr> <tr> <td>Unsigned integer</td> <td>number_unsigned</td> <td>0x1A</td> </tr> <tr> <td>Unsigned integer</td> <td>number_unsigned</td> <td>0x1B</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x20..0x37</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x38</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x39</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x3A</td> </tr> <tr> <td>Negative integer</td> <td>number_integer</td> <td>0x3B</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x40..0x57</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x58</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x59</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x5A</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x5B</td> </tr> <tr> <td>Byte string</td> <td>binary</td> <td>0x5F</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x60..0x77</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x78</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x79</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x7A</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x7B</td> </tr> <tr> <td><abbr title="Unicode Transformation Format">UTF</abbr>-8 string</td> <td>string</td> <td>0x7F</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x80..0x97</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x98</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x99</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x9A</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x9B</td> </tr> <tr> <td>array</td> <td>array</td> <td>0x9F</td> </tr> <tr> <td>map</td> <td>object</td> <td>0xA0..0xB7</td> </tr> <tr> <td>map</td> <td>object</td> <td>0xB8</td> </tr> <tr> <td>map</td> <td>object</td> <td>0xB9</td> </tr> <tr> <td>map</td> <td>object</td> <td>0xBA</td> </tr> <tr> <td>map</td> <td>object</td> <td>0xBB</td> </tr> <tr> <td>map</td> <td>object</td> <td>0xBF</td> </tr> <tr> <td>False</td> <td><code>false</code></td> <td>0xF4</td> </tr> <tr> <td>True</td> <td><code>true</code></td> <td>0xF5</td> </tr> <tr> <td>Null</td> <td><code>null</code></td> <td>0xF6</td> </tr> <tr> <td>Half-Precision Float</td> <td>number_float</td> <td>0xF9</td> </tr> <tr> <td>Single-Precision Float</td> <td>number_float</td> <td>0xFA</td> </tr> <tr> <td>Double-Precision Float</td> <td>number_float</td> <td>0xFB</td> </tr> </tbody> </table> <p>Indefinite-length <abbr title="Unicode Transformation Format">UTF</abbr>-8 strings (0x7F) and byte strings (0x5F) are supported. Each chunk must be a definite-length string of the same major type, as required by <a href="https://www.rfc-editor.org/rfc/rfc8949.html#section-3.2.3"><abbr title="Request for Comments">RFC</abbr> 8949, Section 3.2.3</a>.</p> <div class="admonition warning"> <p class=admonition-title>Incomplete mapping</p> <p>The mapping is <strong>incomplete</strong> in the sense that not all <abbr title="Concise Binary Object Representation">CBOR</abbr> types can be converted to a <abbr title="JavaScript Object Notation">JSON</abbr> value. The following <abbr title="Concise Binary Object Representation">CBOR</abbr> types are not supported and will yield parse errors:</p> <ul> <li>simple values (0xE0..0xF3, 0xF8)</li> <li>undefined (0xF7)</li> </ul> <p>Tagged items (0xC0..0xDB) are not interpreted either; see the note on tagged items below.</p> </div> <div class="admonition warning"> <p class=admonition-title>Negative integer overflow</p> <p><abbr title="Concise Binary Object Representation">CBOR</abbr> negative integers (major type 1) are decoded as <code>-1 - n</code>. If the encoded magnitude <code>n</code> is too large for the result to fit into <code>number_integer_t</code> (<code>std::int64_t</code> by default), the result is stored as <code>number_float_t</code>, like a too small integer in <abbr title="JavaScript Object Notation">JSON</abbr> text. For example, <code>-184Line truncated
</code></pre></div> <p>This applies to the <a href=../../parsing/sax_interface/ ><abbr title="Simple API for XML">SAX</abbr> interface</a> as well, as the key is read before it is passed on. This is a deliberate restriction of the library's <abbr title="JavaScript Object Notation">JSON</abbr> value model, not an oversight: formats built on <abbr title="Concise Binary Object Representation">CBOR</abbr> maps with integer keys, such as COSE (<a href="https://www.rfc-editor.org/rfc/rfc9052.html"><abbr title="Request for Comments">RFC</abbr> 9052</a>) or CWT (<a href="https://www.rfc-editor.org/rfc/rfc8392.html"><abbr title="Request for Comments">RFC</abbr> 8392</a>), cannot be read with this library and need a general-purpose <abbr title="Concise Binary Object Representation">CBOR</abbr> library instead.</p> <p>A tag in front of a text string key is handled like a tag in front of a value: it is skipped with <code>cbor_tag_handler_t::ignore</code> or <code>cbor_tag_handler_t::store</code> and rejected with the default <code>error</code>. A tag does not turn a key of another type into a text string.</p> <p>A map with duplicate keys is well-formed, but invalid under <a href="https://www.rfc-editor.org/rfc/rfc8949.html#section-5.3.1"><abbr title="Request for Comments">RFC</abbr> 8949, Section 5.3.1</a>. <code>from_cbor</code> accepts it and the last value wins; the <a href=../../parsing/sax_interface/ ><abbr title="Simple API for XML">SAX</abbr> interface</a> reports every occurrence.</p> </div> <div class="admonition warning"> <p class=admonition-title>Ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8 in text strings</p> <p><a href="https://www.rfc-editor.org/rfc/rfc8949.html#section-3.1"><abbr title="Request for Comments">RFC</abbr> 8949, Section 3.1</a> requires <abbr title="Concise Binary Object Representation">CBOR</abbr> text strings (major type 3) to be valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 (a string with ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8 is "well-formed but invalid"), and <a href="https://www.rfc-editor.org/rfc/rfc8949.html#section-5.3.1">Section 5.3.1</a> leaves it up to the decoder whether to verify this, so checking is opt-in: with the <a href=../../../api/basic_json/from_cbor/ ><code>error_handler</code></a> parameter left at <code>keep</code> (the default), <code>from_cbor()</code> accepts a text string (object keys included) whose bytes are not valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 and hands them back unchanged. Passing <code>error_handler_t::strict</code> makes <code>from_cbor()</code> check and throw <a href=../../../home/exceptions/#jsonexceptionparse_error113><code>parse_error.113</code></a> for ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8, and <code>replace</code>/<code>ignore</code> sanitize the string instead of keeping it. However, <a href=../../../api/basic_json/dump/ ><code>dump()</code></a> still requires valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 and throws <a href=../../../home/exceptions/#jsonexceptiontype_error316><code>type_error.316</code></a> for a value read with the default <code>keep</code> handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. <code>to_cbor()</code>'s own <a href=../../../api/basic_json/to_cbor/ ><code>error_handler</code></a> parameter defaults to <code>keep</code>, so such a value is written back unchanged; with <code>strict</code> (the default if <a href=../../../api/macros/json_strict_binary_utf8/ ><code>JSON_STRICT_BINARY_UTF8</code></a> is enabled), it throws the same exception instead. Byte strings (major type 2) are unaffected, since they are not required to hold text.</p> <p>The chunks of an indefinite-length text string are checked one by one, as required by <a href="https://www.rfc-editor.org/rfc/rfc8949.html#section-3.2.3"><abbr title="Request for Comments">RFC</abbr> 8949, Section 3.2.3</a>: a code point cannot be spread over two chunks. With <code>error_handler_t::strict</code> such a string throws <code>parse_error.113</code>; <code>replace</code> and <code>ignore</code> sanitize each chunk on its own.</p> </div> <div class="admonition warning"> <p class=admonition-title>Tagged items</p> <p>Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing <code>cbor_tag_handler_t::ignore</code> to function <code>from_cbor</code>, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing <code>cbor_tag_handler_t::store</code> to function <code>from_cbor</code> stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the tag directly in front of it is stored, and only when it has the form 0xD8..0xDB (a tag 0..23 written as 0xC0..0xD7 is skipped); for example, <code>D8 05 C1 41 AA<Line truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
<span class=p>{</span>
@@ -49,10 +49,10 @@
<span class=w> </span><span class=n>json</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>json</span><span class=o>::</span><span class=n>from_cbor</span><span class=p>(</span><span class=n>v</span><span class=p>);</span>
<span class=w> </span><span class=c1>// print the deserialized JSON value</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"compact"</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=p>,</span>
<span class=w> </span><span class=nt>"schema"</span><span class=p>:</span><span class=w> </span><span class=mi>0</span>
<span class=p>}</span>
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 24, 2026 15:57:44 UTC">October 24, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 26, 2026 17:25:44 UTC">October 26, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
features/binary_formats/cbor/index.md
+62 -45
View File
@@ -8,6 +8,38 @@ References
- [CBOR Playground](http://cbor.me) - an interactive webpage to translate between JSON and CBOR
- [RFC 8949](https://www.rfc-editor.org/rfc/rfc8949.html) - the CBOR specification
## Byte layout
CBOR encodes every data item as a *head* followed by optional content. The first byte of the head holds the *major type* in its upper three bits (unsigned integer, negative integer, byte string, text string, array, map, tag, or simple/float) and an *additional information* value in its lower five bits. Additional information below 24 is the argument itself (for example, a small integer or a short length); the values 24 to 27 announce that the argument follows in 1, 2, 4, or 8 big-endian bytes. Strings, arrays, and maps are prefixed with their length (or count of elements/pairs), and content follows directly. A reader of definite-length items knows how many items to expect without a terminator; indefinite-length items (additional information 31) are instead closed by the "break" byte 0xFF, which `from_cbor` reads but `to_cbor` never writes.
Consider the following JSON value (shown formatted for readability):
```
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_cbor` encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: type/marker data.
| JSON | Bytes | Meaning |
| ----------------- | ----------------------------------- | -------------------------------------------------------------------------------------------------------- |
| `{` | A4 | map with 4 key/value pairs |
| `"id": 300,` | 62 69 64 19 01 2C | key: text string of length 2, then its UTF-8 bytes; value: unsigned integer, 2 bytes follow (big-endian) |
| `"name": "json",` | 64 6E 61 6D 65 64 6A 73 6F 6E | key: text string of length 4; value: text string of length 4 |
| `"pi": 3.14,` | 62 70 69 FB 40 09 1E B8 51 EB 85 1F | key: text string of length 2; value: double-precision float (IEEE 754, big-endian) |
| `"tags": [` | 64 74 61 67 73 82 | key: text string of length 4; value: array with 2 elements |
| `"a",` | 61 61 | text string of length 1 |
| `"b"` | 61 62 | text string of length 1 |
| `]` | | no end marker: the element count was given up front |
| `}` | | no end marker: the member count was given up front |
## Serialization
The library uses the following mapping from JSON values types to CBOR types according to the CBOR specification ([RFC 8949](https://www.rfc-editor.org/rfc/rfc8949.html)):
@@ -17,16 +49,16 @@ The library uses the following mapping from JSON values types to CBOR types acco
| null | `null` | Null | 0xF6 |
| boolean | `true` | True | 0xF5 |
| boolean | `false` | False | 0xF4 |
| number_integer | -9223372036854775808..-2147483649 | Negative integer (8 bytes follow) | 0x3B |
| number_integer | -2147483648..-32769 | Negative integer (4 bytes follow) | 0x3A |
| number_integer | -32768..-129 | Negative integer (2 bytes follow) | 0x39 |
| number_integer | -128..-25 | Negative integer (1 byte follow) | 0x38 |
| number_integer | -9223372036854775808..-4294967297 | Negative integer (8 bytes follow) | 0x3B |
| number_integer | -4294967296..-65537 | Negative integer (4 bytes follow) | 0x3A |
| number_integer | -65536..-257 | Negative integer (2 bytes follow) | 0x39 |
| number_integer | -256..-25 | Negative integer (1 byte follow) | 0x38 |
| number_integer | -24..-1 | Negative integer | 0x20..0x37 |
| number_integer | 0..23 | Integer | 0x00..0x17 |
| number_integer | 24..255 | Unsigned integer (1 byte follow) | 0x18 |
| number_integer | 256..65535 | Unsigned integer (2 bytes follow) | 0x19 |
| number_integer | 65536..4294967295 | Unsigned integer (4 bytes follow) | 0x1A |
| number_integer | 4294967296..18446744073709551615 | Unsigned integer (8 bytes follow) | 0x1B |
| number_integer | 4294967296..9223372036854775807 | Unsigned integer (8 bytes follow) | 0x1B |
| number_unsigned | 0..23 | Integer | 0x00..0x17 |
| number_unsigned | 24..255 | Unsigned integer (1 byte follow) | 0x18 |
| number_unsigned | 256..65535 | Unsigned integer (2 bytes follow) | 0x19 |
@@ -59,7 +91,17 @@ Binary values with subtype are mapped to tagged values (0xD8..0xDB) depending on
Complete mapping
The mapping is **complete** in the sense that any JSON value type can be converted to a CBOR value.
The mapping is **complete** in the sense that any JSON value type can be converted to a CBOR value, with one exception: an integer outside the range of CBOR integers (only possible with a number type wider than 64 bits) throws [`out_of_range.407`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range407).
Not a deterministic encoding
`to_cbor` does not produce the deterministic encoding of [RFC 8949, Section 4.2.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-4.2.1), and not always the preferred serialization of Section 4.1:
- floating-point numbers are written as single precision (0xFA) when that is exact and as double precision (0xFB) otherwise; half precision (0xF9) is used only for `NaN` and `Infinity`/`-Infinity` (`1.0` is `fa 3f 80 00 00`);
- a binary subtype 0..23 is written as `d8 xx` instead of the one-byte tag 0xC0..0xD7 (`d8 01 41 01`);
- object members are written in the order of `object_t` (for the default `std::map`: sorted by the key string, not by the encoded key bytes).
Integers and lengths use the shortest form, and indefinite-length items are never written. Decoders must accept all of this; the RFC does not require the preferred serialization.
NaN/infinity handling
@@ -67,7 +109,7 @@ NaN/infinity handling
Note
Prior to version 3.13.0 unreleased, NaN and Infinity were instead serialized as a CBOR double-precision float (type 0xFB, 9 bytes total), because the check used to select a smaller encoding compared magnitudes with NaN, which is always `false` and caused the intended half-precision path to be skipped.
Prior to version 3.8.0, NaN and Infinity were instead serialized as a CBOR double-precision float (type 0xFB, 9 bytes total).
Unused CBOR types
@@ -84,7 +126,6 @@ The following CBOR types are not used in the conversion:
- expected conversions (0xD5..0xD7)
- simple values (0xE0..0xF3, 0xF8)
- undefined (0xF7)
- half-precision floats (0xF9)
- break (0xFF)
Tagged items
@@ -98,7 +139,7 @@ Example: serialize a JSON value to CBOR
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
using namespace nlohmann::literals;
int main()
@@ -112,9 +153,9 @@ int main()
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << std::endl;
std::cout << '\n';
}
```
@@ -124,36 +165,6 @@ Output:
0xa2 0x67 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0xf5 0x66 0x73 0x63 0x68 0x65 0x6d 0x61 0x00
```
### Byte layout
Consider the following JSON value (shown formatted for readability):
```
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_cbor` encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: type/marker length/count/size data.
| JSON | Bytes | Meaning |
| ----------------- | ----------------------------------- | -------------------------------------------------------------------------------------------------------- |
| `{` | A4 | map with 4 key/value pairs |
| `"id": 300,` | 62 69 64 19 01 2C | key: text string of length 2, then its UTF-8 bytes; value: unsigned integer, 2 bytes follow (big-endian) |
| `"name": "json",` | 64 6E 61 6D 65 64 6A 73 6F 6E | key: text string of length 4; value: text string of length 4 |
| `"pi": 3.14,` | 62 70 69 FB 40 09 1E B8 51 EB 85 1F | key: text string of length 2; value: double-precision float (IEEE 754, big-endian) |
| `"tags": [` | 64 74 61 67 73 82 | key: text string of length 4; value: array with 2 elements |
| `"a",` | 61 61 | text string of length 1 |
| `"b"` | 61 62 | text string of length 1 |
| `]` | | no end marker: the element count was given up front |
| `}` | | no end marker: the member count was given up front |
## Deserialization
The library maps CBOR types to JSON value types as follows:
@@ -226,13 +237,19 @@ CBOR allows map keys of any type, whereas JSON only allows strings as keys in ob
This applies to the [SAX interface](https://json.nlohmann.me/features/parsing/sax_interface/index.md) as well, as the key is read before it is passed on. This is a deliberate restriction of the library's JSON value model, not an oversight: formats built on CBOR maps with integer keys, such as COSE ([RFC 9052](https://www.rfc-editor.org/rfc/rfc9052.html)) or CWT ([RFC 8392](https://www.rfc-editor.org/rfc/rfc8392.html)), cannot be read with this library and need a general-purpose CBOR library instead.
A tag in front of a text string key is handled like a tag in front of a value: it is skipped with `cbor_tag_handler_t::ignore` or `cbor_tag_handler_t::store` and rejected with the default `error`. A tag does not turn a key of another type into a text string.
A map with duplicate keys is well-formed, but invalid under [RFC 8949, Section 5.3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-5.3.1). `from_cbor` accepts it and the last value wins; the [SAX interface](https://json.nlohmann.me/features/parsing/sax_interface/index.md) reports every occurrence.
Ill-formed UTF-8 in text strings
[RFC 8949, Section 3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-3.1) requires CBOR text strings (major type 3) to be valid UTF-8, but leaves it up to the decoder whether to enforce this, so checking is opt-in: with the [`error_handler`](https://json.nlohmann.me/api/basic_json/from_cbor/index.md) parameter left at `keep` (the default), `from_cbor()` accepts a text string (object keys included) whose bytes are not valid UTF-8 and hands them back unchanged. Passing `error_handler_t::strict` makes `from_cbor()` check and throw [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113) for ill-formed UTF-8, and `replace`/`ignore` sanitize the string instead of keeping it. However, [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md) still requires valid UTF-8 and throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for a value read with the default `keep` handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. `to_cbor()`'s own [`error_handler`](https://json.nlohmann.me/api/basic_json/to_cbor/index.md) parameter defaults to `keep`, so such a value is written back unchanged; with `strict` (the default if [`JSON_STRICT_BINARY_UTF8`](https://json.nlohmann.me/api/macros/json_strict_binary_utf8/index.md) is enabled), it throws the same exception instead. Byte strings (major type 2) are unaffected, since they are not required to hold text.
[RFC 8949, Section 3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-3.1) requires CBOR text strings (major type 3) to be valid UTF-8 (a string with ill-formed UTF-8 is "well-formed but invalid"), and [Section 5.3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-5.3.1) leaves it up to the decoder whether to verify this, so checking is opt-in: with the [`error_handler`](https://json.nlohmann.me/api/basic_json/from_cbor/index.md) parameter left at `keep` (the default), `from_cbor()` accepts a text string (object keys included) whose bytes are not valid UTF-8 and hands them back unchanged. Passing `error_handler_t::strict` makes `from_cbor()` check and throw [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113) for ill-formed UTF-8, and `replace`/`ignore` sanitize the string instead of keeping it. However, [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md) still requires valid UTF-8 and throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for a value read with the default `keep` handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. `to_cbor()`'s own [`error_handler`](https://json.nlohmann.me/api/basic_json/to_cbor/index.md) parameter defaults to `keep`, so such a value is written back unchanged; with `strict` (the default if [`JSON_STRICT_BINARY_UTF8`](https://json.nlohmann.me/api/macros/json_strict_binary_utf8/index.md) is enabled), it throws the same exception instead. Byte strings (major type 2) are unaffected, since they are not required to hold text.
The chunks of an indefinite-length text string are checked one by one, as required by [RFC 8949, Section 3.2.3](https://www.rfc-editor.org/rfc/rfc8949.html#section-3.2.3): a code point cannot be spread over two chunks. With `error_handler_t::strict` such a string throws `parse_error.113`; `replace` and `ignore` sanitize each chunk on its own.
Tagged items
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing `cbor_tag_handler_t::store` to function `from_cbor` stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the innermost one is stored. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing `cbor_tag_handler_t::store` to function `from_cbor` stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the tag directly in front of it is stored, and only when it has the form 0xD8..0xDB (a tag 0..23 written as 0xC0..0xD7 is skipped); for example, `D8 05 C1 41 AA` yields subtype `null`, whereas `C1 D8 05 41 AA` yields 5. Tags in front of a text string map key are skipped as well. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
Example: deserialize a JSON value from CBOR
@@ -241,7 +258,7 @@ Example: deserialize a JSON value from CBOR
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -255,7 +272,7 @@ int main()
json j = json::from_cbor(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/index.html
+1 -1
View File
@@ -1 +1 @@
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/" rel="canonical"><link href=../arbitrary_types/ rel=prev><link href=bjdata/ rel=next><link rel=icon href=../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>Binary Formats - JSON for Modern C++</title><link rel=stylesheet href=../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../css/custom.css><script>__md_scope=new URL("../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#binary-formats class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> Binary Formats </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div claLine truncated
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/" rel="canonical"><link href=../type_recipes/ rel=prev><link href=bjdata/ rel=next><link rel=icon href=../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>Binary Formats - JSON for Modern C++</title><link rel=stylesheet href=../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../css/custom.css><script>__md_scope=new URL("../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#binary-formats class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> Binary Formats </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </label> <div class=Line truncated
features/binary_formats/index.md
+16 -16
View File
@@ -15,25 +15,25 @@ to efficiently encode JSON values to byte vectors and to decode such vectors.
### Completeness
| Format | Serialization | Deserialization |
| ----------- | --------------------------------------------- | -------------------------------------------- |
| BJData | complete | complete |
| BON8 | incomplete: no unsigned integers above int64 | complete |
| BSON | incomplete: top-level value must be an object | incomplete, but all JSON types are supported |
| CBOR | complete | incomplete, but all JSON types are supported |
| MessagePack | complete | complete |
| UBJSON | complete | complete |
| Format | Serialization | Deserialization |
| ----------- | --------------------------------------------- | --------------------------------------------------------------------- |
| BJData | complete | incomplete: no column-major ND-arrays, no Draft 4 SoA/extension types |
| BON8 | incomplete: no unsigned integers above int64 | complete |
| BSON | incomplete: top-level value must be an object | incomplete, but all JSON types are supported |
| CBOR | complete | incomplete, but all JSON types are supported |
| MessagePack | complete | incomplete, but all JSON types are supported |
| UBJSON | complete | complete |
### Binary values
| Format | Binary values | Binary subtypes |
| ----------- | ------------- | --------------- |
| BJData | not supported | not supported |
| BON8 | not supported | not supported |
| BSON | supported | supported |
| CBOR | supported | supported |
| MessagePack | supported | supported |
| UBJSON | not supported | not supported |
| Format | Binary values | Binary subtypes |
| ----------- | ------------------------ | --------------- |
| BJData | supported (Draft 3 only) | not supported |
| BON8 | not supported | not supported |
| BSON | supported | supported |
| CBOR | supported | supported |
| MessagePack | supported | supported |
| UBJSON | not supported | not supported |
See [binary values](https://json.nlohmann.me/features/binary_values/index.md) for more information.
features/binary_formats/messagepack.md
+69 -44
View File
@@ -9,6 +9,39 @@ extra byte in addition to the strings themselves.
- [MessagePack website](https://msgpack.org)
- [MessagePack specification](https://github.com/msgpack/msgpack/blob/master/spec.md)
## Byte layout
MessagePack encodes each value as a one-byte *format marker*, optionally followed by a length or the payload. The marker range is partitioned to keep common values small: small non-negative integers (0 to 127) and small negative integers (-32 to -1) are the marker byte itself; short strings, arrays, and maps (up to 31 bytes for strings and up to 15 elements for arrays and maps) hide their length in the marker's lower bits. Larger values use dedicated markers that announce the width (8, 16, 32, or 64 bits) of the length or number that follows, in big-endian byte order. Like CBOR, containers are length-prefixed, so no terminator is needed.
Consider the following JSON value (shown formatted for readability):
```json
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_msgpack` encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-d">data</span>.
<div class="byte-layout" markdown="0"><table>
<thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody>
<tr><td><code>{</code></td><td><span class="bl bl-t">84</span></td><td>map with 4 key/value pairs</td></tr>
<tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">A2</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">CD</span> <span class="bl bl-d">01 2C</span></td><td>key: fixstr of length 2, then its UTF-8 bytes; value: uint 16 (big-endian)</td></tr>
<tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">A4</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">A4</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: fixstr of length 4; value: fixstr of length 4</td></tr>
<tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">A2</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">CB</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: fixstr of length 2; value: float 64 (IEEE 754, big-endian)</td></tr>
<tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">A4</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">92</span></td><td>key: fixstr of length 4; value: fixarray with 2 elements</td></tr>
<tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">A1</span> <span class="bl bl-d">61</span></td><td>fixstr of length 1</td></tr>
<tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">A1</span> <span class="bl bl-d">62</span></td><td>fixstr of length 1</td></tr>
<tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr>
<tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr>
</tbody></table></div>
## Serialization
The library uses the following mapping from JSON values types to MessagePack types according to the MessagePack
@@ -76,8 +109,9 @@ specification:
!!! note
Prior to version 3.13.0, NaN and Infinity were instead serialized as a MessagePack float 64 (type 0xCB, 9 bytes
total), because the check used to select the smaller float 32 encoding compared magnitudes with NaN, which is
always `false` and caused the float 32 path to be skipped.
total), because the check used to select the smaller float 32 encoding failed for NaN (all comparisons are `false`)
and for Infinity and -Infinity (outside the float range checked against `std::numeric_limits<float>::lowest()` and
`max()`), so the float 32 path was skipped.
??? example "Example: serialize a JSON value to MessagePack"
@@ -86,7 +120,7 @@ specification:
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
using namespace nlohmann::literals;
int main()
@@ -100,9 +134,9 @@ specification:
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << std::endl;
std::cout << '\n';
}
```
@@ -114,41 +148,6 @@ specification:
```
### Byte layout
Consider the following JSON value (shown formatted for readability):
```json
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_msgpack` encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.
<div class="byte-layout" markdown="0"><table>
<thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody>
<tr><td><code>{</code></td><td><span class="bl bl-t">84</span></td><td>map with 4 key/value pairs</td></tr>
<tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">A2</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">CD</span> <span class="bl bl-d">01 2C</span></td><td>key: fixstr of length 2, then its UTF-8 bytes; value: uint 16 (big-endian)</td></tr>
<tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">A4</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">A4</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: fixstr of length 4; value: fixstr of length 4</td></tr>
<tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">A2</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">CB</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: fixstr of length 2; value: float 64 (IEEE 754, big-endian)</td></tr>
<tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">A4</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">92</span></td><td>key: fixstr of length 4; value: fixarray with 2 elements</td></tr>
<tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">A1</span> <span class="bl bl-d">61</span></td><td>fixstr of length 1</td></tr>
<tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">A1</span> <span class="bl bl-d">62</span></td><td>fixstr of length 1</td></tr>
<tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr>
<tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr>
</tbody></table></div>
## Deserialization
The library maps MessagePack types to JSON value types as follows:
@@ -160,6 +159,7 @@ The library maps MessagePack types to JSON value types as follows:
| fixarray | array | 0x90..0x9F |
| fixstr | string | 0xA0..0xBF |
| nil | `null` | 0xC0 |
| (never used) | – | 0xC1 |
| false | `false` | 0xC2 |
| true | `true` | 0xC3 |
| float 32 | number_float | 0xCA |
@@ -192,6 +192,30 @@ The library maps MessagePack types to JSON value types as follows:
| fixext 16 | binary | 0xD8 |
| negative fixint | number_integer | 0xE0-0xFF |
The byte 0xC1 is reserved as "never used" by the specification. As a value it is rejected with
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) (`invalid byte: 0xC1`); as a map key it is
rejected with [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) and the generic "expected length
specification" message, because the key is read as a string.
Duplicate keys in a map are accepted; the last value wins.
### Extension types (ext, fixext)
MessagePack ext and fixext values are read as binary values, and the ext type byte is stored as the binary subtype.
Although the specification defines the type as a signed 8-bit integer (0 to 127 for applications, -128 to -1 reserved,
with -1 being the Timestamp extension), the library reads it as an unsigned byte: a negative ext type `n` becomes
subtype `256+n`, so ext type -1 is read as subtype 255 and -128 as subtype 128. The Timestamp extension is not
interpreted; it is returned as binary with subtype 255. Use
[`has_subtype()`](../../api/byte_container_with_subtype/has_subtype.md) to tell a `bin` value (no subtype) from an ext
value (subtype, even if it is 0).
When serializing, a binary value with a subtype is always written as fixext (1, 2, 4, 8, or 16 bytes) or ext, even
for subtype 0 (`d6 00 01 02 03 04`); a binary value without a subtype is written as bin. Subtypes 128 to 255 are
written as the reserved negative ext types (255 as -1), so `json::binary({1,2,3,4}, 255)` becomes
`d6 ff 01 02 03 04`, which other MessagePack libraries decode as a 32-bit timestamp. A subtype above 255 throws
[`out_of_range.415`](../../home/exceptions.md#jsonexceptionout_of_range415). The payload length is limited to
4294967295 bytes like all other lengths.
!!! info
Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`.
@@ -202,7 +226,8 @@ The library maps MessagePack types to JSON value types as follows:
JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the
MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are
rejected with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
`allow_exceptions` set to `false`, a discarded value) naming the type of the key that was found, for instance:
`allow_exceptions` set to `false`, a discarded value). For most key types the message names the type that was
found, for instance:
```
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0
@@ -214,7 +239,7 @@ The library maps MessagePack types to JSON value types as follows:
!!! warning "Ill-formed UTF-8 in string values"
The MessagePack specification explicitly allows a `str` value (`fixstr`, `str 8`, `str 16`, `str 32`) to contain
a byte sequence that is not valid UTF-8, and expects a deserializer to hand the original bytes back unchanged.
a byte sequence that is not valid UTF-8, and recommends that a deserializer provide a way to obtain the original bytes.
This library follows that by default: with its
[`error_handler`](../../api/basic_json/from_msgpack.md) parameter left at `keep` (the default),
`from_msgpack()` reads `str` bytes (object keys included) as-is, without validating them, so such a value
@@ -236,7 +261,7 @@ The library maps MessagePack types to JSON value types as follows:
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -250,7 +275,7 @@ The library maps MessagePack types to JSON value types as follows:
json j = json::from_msgpack(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/messagepack/index.html
+18 -18
View File
@@ -1,8 +1,17 @@
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/messagepack/" rel="canonical"><link href=../cbor/ rel=prev><link href=../ubjson/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>MessagePack - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#messagepack class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> MessagePack </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> <Line truncated
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/messagepack/" rel="canonical"><link href=../cbor/ rel=prev><link href=../ubjson/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>MessagePack - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#messagepack class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> MessagePack </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> <Line truncated
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_msgpack</code> encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">84</span></td><td>map with 4 key/value pairs</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">A2</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">CD</span> <span class="bl bl-d">01 2C</span></td><td>key: fixstr of length 2, then its UTF-8 bytes; value: uint 16 (big-endian)</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">A4</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">A4</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: fixstr of length 4; value: fixstr of length 4</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">A2</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">CB</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: fixstr of length 2; value: float 64 (IEEE 754, big-endian)</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">A4</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">92</span></td><td>key: fixstr of length 4; value: fixarray with 2 elements</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">A1</span> <span class="bl bl-d">61</span></td><td>fixstr of length 1</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">A1</span> <span class="bl bl-d">62</span></td><td>fixstr of length 1</td></tr> <tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr> <tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr> </tbody></table></div> <h2 id=serialization>Serialization<a class=headerlink href=#serialization title="Permanent link">&para;</a></h2> <p>The library uses the following mapping from <abbr title="JavaScript Object Notation">JSON</abbr> values types to MessagePack types according to the MessagePack specification:</p> <table> <thead> <tr> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>value/range</th> <th>MessagePack type</th> <th>first byte</th> </tr> </thead> <tbody> <tr> <td>null</td> <td><code>null</code></td> <td>nil</td> <td>0xC0</td> </tr> <tr> <td>boolean</td> <td><code>true</code></td> <td>true</td> <td>0xC3</td> </tr> <tr> <td>boolean</td> <td><code>false</code></td> <td>false</td> <td>0xC2</td> </tr> <tr> <td>number_integer</td> <td>-9223372036854775808..-2147483649</td> <td>int64</td> <td>0xD3</td> </tr> <tr> <td>number_integer</td> <td>-2147483648..-32769</td> <td>int32</td> <td>0xD2</td> </tr> <tr> <td>number_integer</td> <td>-32768..-129</td> <td>int16</td> <td>0xD1</td> </tr> <tr> <td>number_integer</td> <td>-128..-33</td> <td>int8</td> <td>0xD0</td> </tr> <tr> <td>number_integer</td> <td>-32..-1</td> <td>negative fixint</td> <td>0xE0..0xFF</td> </tr> <tr> <td>number_integer</td> <td>0..127</td> <td>positive fixint</td> <td>0x00..0x7F</td> </tr> <tr> <td>number_integer</td> <td>128..255</td> <td>uint 8</td> <td>0xCC</td> </tr> <tr> <td>number_integer</td> <td>256..65535</td> <td>uint 16</td> <td>0xCD</td> </tr> <tr> <td>number_integer</td> <td>65536..4294967295</td> <td>uint 32</td> <td>0xCE</td> </tr> <tr> <td>number_integer</td> <td>4294967296..18446744073709551615</td> <td>uint 64</td> <td>0xCF</td> </tr> <tr> <td>number_unsigned</td> <td>0..127</td> <td>positive fixint</td> <td>0x00..0x7F</td> </tr> <tr> <td>number_unsigned</td> <td>128..255</td> <td>uint 8</td> <td>0xCC</td> </tr> <tr> <td>number_unsigned</td> <td>256..65535</td> <td>uint 16</td> <td>0xCD</td> </tr> <tr> <td>number_unsigned</td> <td>65536..4294967295</td> <td>uint 32</td> <td>0xCE</td> </tr> <tr> <td>number_unsigned</td> <td>4294967296..18446744073709551615</td> <td>uint 64</td> <td>0xCF</td> </tr> <tr> <td>number_float</td> <td><em>any value representable by a float</em></td> <td>float 32</td> <td>0xCA</td> </tr> <tr> <td>number_float</td> <td><em>any value NOT representable by a float</em></td> <td>float 64</td> <td>0xCB</td> </tr> <tr> <td>string</td> <td><em>length</em>: 0..31</td> <td>fixstr</td> <td>0xA0..0xBF</td> </tr> <tr> <td>string</td> <td><em>length</em>: 32..255</td> <td>str 8</td> <td>0xD9</td> </tr> <tr> <td>string</td> <td><em>length</em>: 256..65535</td> <td>str 16</td> <td>0xDA</td> </tr> <tr> <td>string</td> <td><em>length</em>: 65536..4294967295</td> <td>str 32</td> <td>0xDB</td> </tr> <tr> <td>array</td> <td><em>size</em>: 0..15</td> <td>fixarray</td> <td>0x90..0x9F</td> </tr> <tr> <td>array</td> <td><em>size</em>: 16..65535</td> <td>array 16</td> <td>0xDC</td> </tr> <tr>Line truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=k>namespace</span><span class=w> </span><span class=nn>nlohmann</span><span class=o>::</span><span class=nn>literals</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
@@ -16,26 +25,17 @@
<span class=w> </span><span class=c1>// print the vector content</span>
<span class=w> </span><span class=k>for</span><span class=w> </span><span class=p>(</span><span class=k>auto</span><span class=o>&amp;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>:</span><span class=w> </span><span class=n>v</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>"0x"</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>hex</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setfill</span><span class=p>(</span><span class=sc>'0'</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=p>(</span><span class=kt>int</span><span class=p>)</span><span class=kt>byte</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>" "</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>"0x"</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>hex</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setfill</span><span class=p>(</span><span class=sc>'0'</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=k>static_cast</span><span class=o>&lt;</span><span class=kt>int</span><span class=o>&gt;</span><span class=p>(</span><span class=kt>byte</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=s>" "</span><span class=p>;</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=mh>0x82</span><span class=w> </span><span class=mh>0xa7</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x6f</span><span class=w> </span><span class=mh>0x6d</span><span class=w> </span><span class=mh>0x70</span><span class=w> </span><span class=mh>0x61</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x74</span><span class=w> </span><span class=mh>0xc3</span><span class=w> </span><span class=mh>0xa6</span><span class=w> </span><span class=mh>0x73</span><span class=w> </span><span class=mh>0x63</span><span class=w> </span><span class=mh>0x68</span><span class=w> </span><span class=mh>0x65</span><span class=w> </span><span class=mh>0x6d</span><span class=w> </span><span class=mh>0x61</span><span class=w> </span><span class=mh>0x00</span><span class=w> </span>
</code></pre></div> </details> <h3 id=byte-layout>Byte layout<a class=headerlink href=#byte-layout title="Permanent link">&para;</a></h3> <p>Consider the following <abbr title="JavaScript Object Notation">JSON</abbr> value (shown formatted for readability):</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_msgpack</code> encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">84</span></td><td>map with 4 key/value pairs</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-t">A2</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">CD</span> <span class="bl bl-d">01 2C</span></td><td>key: fixstr of length 2, then its UTF-8 bytes; value: uint 16 (big-endian)</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-t">A4</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">A4</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: fixstr of length 4; value: fixstr of length 4</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-t">A2</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">CB</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: fixstr of length 2; value: float 64 (IEEE 754, big-endian)</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-t">A4</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">92</span></td><td>key: fixstr of length 4; value: fixarray with 2 elements</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">A1</span> <span class="bl bl-d">61</span></td><td>fixstr of length 1</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">A1</span> <span class="bl bl-d">62</span></td><td>fixstr of length 1</td></tr> <tr><td><code>]</code></td><td></td><td>no end marker: the element count was given up front</td></tr> <tr><td><code>}</code></td><td></td><td>no end marker: the member count was given up front</td></tr> </tbody></table></div> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps MessagePack types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th>MessagePack type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>first byte</th> </tr> </thead> <tbody> <tr> <td>positive fixint</td> <td>number_unsigned</td> <td>0x00..0x7F</td> </tr> <tr> <td>fixmap</td> <td>object</td> <td>0x80..0x8F</td> </tr> <tr> <td>fixarray</td> <td>array</td> <td>0x90..0x9F</td> </tr> <tr> <td>fixstr</td> <td>string</td> <td>0xA0..0xBF</td> </tr> <tr> <td>nil</td> <td><code>null</code></td> <td>0xC0</td> </tr> <tr> <td>false</td> <td><code>false</code></td> <td>0xC2</td> </tr> <tr> <td>true</td> <td><code>true</code></td> <td>0xC3</td> </tr> <tr> <td>float 32</td> <td>number_float</td> <td>0xCA</td> </tr> <tr> <td>float 64</td> <td>number_float</td> <td>0xCB</td> </tr> <tr> <td>uint 8</td> <td>number_unsigned</td> <td>0xCC</td> </tr> <tr> <td>uint 16</td> <td>number_unsigned</td> <td>0xCD</td> </tr> <tr> <td>uint 32</td> <td>number_unsigned</td> <td>0xCE</td> </tr> <tr> <td>uint 64</td> <td>number_unsigned</td> <td>0xCF</td> </tr> <tr> <td>int 8</td> <td>number_integer</td> <td>0xD0</td> </tr> <tr> <td>int 16</td> <td>number_integer</td> <td>0xD1</td> </tr> <tr> <td>int 32</td> <td>number_integer</td> <td>0xD2</td> </tr> <tr> <td>int 64</td> <td>number_integer</td> <td>0xD3</td> </tr> <tr> <td>str 8</td> <td>string</td> <td>0xD9</td> </tr> <tr> <td>str 16</td> <td>string</td> <td>0xDA</td> </tr> <tr> <td>str 32</td> <td>string</td> <td>0xDB</td> </tr> <tr> <td>array 16</td> <td>array</td> <td>0xDC</td> </tr> <tr> <td>array 32</td> <td>array</td> <td>0xDD</td> </tr> <tr> <td>map 16</td> <td>object</td> <td>0xDE</td> </tr> <tr> <td>map 32</td> <td>object</td> <td>0xDF</td> </tr> <tr> <td>bin 8</td> <td>binary</td> <td>0xC4</td> </tr> <tr> <td>bin 16</td> <td>binary</td> <td>0xC5</td> </tr> <tr> <td>bin 32</td> <td>binary</td> <td>0xC6</td> </tr> <tr> <td>ext 8</td> <td>binary</td> <td>0xC7</td> </tr> <tr> <td>ext 16</td> <td>binary</td> <td>0xC8</td> </tr> <tr> <td>ext 32</td> <td>binary</td> <td>0xC9</td> </tr> <tr> <td>fixext 1</td> <td>binary</td> <td>0xD4</td> </tr> <tr> <td>fixext 2</td> <td>binary</td> <td>0xD5</td> </tr> <tr> <td>fixext 4</td> <td>binary</td> <td>0xD6</td> </tr> <tr> <td>fixext 8</td> <td>binary</td> <td>0xD7</td> </tr> <tr> <td>fixext 16</td> <td>binary</td> <td>0xD8</td> </tr> <tr> <td>negative fixint</td> <td>number_integer</td> <td>0xE0-0xFF</td> </tr> </tbody> </table> <div class="admonition info"> <p class=admonition-title>Info</p> <p>Any MessagePack output created by <code>to_msgpack</code> can be successfully parsed by <code>from_msgpack</code>.</p> </div> <div class="admonition warning"> <Line truncated
</code></pre></div> <p>This applies to the <a href=../../parsing/sax_interface/ ><abbr title="Simple API for XML">SAX</abbr> interface</a> as well, as the key is read before it is passed on. Such input needs a general-purpose MessagePack library instead.</p> </div> <div class="admonition warning"> <p class=admonition-title>Ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8 in string values</p> <p>The MessagePack specification explicitly allows a <code>str</code> value (<code>fixstr</code>, <code>str 8</code>, <code>str 16</code>, <code>str 32</code>) to contain a byte sequence that is not valid <abbr title="Unicode Transformation Format">UTF</abbr>-8, and expects a deserializer to hand the original bytes back unchanged. This library follows that by default: with its <a href=../../../api/basic_json/from_msgpack/ ><code>error_handler</code></a> parameter left at <code>keep</code> (the default), <code>from_msgpack()</code> reads <code>str</code> bytes (object keys included) as-is, without validating them, so such a value round-trips through <code>from_msgpack(to_msgpack(j))</code> byte for byte. Passing <code>error_handler_t::strict</code> makes <code>from_msgpack()</code> check anyway and throw <a href=../../../home/exceptions/#jsonexceptionparse_error113><code>parse_error.113</code></a> for ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8, and <code>replace</code>/<code>ignore</code> sanitize the string instead of keeping it. <code>to_msgpack()</code> also writes <code>str</code> bytes as-is by default, since the specification permits it; its <a href=../../../api/basic_json/to_msgpack/ ><code>error_handler</code></a> parameter can be set to <code>strict</code> to throw <a href=../../../home/exceptions/#jsonexceptiontype_error316><code>type_error.316</code></a> instead, or to <code>replace</code>/<code>ignore</code> to sanitize the string, for instance for a decoder that rejects ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8. However, <a href=../../../api/basic_json/dump/ ><code>dump()</code></a> still requires valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 and throws <a href=../../../home/exceptions/#jsonexceptiontype_error316><code>type_error.316</code></a> for a value read this way with the default <code>keep</code> handler, unless an error handler is passed that replaces or ignores the ill-formed bytes.</p> </div> <details class=example> <summary>Example: deserialize a <abbr title="JavaScript Object Notation">JSON</abbr> value from MessagePack</summary> <div class=highlight><pre><span></span><code><span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iostream&gt;</span>
</code></pre></div> </details> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps MessagePack types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th>MessagePack type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>first byte</th> </tr> </thead> <tbody> <tr> <td>positive fixint</td> <td>number_unsigned</td> <td>0x00..0x7F</td> </tr> <tr> <td>fixmap</td> <td>object</td> <td>0x80..0x8F</td> </tr> <tr> <td>fixarray</td> <td>array</td> <td>0x90..0x9F</td> </tr> <tr> <td>fixstr</td> <td>string</td> <td>0xA0..0xBF</td> </tr> <tr> <td>nil</td> <td><code>null</code></td> <td>0xC0</td> </tr> <tr> <td>(never used)</td> <td>–</td> <td>0xC1</td> </tr> <tr> <td>false</td> <td><code>false</code></td> <td>0xC2</td> </tr> <tr> <td>true</td> <td><code>true</code></td> <td>0xC3</td> </tr> <tr> <td>float 32</td> <td>number_float</td> <td>0xCA</td> </tr> <tr> <td>float 64</td> <td>number_float</td> <td>0xCB</td> </tr> <tr> <td>uint 8</td> <td>number_unsigned</td> <td>0xCC</td> </tr> <tr> <td>uint 16</td> <td>number_unsigned</td> <td>0xCD</td> </tr> <tr> <td>uint 32</td> <td>number_unsigned</td> <td>0xCE</td> </tr> <tr> <td>uint 64</td> <td>number_unsigned</td> <td>0xCF</td> </tr> <tr> <td>int 8</td> <td>number_integer</td> <td>0xD0</td> </tr> <tr> <td>int 16</td> <td>number_integer</td> <td>0xD1</td> </tr> <tr> <td>int 32</td> <td>number_integer</td> <td>0xD2</td> </tr> <tr> <td>int 64</td> <td>number_integer</td> <td>0xD3</td> </tr> <tr> <td>str 8</td> <td>string</td> <td>0xD9</td> </tr> <tr> <td>str 16</td> <td>string</td> <td>0xDA</td> </tr> <tr> <td>str 32</td> <td>string</td> <td>0xDB</td> </tr> <tr> <td>array 16</td> <td>array</td> <td>0xDC</td> </tr> <tr> <td>array 32</td> <td>array</td> <td>0xDD</td> </tr> <tr> <td>map 16</td> <td>object</td> <td>0xDE</td> </tr> <tr> <td>map 32</td> <td>object</td> <td>0xDF</td> </tr> <tr> <td>bin 8</td> <td>binary</td> <td>0xC4</td> </tr> <tr> <td>bin 16</td> <td>binary</td> <td>0xC5</td> </tr> <tr> <td>bin 32</td> <td>binary</td> <td>0xC6</td> </tr> <tr> <td>ext 8</td> <td>binary</td> <td>0xC7</td> </tr> <tr> <td>ext 16</td> <td>binary</td> <td>0xC8</td> </tr> <tr> <td>ext 32</td> <td>binary</td> <td>0xC9</td> </tr> <tr> <td>fixext 1</td> <td>binary</td> <td>0xD4</td> </tr> <tr> <td>fixext 2</td> <td>binary</td> <td>0xD5</td> </tr> <tr> <td>fixext 4</td> <td>binary</td> <td>0xD6</td> </tr> <tr> <td>fixext 8</td> <td>binary</td> <td>0xD7</td> </tr> <tr> <td>fixext 16</td> <td>binary</td> <td>0xD8</td> </tr> <tr> <td>negative fixint</td> <td>number_integer</td> <td>0xE0-0xFF</td> </tr> </tbody> </table> <p>The byte 0xC1 is reserved as "never used" by the specification. As a value it is rejected with <a href=../../../home/exceptions/#jsonexceptionparse_error112><code>parse_error.112</code></a> (<code>invalid byte: 0xC1</code>); as a map key it is rejected with <a href=../../../home/exceptions/#jsonexceptionparse_error113><code>parse_error.113</code></a> and the generic "expected length specification" message, because the key is read as a string.</p> <p>Duplicate keys in a map are accepted; the last value wins.</p> <h3 id=extension-types-ext-fixext>Extension types (ext, fixext)<a class=headerlink href=#extension-types-ext-fixext title="Permanent link">&para;</a></h3> <p>MessagePack ext and fixext values are read as binary values, and the ext type byte is stored as the binary subtype. Although the specification defines the type as a signed 8-bit integer (0 to 127 for applications, -128 to -1 reserved, with -1 being the Timestamp extension), the library reads it as an unsigned byte: a negative ext type <code>n</code> becomes subtype <code>256+n</code>, so ext type -1 is read as subtype 255 and -128 as subtype 128. The Timestamp extension is not interpreted; it is returned as binary with subtype 255. Use <a href=../../../api/byte_container_with_subtype/has_subtype/ ><code>has_subtype()</code></a> to tell a <code>bin</code> value (no subtype) from an ext value (subtype, even if it is 0).</p> <p>When serializing, a binary value with a subtype is always written as fixext (1, 2, 4, 8, or 16 bytes) or ext, even for subtype 0 (<code>d6 00 01 02 03 04</code>); a binary value without a subtype is written as bin. Subtypes 128 to 255 are written as the reserved negative ext types (255 as -1), so <code>json::binary({1,2,3,4}, 255)</code> becomes <code>d6 ff 01 02 03 04</code>, which other MessagePack libraries decode as a 32-bit timestamp. A subtype above 255 throws <a href=../../../home/exceptions/#jsonexceptionout_of_range415><code>out_of_range.415</code></a>. The payload length is limited to 4294967295 bytes like all other lengths.</p> <div class="admonition info"> <p class=admonition-title>Info</p> <p>Any MessagePack output created by <code>to_msgpack</code> can be successfully parsed by <code>from_msgpack</code>.</pLine truncated
</code></pre></div> <p>This applies to the <a href=../../parsing/sax_interface/ ><abbr title="Simple API for XML">SAX</abbr> interface</a> as well, as the key is read before it is passed on. Such input needs a general-purpose MessagePack library instead.</p> </div> <div class="admonition warning"> <p class=admonition-title>Ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8 in string values</p> <p>The MessagePack specification explicitly allows a <code>str</code> value (<code>fixstr</code>, <code>str 8</code>, <code>str 16</code>, <code>str 32</code>) to contain a byte sequence that is not valid <abbr title="Unicode Transformation Format">UTF</abbr>-8, and recommends that a deserializer provide a way to obtain the original bytes. This library follows that by default: with its <a href=../../../api/basic_json/from_msgpack/ ><code>error_handler</code></a> parameter left at <code>keep</code> (the default), <code>from_msgpack()</code> reads <code>str</code> bytes (object keys included) as-is, without validating them, so such a value round-trips through <code>from_msgpack(to_msgpack(j))</code> byte for byte. Passing <code>error_handler_t::strict</code> makes <code>from_msgpack()</code> check anyway and throw <a href=../../../home/exceptions/#jsonexceptionparse_error113><code>parse_error.113</code></a> for ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8, and <code>replace</code>/<code>ignore</code> sanitize the string instead of keeping it. <code>to_msgpack()</code> also writes <code>str</code> bytes as-is by default, since the specification permits it; its <a href=../../../api/basic_json/to_msgpack/ ><code>error_handler</code></a> parameter can be set to <code>strict</code> to throw <a href=../../../home/exceptions/#jsonexceptiontype_error316><code>type_error.316</code></a> instead, or to <code>replace</code>/<code>ignore</code> to sanitize the string, for instance for a decoder that rejects ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8. However, <a href=../../../api/basic_json/dump/ ><code>dump()</code></a> still requires valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 and throws <a href=../../../home/exceptions/#jsonexceptiontype_error316><code>type_error.316</code></a> for a value read this way with the default <code>keep</code> handler, unless an error handler is passed that replaces or ignores the ill-formed bytes.</p> </div> <details class=example> <summary>Example: deserialize a <abbr title="JavaScript Object Notation">JSON</abbr> value from MessagePack</summary> <div class=highlight><pre><span></span><code><span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iostream&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
<span class=p>{</span>
@@ -49,10 +49,10 @@
<span class=w> </span><span class=n>json</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>json</span><span class=o>::</span><span class=n>from_msgpack</span><span class=p>(</span><span class=n>v</span><span class=p>);</span>
<span class=w> </span><span class=c1>// print the deserialized JSON value</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"compact"</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=p>,</span>
<span class=w> </span><span class=nt>"schema"</span><span class=p>:</span><span class=w> </span><span class=mi>0</span>
<span class=p>}</span>
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 24, 2026 15:57:44 UTC">October 24, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 26, 2026 17:25:44 UTC">October 26, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
features/binary_formats/messagepack/index.md
+51 -38
View File
@@ -7,6 +7,38 @@ References
- [MessagePack website](https://msgpack.org)
- [MessagePack specification](https://github.com/msgpack/msgpack/blob/master/spec.md)
## Byte layout
MessagePack encodes each value as a one-byte *format marker*, optionally followed by a length or the payload. The marker range is partitioned to keep common values small: small non-negative integers (0 to 127) and small negative integers (-32 to -1) are the marker byte itself; short strings, arrays, and maps (up to 31 bytes for strings and up to 15 elements for arrays and maps) hide their length in the marker's lower bits. Larger values use dedicated markers that announce the width (8, 16, 32, or 64 bits) of the length or number that follows, in big-endian byte order. Like CBOR, containers are length-prefixed, so no terminator is needed.
Consider the following JSON value (shown formatted for readability):
```
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_msgpack` encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: type/marker data.
| JSON | Bytes | Meaning |
| ----------------- | ----------------------------------- | -------------------------------------------------------------------------- |
| `{` | 84 | map with 4 key/value pairs |
| `"id": 300,` | A2 69 64 CD 01 2C | key: fixstr of length 2, then its UTF-8 bytes; value: uint 16 (big-endian) |
| `"name": "json",` | A4 6E 61 6D 65 A4 6A 73 6F 6E | key: fixstr of length 4; value: fixstr of length 4 |
| `"pi": 3.14,` | A2 70 69 CB 40 09 1E B8 51 EB 85 1F | key: fixstr of length 2; value: float 64 (IEEE 754, big-endian) |
| `"tags": [` | A4 74 61 67 73 92 | key: fixstr of length 4; value: fixarray with 2 elements |
| `"a",` | A1 61 | fixstr of length 1 |
| `"b"` | A1 62 | fixstr of length 1 |
| `]` | | no end marker: the element count was given up front |
| `}` | | no end marker: the member count was given up front |
## Serialization
The library uses the following mapping from JSON values types to MessagePack types according to the MessagePack specification:
@@ -70,7 +102,7 @@ NaN/infinity handling
Note
Prior to version 3.13.0 unreleased, NaN and Infinity were instead serialized as a MessagePack float 64 (type 0xCB, 9 bytes total), because the check used to select the smaller float 32 encoding compared magnitudes with NaN, which is always `false` and caused the float 32 path to be skipped.
Prior to version 3.13.0 unreleased, NaN and Infinity were instead serialized as a MessagePack float 64 (type 0xCB, 9 bytes total), because the check used to select the smaller float 32 encoding failed for NaN (all comparisons are `false`) and for Infinity and -Infinity (outside the float range checked against `std::numeric_limits<float>::lowest()` and `max()`), so the float 32 path was skipped.
Example: serialize a JSON value to MessagePack
@@ -79,7 +111,7 @@ Example: serialize a JSON value to MessagePack
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
using namespace nlohmann::literals;
int main()
@@ -93,9 +125,9 @@ int main()
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << std::endl;
std::cout << '\n';
}
```
@@ -105,36 +137,6 @@ Output:
0x82 0xa7 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0xc3 0xa6 0x73 0x63 0x68 0x65 0x6d 0x61 0x00
```
### Byte layout
Consider the following JSON value (shown formatted for readability):
```
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
```
`to_msgpack` encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: type/marker length/count/size data.
| JSON | Bytes | Meaning |
| ----------------- | ----------------------------------- | -------------------------------------------------------------------------- |
| `{` | 84 | map with 4 key/value pairs |
| `"id": 300,` | A2 69 64 CD 01 2C | key: fixstr of length 2, then its UTF-8 bytes; value: uint 16 (big-endian) |
| `"name": "json",` | A4 6E 61 6D 65 A4 6A 73 6F 6E | key: fixstr of length 4; value: fixstr of length 4 |
| `"pi": 3.14,` | A2 70 69 CB 40 09 1E B8 51 EB 85 1F | key: fixstr of length 2; value: float 64 (IEEE 754, big-endian) |
| `"tags": [` | A4 74 61 67 73 92 | key: fixstr of length 4; value: fixarray with 2 elements |
| `"a",` | A1 61 | fixstr of length 1 |
| `"b"` | A1 62 | fixstr of length 1 |
| `]` | | no end marker: the element count was given up front |
| `}` | | no end marker: the member count was given up front |
## Deserialization
The library maps MessagePack types to JSON value types as follows:
@@ -146,6 +148,7 @@ The library maps MessagePack types to JSON value types as follows:
| fixarray | array | 0x90..0x9F |
| fixstr | string | 0xA0..0xBF |
| nil | `null` | 0xC0 |
| (never used) | – | 0xC1 |
| false | `false` | 0xC2 |
| true | `true` | 0xC3 |
| float 32 | number_float | 0xCA |
@@ -178,13 +181,23 @@ The library maps MessagePack types to JSON value types as follows:
| fixext 16 | binary | 0xD8 |
| negative fixint | number_integer | 0xE0-0xFF |
The byte 0xC1 is reserved as "never used" by the specification. As a value it is rejected with [`parse_error.112`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error112) (`invalid byte: 0xC1`); as a map key it is rejected with [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113) and the generic "expected length specification" message, because the key is read as a string.
Duplicate keys in a map are accepted; the last value wins.
### Extension types (ext, fixext)
MessagePack ext and fixext values are read as binary values, and the ext type byte is stored as the binary subtype. Although the specification defines the type as a signed 8-bit integer (0 to 127 for applications, -128 to -1 reserved, with -1 being the Timestamp extension), the library reads it as an unsigned byte: a negative ext type `n` becomes subtype `256+n`, so ext type -1 is read as subtype 255 and -128 as subtype 128. The Timestamp extension is not interpreted; it is returned as binary with subtype 255. Use [`has_subtype()`](https://json.nlohmann.me/api/byte_container_with_subtype/has_subtype/index.md) to tell a `bin` value (no subtype) from an ext value (subtype, even if it is 0).
When serializing, a binary value with a subtype is always written as fixext (1, 2, 4, 8, or 16 bytes) or ext, even for subtype 0 (`d6 00 01 02 03 04`); a binary value without a subtype is written as bin. Subtypes 128 to 255 are written as the reserved negative ext types (255 as -1), so `json::binary({1,2,3,4}, 255)` becomes `d6 ff 01 02 03 04`, which other MessagePack libraries decode as a 32-bit timestamp. A subtype above 255 throws [`out_of_range.415`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range415). The payload length is limited to 4294967295 bytes like all other lengths.
Info
Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`.
Object keys
MessagePack allows map keys of any type, whereas JSON only allows strings as keys in object values. Like the JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are rejected with a [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113) exception (or, with `allow_exceptions` set to `false`, a discarded value) naming the type of the key that was found, for instance:
MessagePack allows map keys of any type, whereas JSON only allows strings as keys in object values. Like the JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are rejected with a [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113) exception (or, with `allow_exceptions` set to `false`, a discarded value). For most key types the message names the type that was found, for instance:
```
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0
@@ -194,7 +207,7 @@ This applies to the [SAX interface](https://json.nlohmann.me/features/parsing/sa
Ill-formed UTF-8 in string values
The MessagePack specification explicitly allows a `str` value (`fixstr`, `str 8`, `str 16`, `str 32`) to contain a byte sequence that is not valid UTF-8, and expects a deserializer to hand the original bytes back unchanged. This library follows that by default: with its [`error_handler`](https://json.nlohmann.me/api/basic_json/from_msgpack/index.md) parameter left at `keep` (the default), `from_msgpack()` reads `str` bytes (object keys included) as-is, without validating them, so such a value round-trips through `from_msgpack(to_msgpack(j))` byte for byte. Passing `error_handler_t::strict` makes `from_msgpack()` check anyway and throw [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113) for ill-formed UTF-8, and `replace`/`ignore` sanitize the string instead of keeping it. `to_msgpack()` also writes `str` bytes as-is by default, since the specification permits it; its [`error_handler`](https://json.nlohmann.me/api/basic_json/to_msgpack/index.md) parameter can be set to `strict` to throw [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) instead, or to `replace`/`ignore` to sanitize the string, for instance for a decoder that rejects ill-formed UTF-8. However, [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md) still requires valid UTF-8 and throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for a value read this way with the default `keep` handler, unless an error handler is passed that replaces or ignores the ill-formed bytes.
The MessagePack specification explicitly allows a `str` value (`fixstr`, `str 8`, `str 16`, `str 32`) to contain a byte sequence that is not valid UTF-8, and recommends that a deserializer provide a way to obtain the original bytes. This library follows that by default: with its [`error_handler`](https://json.nlohmann.me/api/basic_json/from_msgpack/index.md) parameter left at `keep` (the default), `from_msgpack()` reads `str` bytes (object keys included) as-is, without validating them, so such a value round-trips through `from_msgpack(to_msgpack(j))` byte for byte. Passing `error_handler_t::strict` makes `from_msgpack()` check anyway and throw [`parse_error.113`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error113) for ill-formed UTF-8, and `replace`/`ignore` sanitize the string instead of keeping it. `to_msgpack()` also writes `str` bytes as-is by default, since the specification permits it; its [`error_handler`](https://json.nlohmann.me/api/basic_json/to_msgpack/index.md) parameter can be set to `strict` to throw [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) instead, or to `replace`/`ignore` to sanitize the string, for instance for a decoder that rejects ill-formed UTF-8. However, [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md) still requires valid UTF-8 and throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for a value read this way with the default `keep` handler, unless an error handler is passed that replaces or ignores the ill-formed bytes.
Example: deserialize a JSON value from MessagePack
@@ -203,7 +216,7 @@ Example: deserialize a JSON value from MessagePack
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -217,7 +230,7 @@ int main()
json j = json::from_msgpack(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/ubjson.md
+222 -189
View File
@@ -5,177 +5,11 @@ achieve the generality of JSON, combined with being much easier to process than
!!! abstract "References"
- [UBJSON Website](http://ubjson.org)
- [UBJSON Website](https://ubjson.org)
## Serialization
## Byte layout
The library uses the following mapping from JSON values types to UBJSON types according to the UBJSON specification:
| JSON value type | value/range | UBJSON type | marker |
|-----------------|-------------------------------------------|----------------|--------|
| null | `null` | null | `Z` |
| boolean | `true` | true | `T` |
| boolean | `false` | false | `F` |
| number_integer | -9223372036854775808..-2147483649 | int64 | `L` |
| number_integer | -2147483648..-32769 | int32 | `l` |
| number_integer | -32768..-129 | int16 | `I` |
| number_integer | -128..127 | int8 | `i` |
| number_integer | 128..255 | uint8 | `U` |
| number_integer | 256..32767 | int16 | `I` |
| number_integer | 32768..2147483647 | int32 | `l` |
| number_integer | 2147483648..9223372036854775807 | int64 | `L` |
| number_unsigned | 0..127 | int8 | `i` |
| number_unsigned | 128..255 | uint8 | `U` |
| number_unsigned | 256..32767 | int16 | `I` |
| number_unsigned | 32768..2147483647 | int32 | `l` |
| number_unsigned | 2147483648..9223372036854775807 | int64 | `L` |
| number_unsigned | 9223372036854775808..18446744073709551615 | high-precision | `H` |
| number_float | *any value* | float64 | `D` |
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format* | array | `[` |
| object | *see notes on optimized format* | map | `{` |
!!! success "Complete mapping"
The mapping is **complete** in the sense that any JSON value type can be converted to a UBJSON value.
Any UBJSON output created by `to_ubjson` can be successfully parsed by `from_ubjson`.
!!! warning "Size constraints"
The following values can **not** be converted to a UBJSON value:
- strings with more than 9223372036854775807 bytes (theoretical)
!!! warning "UTF-8 validation of string values and object keys"
UBJSON's required string encoding is UTF-8. By default (the [`error_handler`](../../api/basic_json/to_ubjson.md)
parameter left at `keep`), `to_ubjson()` writes the bytes of string values and object keys unchanged, even if they
are not valid UTF-8. With `error_handler_t::strict`, it throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for ill-formed UTF-8 instead;
`replace`/`ignore` sanitize the string. [`JSON_STRICT_BINARY_UTF8`](../../api/macros/json_strict_binary_utf8.md)
makes `strict` the default.
!!! info "Unused UBJSON markers"
The following markers are not used in the conversion:
- `Z`: no-op values are not created.
- `C`: single-byte strings are serialized with `S` markers.
!!! info "NaN/infinity handling"
If NaN or Infinity are stored inside a JSON number, they are serialized properly. This behavior differs from the
[`dump()`](../../api/basic_json/dump.md) function which serializes NaN or Infinity to `null`.
!!! info "Optimized formats"
The optimized formats for containers are supported: Parameter `use_size` adds size information to the beginning of a
container and removes the closing marker. Parameter `use_type` further checks whether all elements of a container
have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be
used together with `use_size = true`.
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the
receiving side is immediately informed on the number of elements of the container.
An array whose type marker is `Z` (null), `T` (true) or `F` (false) stores no payload at all, because the marker
already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side
allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
[`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576
(`1 << 20`), and `to_ubjson` writes longer arrays of these types without the annotation, so any value it produces
can be read back.
!!! info "Binary values"
If the JSON data contains the binary type, the value stored is a list of integers, as suggested by the UBJSON
documentation. In particular, this means that serialization and the deserialization of a JSON containing binary
values into UBJSON and back will result in a different JSON object.
??? example "Example: serialize JSON values to UBJSON, with and without size/type optimization"
```cpp
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using namespace nlohmann::literals;
// function to print UBJSON's diagnostic format
void print_byte(uint8_t byte)
{
if (32 < byte and byte < 128)
{
std::cout << (char)byte;
}
else
{
std::cout << (int)byte;
}
}
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": false})"_json;
// serialize it to UBJSON
std::vector<std::uint8_t> v = json::to_ubjson(j);
// print the vector content
for (auto& byte : v)
{
print_byte(byte);
}
std::cout << std::endl;
// create an array of numbers
json array = {1, 2, 3, 4, 5, 6, 7, 8};
// serialize it to UBJSON using default representation
std::vector<std::uint8_t> v_array = json::to_ubjson(array);
// serialize it to UBJSON using size optimization
std::vector<std::uint8_t> v_array_size = json::to_ubjson(array, true);
// serialize it to UBJSON using type optimization
std::vector<std::uint8_t> v_array_size_and_type = json::to_ubjson(array, true, true);
// print the vector contents
for (auto& byte : v_array)
{
print_byte(byte);
}
std::cout << std::endl;
for (auto& byte : v_array_size)
{
print_byte(byte);
}
std::cout << std::endl;
for (auto& byte : v_array_size_and_type)
{
print_byte(byte);
}
std::cout << std::endl;
}
```
Output:
```c
{i7compactTi6schemaF}
[i1i2i3i4i5i6i7i8]
[#i8i1i2i3i4i5i6i7i8
[$i#i812345678
```
### Byte layout
UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares a count (`#`) after the opening marker, optionally preceded by an element type (`$`); an element type is only allowed together with a count. An optimized container saves the closing marker and, with a type, the per-element markers.
Consider the following JSON value (shown formatted for readability):
@@ -206,31 +40,230 @@ Consider the following JSON value (shown formatted for readability):
<tr><td><code>}</code></td><td><span class="bl bl-t">7D</span></td><td>object end (<code>}</code>)</td></tr>
</tbody></table></div>
## Serialization
The library uses the following mapping from JSON values types to UBJSON types according to the UBJSON specification:
| JSON value type | value/range | UBJSON type | marker |
|-----------------|-------------------------------------------|--------------------------------------------------|-----------|
| null | `null` | null | `Z` |
| boolean | `true` | true | `T` |
| boolean | `false` | false | `F` |
| number_integer | -9223372036854775808..-2147483649 | int64 | `L` |
| number_integer | -2147483648..-32769 | int32 | `l` |
| number_integer | -32768..-129 | int16 | `I` |
| number_integer | -128..127 | int8 | `i` |
| number_integer | 128..255 | uint8 | `U` |
| number_integer | 256..32767 | int16 | `I` |
| number_integer | 32768..2147483647 | int32 | `l` |
| number_integer | 2147483648..9223372036854775807 | int64 | `L` |
| number_unsigned | 0..127 | int8 | `i` |
| number_unsigned | 128..255 | uint8 | `U` |
| number_unsigned | 256..32767 | int16 | `I` |
| number_unsigned | 32768..2147483647 | int32 | `l` |
| number_unsigned | 2147483648..9223372036854775807 | int64 | `L` |
| number_unsigned | 9223372036854775808..18446744073709551615 | high-precision | `H` |
| number_float | *any value* | float64 (float32 if `number_float_t` is `float`) | `D` (`d`) |
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format* | array | `[` |
| object | *see notes on optimized format* | object | `{` |
!!! success "Complete mapping"
The mapping is **complete** in the sense that any JSON value type can be converted to a UBJSON value.
Any UBJSON output created by `to_ubjson` can be successfully parsed by `from_ubjson`.
!!! warning "Size constraints"
The following values can **not** be converted to a UBJSON value:
- strings with more than 9223372036854775807 bytes (theoretical)
!!! warning "UTF-8 validation of string values and object keys"
UBJSON's required string encoding is UTF-8. By default (the [`error_handler`](../../api/basic_json/to_ubjson.md)
parameter left at `keep`), `to_ubjson()` writes the bytes of string values and object keys unchanged, even if they
are not valid UTF-8. With `error_handler_t::strict`, it throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for ill-formed UTF-8 instead;
`replace`/`ignore` sanitize the string. [`JSON_STRICT_BINARY_UTF8`](../../api/macros/json_strict_binary_utf8.md)
makes `strict` the default.
!!! info "Unused UBJSON markers"
The following markers are not used in the conversion:
- `N`: no-op values are never written (they are skipped when reading).
- `C`: single-byte strings are serialized with `S` markers.
!!! info "NaN/infinity handling"
NaN and infinity are written as IEEE 754 float64 values (`D`) and read back unchanged. This differs from
[`dump()`](../../api/basic_json/dump.md), which serializes them to `null`, and from the UBJSON specification, which
encodes infinity as `null` (it does not mention NaN); other UBJSON readers may therefore not accept these values.
!!! info "Optimized formats"
The optimized formats for containers are supported: Parameter `use_size` adds size information to the beginning of a
container and removes the closing marker. Parameter `use_type` further checks whether all elements of a container
have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be
used together with `use_size = true`: combining `use_type` with `use_size = false` throws
[`other_error.502`](../../home/exceptions.md#jsonexceptionother_error502) if the value contains a non-empty array,
object, or binary value.
Two elements have the same type if they are written with the same type marker. For example, `[1, 2]` is written with
the type `i`, but `[1, 200]` (`i`, `U`), `[true, false]` (`T`, `F`), and `[1, 2.5]` (`i`, `D`) are not; an empty
container is never given a `$` annotation of its own (an array of empty arrays is still typed `[`).
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the
receiving side is immediately informed on the number of elements of the container.
An array whose type marker is `Z` (null), `T` (true) or `F` (false) stores no payload at all, because the marker
already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side
allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
[`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576
(`1 << 20`), and `to_ubjson` writes longer arrays of these types in the counted but untyped form: the `$` type marker is
dropped, the `#` count is kept, and every element is written with its own marker (`[#l 1048577 Z Z ...`), so any value
it produces can be read back. Objects and untyped counted arrays are not subject to this limit.
!!! info "Binary values"
If the JSON data contains the binary type, the value stored is a list of integers, as suggested by the UBJSON
documentation. In particular, this means that serialization and the deserialization of a JSON containing binary
values into UBJSON and back will result in a different JSON object.
??? example "Example: serialize JSON values to UBJSON, with and without size/type optimization"
```cpp
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using nlohmann::json;
using namespace nlohmann::literals;
// function to print UBJSON's diagnostic format
void print_byte(uint8_t byte)
{
if (32 < byte and byte < 128)
{
std::cout << static_cast<char>(byte);
}
else
{
std::cout << static_cast<int>(byte);
}
}
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": false})"_json;
// serialize it to UBJSON
std::vector<std::uint8_t> v = json::to_ubjson(j);
// print the vector content
for (auto& byte : v)
{
print_byte(byte);
}
std::cout << '\n';
// create an array of numbers
json array = {1, 2, 3, 4, 5, 6, 7, 8};
// serialize it to UBJSON using default representation
std::vector<std::uint8_t> v_array = json::to_ubjson(array);
// serialize it to UBJSON using size optimization
std::vector<std::uint8_t> v_array_size = json::to_ubjson(array, true);
// serialize it to UBJSON using type optimization
std::vector<std::uint8_t> v_array_size_and_type = json::to_ubjson(array, true, true);
// print the vector contents
for (auto& byte : v_array)
{
print_byte(byte);
}
std::cout << '\n';
for (auto& byte : v_array_size)
{
print_byte(byte);
}
std::cout << '\n';
for (auto& byte : v_array_size_and_type)
{
print_byte(byte);
}
std::cout << '\n';
}
```
Output:
```c
{i7compactTi6schemaF}
[i1i2i3i4i5i6i7i8]
[#i8i1i2i3i4i5i6i7i8
[$i#i812345678
```
## Deserialization
The library maps UBJSON types to JSON value types as follows:
| UBJSON type | JSON value type | marker |
|-------------|-----------------------------------------|--------|
| no-op | *no value, next value is read* | `N` |
| null | `null` | `Z` |
| false | `false` | `F` |
| true | `true` | `T` |
| float32 | number_float | `d` |
| float64 | number_float | `D` |
| uint8 | number_unsigned | `U` |
| int8 | number_integer | `i` |
| int16 | number_integer | `I` |
| int32 | number_integer | `l` |
| int64 | number_integer | `L` |
| string | string | `S` |
| char | string | `C` |
| array | array (optimized values are supported) | `[` |
| object | object (optimized values are supported) | `{` |
| UBJSON type | JSON value type | marker |
|----------------|--------------------------------------------------|--------|
| no-op | *no value, next value is read* | `N` |
| null | `null` | `Z` |
| false | `false` | `F` |
| true | `true` | `T` |
| float32 | number_float | `d` |
| float64 | number_float | `D` |
| uint8 | number_unsigned | `U` |
| int8 | number_integer | `i` |
| int16 | number_integer | `I` |
| int32 | number_integer | `l` |
| int64 | number_integer | `L` |
| string | string | `S` |
| char | string | `C` |
| high-precision | number_unsigned, number_integer, or number_float | `H` |
| array | array (optimized values are supported) | `[` |
| object | object (optimized values are supported) | `{` |
!!! info "High-precision numbers"
The payload of `H` is a length-prefixed string that is parsed as a JSON number (surrounding whitespace is ignored).
Integer text in the range of `number_unsigned_t` (non-negative) or `number_integer_t` (negative) is stored as such;
any other valid number text (larger or smaller integers, fractions, and exponents such as `1e2`) is stored as
`number_float`, so precision may be lost. Text that is not a valid JSON number raises
[`parse_error.115`](../../home/exceptions.md#jsonexceptionparse_error115), a number outside the range of
`number_float_t` (`1e999`) raises [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406).
!!! success "Complete mapping"
The mapping is **complete** in the sense that any UBJSON value can be converted to a JSON value.
Every well-formed UBJSON value can be converted to a JSON value, with these exceptions: a high-precision number
outside the range of `number_float_t` ([`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406)),
an optimized array of `Z`, `T`, or `F` with more than 1,048,576 elements, and a count that exceeds the capacity of the
container ([`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408)). High-precision numbers that
do not fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected
with a parse error.
!!! info "Duplicate object keys"
If an object contains the same key more than once, the last value is kept (the UBJSON specification does not
address duplicate keys).
!!! info "Type marker of optimized containers"
The type marker after `$` must be one of `Z`, `N`, `T`, `F`, `i`, `U`, `I`, `l`, `L`, `d`, `D`, `H`, `C`, `S`, `[`, or
`{`. It is checked even if the count is 0; any other marker raises
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112). A counted object typed `N` (`{$N#i1 ...`)
is not supported.
!!! warning "Ill-formed UTF-8 in string values and object keys"
@@ -252,7 +285,7 @@ The library maps UBJSON types to JSON value types as follows:
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -266,7 +299,7 @@ The library maps UBJSON types to JSON value types as follows:
json j = json::from_ubjson(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```
features/binary_formats/ubjson/index.html
+21 -21
View File
@@ -1,8 +1,17 @@
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/ubjson/" rel="canonical"><link href=../messagepack/ rel=prev><link href=../../binary_values/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>UBJSON - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#ubjson class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> UBJSON </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </laLine truncated
<!doctype html><html lang=en class=no-js> <head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=author content="Niels Lohmann"><link href="https://json.nlohmann.me/features/binary_formats/ubjson/" rel="canonical"><link href=../messagepack/ rel=prev><link href=../../binary_values/ rel=next><link rel=icon href=../../../assets/images/favicon.png><meta name=generator content="mkdocs-1.6.1, mkdocs-material-9.7.7"><title>UBJSON - JSON for Modern C++</title><link rel=stylesheet href=../../../assets/stylesheets/main.ec1eaa64.min.css><link rel=stylesheet href=../../../assets/stylesheets/palette.ab4e12ef.min.css><link rel="stylesheet" href="../../../assets/external/fonts.googleapis.com/css.61a430c9.css"><style>:root{--md-text-font:"Roboto";--md-code-font:"JetBrains Mono"}</style><link rel=stylesheet href=../../../css/custom.css><script>__md_scope=new URL("../../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script></head> <body dir=ltr data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer autocomplete=off> <input class=md-toggle data-md-toggle=search type=checkbox id=__search autocomplete=off> <label class=md-overlay for=__drawer></label> <div data-md-component=skip> <a href=#ubjson class=md-skip> Skip to content </a> </div> <div data-md-component=announce> </div> <header class=md-header data-md-component=header> <nav class="md-header__inner md-grid" aria-label=Header> <a href=../../.. title="JSON for Modern C++" class="md-header__button md-logo" aria-label="JSON for Modern C++" data-md-component=logo> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg> </a> <label class="md-header__button md-icon" for=__drawer> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg> </label> <div class=md-header__title data-md-component=header-title> <div class=md-header__ellipsis> <div class=md-header__topic> <span class=md-ellipsis> JSON for Modern C++ </span> </div> <div class=md-header__topic data-md-component=header-topic> <span class=md-ellipsis> UBJSON </span> </div> </div> </div> <form class=md-header__option data-md-component=palette> <input class=md-option data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme=default data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to dark mode" type=radio name=__palette id=__palette_0> <label class="md-header__button md-icon" title="Switch to dark mode" for=__palette_1 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> <input class=md-option data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme=slate data-md-color-primary=indigo data-md-color-accent=indigo aria-label="Switch to light mode" type=radio name=__palette id=__palette_1> <label class="md-header__button md-icon" title="Switch to light mode" for=__palette_0 hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg> </label> </form> <script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script> <label class="md-header__button md-icon" for=__search> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg> </laLine truncated
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_ubjson</code> encodes it to 51 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">7B</span></td><td>object start (<code>{</code>)</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">49</span> <span class="bl bl-d">01 2C</span></td><td>key: length 2 as int8 (<code>i</code>), then UTF-8 bytes; value: int16 (<code>I</code>), big-endian</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">53</span> <span class="bl bl-l">69 04</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: length 4; value: string (<code>S</code>), length 4 as int8, UTF-8 bytes</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">44</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: length 2; value: float64 (<code>D</code>), big-endian</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">5B</span></td><td>key: length 4; value: array start (<code>[</code>)</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">61</span></td><td>string, length 1</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">62</span></td><td>string, length 1</td></tr> <tr><td><code>]</code></td><td><span class="bl bl-t">5D</span></td><td>array end (<code>]</code>)</td></tr> <tr><td><code>}</code></td><td><span class="bl bl-t">7D</span></td><td>object end (<code>}</code>)</td></tr> </tbody></table></div> <h2 id=serialization>Serialization<a class=headerlink href=#serialization title="Permanent link">&para;</a></h2> <p>The library uses the following mapping from <abbr title="JavaScript Object Notation">JSON</abbr> values types to <abbr title="Universal Binary JSON">UBJSON</abbr> types according to the <abbr title="Universal Binary JSON">UBJSON</abbr> specification:</p> <table> <thead> <tr> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>value/range</th> <th><abbr title="Universal Binary JSON">UBJSON</abbr> type</th> <th>marker</th> </tr> </thead> <tbody> <tr> <td>null</td> <td><code>null</code></td> <td>null</td> <td><code>Z</code></td> </tr> <tr> <td>boolean</td> <td><code>true</code></td> <td>true</td> <td><code>T</code></td> </tr> <tr> <td>boolean</td> <td><code>false</code></td> <td>false</td> <td><code>F</code></td> </tr> <tr> <td>number_integer</td> <td>-9223372036854775808..-2147483649</td> <td>int64</td> <td><code>L</code></td> </tr> <tr> <td>number_integer</td> <td>-2147483648..-32769</td> <td>int32</td> <td><code>l</code></td> </tr> <tr> <td>number_integer</td> <td>-32768..-129</td> <td>int16</td> <td><code>I</code></td> </tr> <tr> <td>number_integer</td> <td>-128..127</td> <td>int8</td> <td><code>i</code></td> </tr> <tr> <td>number_integer</td> <td>128..255</td> <td>uint8</td> <td><code>U</code></td> </tr> <tr> <td>number_integer</td> <td>256..32767</td> <td>int16</td> <td><code>I</code></td> </tr> <tr> <td>number_integer</td> <td>32768..2147483647</td> <td>int32</td> <td><code>l</code></td> </tr> <tr> <td>number_integer</td> <td>2147483648..9223372036854775807</td> <td>int64</td> <td><code>L</code></td> </tr> <tr> <td>number_unsigned</td> <td>0..127</td> <td>int8</td> <td><code>i</code></td> </tr> <tr> <td>number_unsigned</td> <td>128..255</td> <td>uint8</td> <td><code>U</code></td> </tr> <tr> <td>number_unsigned</td> <td>256..32767</td> <td>int16</td> <td><code>I</code></td> </tr> <tr> <td>number_unsigned</td> <td>32768..2147483647</td> <td>int32</td> <td><code>l</code></td> </tr> <tr> <td>number_unsigned</td> <td>2147483648..9223372036854775807</td> <td>int64</td> <td><code>L</code></td> </tr> <tr> <td>number_unsigned</td> <td>9223372036854775808..18446744073709551615</td> <td>high-precision</td> <td><code>H</code></td> </tr> <tr> <td>number_float</td> <td><em>any value</em></td> <td>float64 (float32 if <code>number_float_t</code> is <code>float</code>)</td> <td><code>D</code> (<code>d</code>)</td> </tr> <tr> <td>string</td> <td><em>with shortest length indicator</em></td> <td>string</td> <td><code>S</code></td> </tr> <tr> <td>array</td> <td><em>see notes on optimized format</em></td> <td>array</td> <td><code>[</code></td> </tr> <trLine truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=k>namespace</span><span class=w> </span><span class=nn>nlohmann</span><span class=o>::</span><span class=nn>literals</span><span class=p>;</span>
<span class=c1>// function to print UBJSON's diagnostic format</span>
@@ -10,11 +19,11 @@
<span class=p>{</span>
<span class=w> </span><span class=k>if</span><span class=w> </span><span class=p>(</span><span class=mi>32</span><span class=w> </span><span class=o>&lt;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=k>and</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>&lt;</span><span class=w> </span><span class=mi>128</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=p>(</span><span class=kt>char</span><span class=p>)</span><span class=kt>byte</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=k>static_cast</span><span class=o>&lt;</span><span class=kt>char</span><span class=o>&gt;</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=k>else</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=p>(</span><span class=kt>int</span><span class=p>)</span><span class=kt>byte</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=k>static_cast</span><span class=o>&lt;</span><span class=kt>int</span><span class=o>&gt;</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=p>}</span>
@@ -31,7 +40,7 @@
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>print_byte</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=w> </span><span class=c1>// create an array of numbers</span>
<span class=w> </span><span class=n>json</span><span class=w> </span><span class=n>array</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=p>{</span><span class=mi>1</span><span class=p>,</span><span class=w> </span><span class=mi>2</span><span class=p>,</span><span class=w> </span><span class=mi>3</span><span class=p>,</span><span class=w> </span><span class=mi>4</span><span class=p>,</span><span class=w> </span><span class=mi>5</span><span class=p>,</span><span class=w> </span><span class=mi>6</span><span class=p>,</span><span class=w> </span><span class=mi>7</span><span class=p>,</span><span class=w> </span><span class=mi>8</span><span class=p>};</span>
@@ -48,38 +57,29 @@
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>print_byte</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=w> </span><span class=k>for</span><span class=w> </span><span class=p>(</span><span class=k>auto</span><span class=o>&amp;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>:</span><span class=w> </span><span class=n>v_array_size</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>print_byte</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=w> </span><span class=k>for</span><span class=w> </span><span class=p>(</span><span class=k>auto</span><span class=o>&amp;</span><span class=w> </span><span class=kt>byte</span><span class=w> </span><span class=o>:</span><span class=w> </span><span class=n>v_array_size_and_type</span><span class=p>)</span>
<span class=w> </span><span class=p>{</span>
<span class=w> </span><span class=n>print_byte</span><span class=p>(</span><span class=kt>byte</span><span class=p>);</span>
<span class=w> </span><span class=p>}</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=p>{</span><span class=n>i7compactTi6schemaF</span><span class=p>}</span>
<span class=p>[</span><span class=n>i1i2i3i4i5i6i7i8</span><span class=p>]</span>
<span class=p>[</span><span class=err>#</span><span class=n>i8i1i2i3i4i5i6i7i8</span>
<span class=p>[</span><span class=n>$i</span><span class=err>#</span><span class=n>i812345678</span>
</code></pre></div> </details> <h3 id=byte-layout>Byte layout<a class=headerlink href=#byte-layout title="Permanent link">&para;</a></h3> <p>Consider the following <abbr title="JavaScript Object Notation">JSON</abbr> value (shown formatted for readability):</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"id"</span><span class=p>:</span><span class=w> </span><span class=mi>300</span><span class=p>,</span>
<span class=w> </span><span class=nt>"name"</span><span class=p>:</span><span class=w> </span><span class=s2>"json"</span><span class=p>,</span>
<span class=w> </span><span class=nt>"pi"</span><span class=p>:</span><span class=w> </span><span class=mf>3.14</span><span class=p>,</span>
<span class=w> </span><span class=nt>"tags"</span><span class=p>:</span><span class=w> </span><span class=p>[</span>
<span class=w> </span><span class=s2>"a"</span><span class=p>,</span>
<span class=w> </span><span class=s2>"b"</span>
<span class=w> </span><span class=p>]</span>
<span class=p>}</span>
</code></pre></div> <p><code>to_ubjson</code> encodes it to 51 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: <span class="bl bl-t">type/marker</span> <span class="bl bl-l">length/count/size</span> <span class="bl bl-d">data</span>.</p> <div class=byte-layout markdown=0><table> <thead><tr><th>JSON</th><th>Bytes</th><th>Meaning</th></tr></thead><tbody> <tr><td><code>{</code></td><td><span class="bl bl-t">7B</span></td><td>object start (<code>{</code>)</td></tr> <tr><td><code>&quot;id&quot;: 300,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">69 64</span> <span class="bl bl-t">49</span> <span class="bl bl-d">01 2C</span></td><td>key: length 2 as int8 (<code>i</code>), then UTF-8 bytes; value: int16 (<code>I</code>), big-endian</td></tr> <tr><td><code>&quot;name&quot;: &quot;json&quot;,</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">6E 61 6D 65</span> <span class="bl bl-t">53</span> <span class="bl bl-l">69 04</span> <span class="bl bl-d">6A 73 6F 6E</span></td><td>key: length 4; value: string (<code>S</code>), length 4 as int8, UTF-8 bytes</td></tr> <tr><td><code>&quot;pi&quot;: 3.14,</code></td><td><span class="bl bl-l">69 02</span> <span class="bl bl-d">70 69</span> <span class="bl bl-t">44</span> <span class="bl bl-d">40 09 1E B8 51 EB 85 1F</span></td><td>key: length 2; value: float64 (<code>D</code>), big-endian</td></tr> <tr><td><code>&quot;tags&quot;: [</code></td><td><span class="bl bl-l">69 04</span> <span class="bl bl-d">74 61 67 73</span> <span class="bl bl-t">5B</span></td><td>key: length 4; value: array start (<code>[</code>)</td></tr> <tr><td><code>&quot;a&quot;,</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">61</span></td><td>string, length 1</td></tr> <tr><td><code>&quot;b&quot;</code></td><td><span class="bl bl-t">53</span> <span class="bl bl-l">69 01</span> <span class="bl bl-d">62</span></td><td>string, length 1</td></tr> <tr><td><code>]</code></td><td><span class="bl bl-t">5D</span></td><td>array end (<code>]</code>)</td></tr> <tr><td><code>}</code></td><td><span class="bl bl-t">7D</span></td><td>object end (<code>}</code>)</td></tr> </tbody></table></div> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Universal Binary JSON">UBJSON</abbr> types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Universal Binary JSON">UBJSON</abbr> type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>marker</th> </tr> </thead> <tbody> <tr> <td>no-op</td> <td><em>no value, next value is read</em></td> <td><code>N</code></td> </tr> <tr> <td>null</td> <td><code>null</code></td> <td><code>Z</code></td> </tr> <tr> <td>false</td> <td><code>false</code></td> <td><code>F</code></td> </tr> <tr> <td>true</td> <td><code>true</code></td> <td><code>T</code></td> </tr> <tr> <td>float32</td> <td>number_float</td> <td><code>d</code></td> </tr> <tr> <td>float64</td> <td>number_float</td> <td><code>D</code></td> </tr> <tr> <td>uint8</td> <td>number_unsigned</td> <td><code>U</code></td> </tr> <tr> <td>int8</td> <td>number_integer</td> <td><code>i</code></td> </tr> <tr> <td>int16</td> <td>number_integer</td> <td><code>I</code></td> </tr> <tr> <td>int32</td> <td>number_integer</td> <td><code>l</code></td> </tr> <tr> <td>int64</td> <td>number_integer</td> <td><code>L</code></td> </tr> <tr> <td>string</td> <td>string</td> <td><code>S</code></td> </tr> <tr> <td>char</td> <td>string</td> <td><code>C</code></td> </tr> <tr> <td>array</td> <td>array (optimized values are supported)</td> <td><code>[</code></td> </tr> <tr> <td>object</td> <td>object (optimized values are supported)</td> <td><code>{</code></td> </tr> </tbody> </table> <div class="admonition success"> <p class=admonition-title>Complete mapping</p> <p>The mapping is <strong>complete</strong> in the sense that any <abbr title="Universal Binary JSON">UBJSON</abbr> value can be converted to a <abbr title="JavaScript Object Notation">JSON</abbr> value.</p> </div> <div class="admonition warning"> <p class=admonition-title>Ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8 in string values and object keys</p> <p><abbr title="Universal Binary JSON">UBJSON</abbr>'s required string encoding is <abbr title="Unicode Transformation Format">UTF</abbr>-8, but checking it on read is opt-in: with the <a href=../../../api/basic_json/from_ubjson/ ><code>error_handler</code></a> parameter left at <code>keep</code> (the default), <code>from_ubjson()</code> accepts a string value or object key whose bytes are not valid <abbr title="Unicode Transformation Format">UTF</abbr>-8 and hands them back unchanged. Passing <code>error_handler_t::strict</code> makes <code>from_ubjson()</code> check and throw <a href=.Line truncated
</code></pre></div> </details> <h2 id=deserialization>Deserialization<a class=headerlink href=#deserialization title="Permanent link">&para;</a></h2> <p>The library maps <abbr title="Universal Binary JSON">UBJSON</abbr> types to <abbr title="JavaScript Object Notation">JSON</abbr> value types as follows:</p> <table> <thead> <tr> <th><abbr title="Universal Binary JSON">UBJSON</abbr> type</th> <th><abbr title="JavaScript Object Notation">JSON</abbr> value type</th> <th>marker</th> </tr> </thead> <tbody> <tr> <td>no-op</td> <td><em>no value, next value is read</em></td> <td><code>N</code></td> </tr> <tr> <td>null</td> <td><code>null</code></td> <td><code>Z</code></td> </tr> <tr> <td>false</td> <td><code>false</code></td> <td><code>F</code></td> </tr> <tr> <td>true</td> <td><code>true</code></td> <td><code>T</code></td> </tr> <tr> <td>float32</td> <td>number_float</td> <td><code>d</code></td> </tr> <tr> <td>float64</td> <td>number_float</td> <td><code>D</code></td> </tr> <tr> <td>uint8</td> <td>number_unsigned</td> <td><code>U</code></td> </tr> <tr> <td>int8</td> <td>number_integer</td> <td><code>i</code></td> </tr> <tr> <td>int16</td> <td>number_integer</td> <td><code>I</code></td> </tr> <tr> <td>int32</td> <td>number_integer</td> <td><code>l</code></td> </tr> <tr> <td>int64</td> <td>number_integer</td> <td><code>L</code></td> </tr> <tr> <td>string</td> <td>string</td> <td><code>S</code></td> </tr> <tr> <td>char</td> <td>string</td> <td><code>C</code></td> </tr> <tr> <td>high-precision</td> <td>number_unsigned, number_integer, or number_float</td> <td><code>H</code></td> </tr> <tr> <td>array</td> <td>array (optimized values are supported)</td> <td><code>[</code></td> </tr> <tr> <td>object</td> <td>object (optimized values are supported)</td> <td><code>{</code></td> </tr> </tbody> </table> <div class="admonition info"> <p class=admonition-title>High-precision numbers</p> <p>The payload of <code>H</code> is a length-prefixed string that is parsed as a <abbr title="JavaScript Object Notation">JSON</abbr> number (surrounding whitespace is ignored). Integer text in the range of <code>number_unsigned_t</code> (non-negative) or <code>number_integer_t</code> (negative) is stored as such; any other valid number text (larger or smaller integers, fractions, and exponents such as <code>1e2</code>) is stored as <code>number_float</code>, so precision may be lost. Text that is not a valid <abbr title="JavaScript Object Notation">JSON</abbr> number raises <a href=../../../home/exceptions/#jsonexceptionparse_error115><code>parse_error.115</code></a>, a number outside the range of <code>number_float_t</code> (<code>1e999</code>) raises <a href=../../../home/exceptions/#jsonexceptionout_of_range406><code>out_of_range.406</code></a>.</p> </div> <div class="admonition success"> <p class=admonition-title>Complete mapping</p> <p>Every well-formed <abbr title="Universal Binary JSON">UBJSON</abbr> value can be converted to a <abbr title="JavaScript Object Notation">JSON</abbr> value, with these exceptions: a high-precision number outside the range of <code>number_float_t</code> (<a href=../../../home/exceptions/#jsonexceptionout_of_range406><code>out_of_range.406</code></a>), an optimized array of <code>Z</code>, <code>T</code>, or <code>F</code> with more than 1,048,576 elements, and a count that exceeds the capacity of the container (<a href=../../../home/exceptions/#jsonexceptionout_of_range408><code>out_of_range.408</code></a>). High-precision numbers that do not fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected with a parse error.</p> </div> <div class="admonition info"> <p class=admonition-title>Duplicate object keys</p> <p>If an object contains the same key more than once, the last value is kept (the <abbr title="Universal Binary JSON">UBJSON</abbr> specification does not address duplicate keys).</p> </div> <div class="admonition info"> <p class=admonition-title>Type marker of optimized containers</p> <p>The type marker after <code>$</code> must be one of <code>Z</code>, <code>N</code>, <code>T</code>, <code>F</code>, <code>i</code>, <code>U</code>, <code>I</code>, <code>l</code>, <code>L</code>, <code>d</code>, <code>D</code>, <code>H</code>, <code>C</code>, <code>S</code>, <code>[</code>, or <code>{</code>. It is checked even if the count is 0; any other marker raises <a href=../../../home/exceptions/#jsonexceptionparse_error112><code>parse_error.112</code></a>. A counted object typed <code>N</code> (<code>{$N#i1 ...</code>) is not supported.</p> </div> <div class="admonition warning"> <p class=admonition-title>Ill-formed <abbr title="Unicode Transformation Format">UTF</abbr>-8 in string values and object keys</p> <p><abbr title="Universal Binary JSON">UBJSON</abbr>'s required string encoding is <abbr title="Unicode Transformation Format">UTF</abbr>-8, but checking it on read is opt-in: with the <a href=../../../api/basic_json/from_ubjson/ ><code>error_handler</codLine truncated
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;iomanip&gt;</span>
<span class=cp>#include</span><span class=w> </span><span class=cpf>&lt;nlohmann/json.hpp&gt;</span>
<span class=k>using</span><span class=w> </span><span class=n>json</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=k>using</span><span class=w> </span><span class=n>nlohmann</span><span class=o>::</span><span class=n>json</span><span class=p>;</span>
<span class=kt>int</span><span class=w> </span><span class=nf>main</span><span class=p>()</span>
<span class=p>{</span>
@@ -93,10 +93,10 @@
<span class=w> </span><span class=n>json</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>=</span><span class=w> </span><span class=n>json</span><span class=o>::</span><span class=n>from_ubjson</span><span class=p>(</span><span class=n>v</span><span class=p>);</span>
<span class=w> </span><span class=c1>// print the deserialized JSON value</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>endl</span><span class=p>;</span>
<span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>cout</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>std</span><span class=o>::</span><span class=n>setw</span><span class=p>(</span><span class=mi>2</span><span class=p>)</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=n>j</span><span class=w> </span><span class=o>&lt;&lt;</span><span class=w> </span><span class=sc>'\n'</span><span class=p>;</span>
<span class=p>}</span>
</code></pre></div> <p>Output:</p> <div class=highlight><pre><span></span><code><span class=p>{</span>
<span class=w> </span><span class=nt>"compact"</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=p>,</span>
<span class=w> </span><span class=nt>"schema"</span><span class=p>:</span><span class=w> </span><span class=mi>0</span>
<span class=p>}</span>
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 24, 2026 15:57:44 UTC">October 24, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
</code></pre></div> </details> <!-- https://squidfunk.github.io/mkdocs-material/reference/tooltips/#adding-a-glossary --> <aside class=md-source-file> <span class=md-source-file__fact> <span class=md-icon title="Last update"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg> </span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="October 26, 2026 17:25:45 UTC">October 26, 2026</span> </span> </aside> </article> </div> <script>var tabs=__md_get("__tabs");if(Array.isArray(tabs))e:for(var set of document.querySelectorAll(".tabbed-set")){var labels=set.querySelector(".tabbed-labels");for(var tab of tabs)for(var label of labels.getElementsByTagName("label"))if(label.innerText.trim()===tab){var input=document.getElementById(label.htmlFor);input.checked=!0;continue e}}</script> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> <button type=button class="md-top md-icon" data-md-component=top hidden> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg> Back to top </button> </main> <footer class=md-footer> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-copyright> <div class=md-copyright__highlight> Copyright &copy; 2013-2026 Niels Lohmann </div> </div> <div class=md-social> <a href="https://github.com/nlohmann" target="_blank" rel="noopener" title="github.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 512 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M173.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M252.8 8C114.1 8 8 113.3 8 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C436.2 457.8 504 362.9 504 252 504 113.3 391.5 8 252.8 8M105.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg> </a> <a href="https://www.linkedin.com/in/nielslohmann/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 448 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zM102.2 96a38.5 38.5 0 1 1 0 77 38.5 38.5 0 1 1 0-77m282.1 320h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg> </a> <a href="https://www.xing.com/profile/Niels_Lohmann" target="_blank" rel="noopener" title="www.xing.com" class="md-social__link"> <svg xmlns=http://www.w3.org/2000/svg viewbox="0 0 384 512"><!-- Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M162.7 210c-1.8 3.3-25.2 44.4-70.1 123.5-4.9 8.3-10.8 12.5-17.7 12.5H9.8c-7.7 0-12.1-7.5-8.5-14.4l69-121.3q.3 0 0-.3l-43.9-75.6c-4.3-7.8.3-14.1 8.5-14.1H100c7.3 0 13.3 4.1 18 12.2zM382.6 46.1l-144 253v.3L330.2 466c3.9 7.1.2 14.1-8.5 14.1h-65.2c-7.6 0-13.6-4-18-12.2l-92.4-168.5c3.3-5.8 51.5-90.8 14Line truncated
features/binary_formats/ubjson/index.md
+183 -166
View File
@@ -4,154 +4,11 @@ Universal Binary JSON (UBJSON) is a binary form directly imitating JSON, but req
References
- [UBJSON Website](http://ubjson.org)
- [UBJSON Website](https://ubjson.org)
## Serialization
## Byte layout
The library uses the following mapping from JSON values types to UBJSON types according to the UBJSON specification:
| JSON value type | value/range | UBJSON type | marker |
| --------------- | ----------------------------------------- | -------------- | ------ |
| null | `null` | null | `Z` |
| boolean | `true` | true | `T` |
| boolean | `false` | false | `F` |
| number_integer | -9223372036854775808..-2147483649 | int64 | `L` |
| number_integer | -2147483648..-32769 | int32 | `l` |
| number_integer | -32768..-129 | int16 | `I` |
| number_integer | -128..127 | int8 | `i` |
| number_integer | 128..255 | uint8 | `U` |
| number_integer | 256..32767 | int16 | `I` |
| number_integer | 32768..2147483647 | int32 | `l` |
| number_integer | 2147483648..9223372036854775807 | int64 | `L` |
| number_unsigned | 0..127 | int8 | `i` |
| number_unsigned | 128..255 | uint8 | `U` |
| number_unsigned | 256..32767 | int16 | `I` |
| number_unsigned | 32768..2147483647 | int32 | `l` |
| number_unsigned | 2147483648..9223372036854775807 | int64 | `L` |
| number_unsigned | 9223372036854775808..18446744073709551615 | high-precision | `H` |
| number_float | *any value* | float64 | `D` |
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format* | array | `[` |
| object | *see notes on optimized format* | map | `{` |
Complete mapping
The mapping is **complete** in the sense that any JSON value type can be converted to a UBJSON value.
Any UBJSON output created by `to_ubjson` can be successfully parsed by `from_ubjson`.
Size constraints
The following values can **not** be converted to a UBJSON value:
- strings with more than 9223372036854775807 bytes (theoretical)
UTF-8 validation of string values and object keys
UBJSON's required string encoding is UTF-8. By default (the [`error_handler`](https://json.nlohmann.me/api/basic_json/to_ubjson/index.md) parameter left at `keep`), `to_ubjson()` writes the bytes of string values and object keys unchanged, even if they are not valid UTF-8. With `error_handler_t::strict`, it throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for ill-formed UTF-8 instead; `replace`/`ignore` sanitize the string. [`JSON_STRICT_BINARY_UTF8`](https://json.nlohmann.me/api/macros/json_strict_binary_utf8/index.md) makes `strict` the default.
Unused UBJSON markers
The following markers are not used in the conversion:
- `Z`: no-op values are not created.
- `C`: single-byte strings are serialized with `S` markers.
NaN/infinity handling
If NaN or Infinity are stored inside a JSON number, they are serialized properly. This behavior differs from the [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md) function which serializes NaN or Infinity to `null`.
Optimized formats
The optimized formats for containers are supported: Parameter `use_size` adds size information to the beginning of a container and removes the closing marker. Parameter `use_type` further checks whether all elements of a container have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be used together with `use_size = true`.
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the receiving side is immediately informed on the number of elements of the container.
An array whose type marker is `Z` (null), `T` (true) or `F` (false) stores no payload at all, because the marker already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with [`out_of_range.408`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range408) when the count exceeds 1,048,576 (`1 << 20`), and `to_ubjson` writes longer arrays of these types without the annotation, so any value it produces can be read back.
Binary values
If the JSON data contains the binary type, the value stored is a list of integers, as suggested by the UBJSON documentation. In particular, this means that serialization and the deserialization of a JSON containing binary values into UBJSON and back will result in a different JSON object.
Example: serialize JSON values to UBJSON, with and without size/type optimization
```
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using namespace nlohmann::literals;
// function to print UBJSON's diagnostic format
void print_byte(uint8_t byte)
{
if (32 < byte and byte < 128)
{
std::cout << (char)byte;
}
else
{
std::cout << (int)byte;
}
}
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": false})"_json;
// serialize it to UBJSON
std::vector<std::uint8_t> v = json::to_ubjson(j);
// print the vector content
for (auto& byte : v)
{
print_byte(byte);
}
std::cout << std::endl;
// create an array of numbers
json array = {1, 2, 3, 4, 5, 6, 7, 8};
// serialize it to UBJSON using default representation
std::vector<std::uint8_t> v_array = json::to_ubjson(array);
// serialize it to UBJSON using size optimization
std::vector<std::uint8_t> v_array_size = json::to_ubjson(array, true);
// serialize it to UBJSON using type optimization
std::vector<std::uint8_t> v_array_size_and_type = json::to_ubjson(array, true, true);
// print the vector contents
for (auto& byte : v_array)
{
print_byte(byte);
}
std::cout << std::endl;
for (auto& byte : v_array_size)
{
print_byte(byte);
}
std::cout << std::endl;
for (auto& byte : v_array_size_and_type)
{
print_byte(byte);
}
std::cout << std::endl;
}
```
Output:
```
{i7compactTi6schemaF}
[i1i2i3i4i5i6i7i8]
[#i8i1i2i3i4i5i6i7i8
[$i#i812345678
```
### Byte layout
UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares a count (`#`) after the opening marker, optionally preceded by an element type (`$`); an element type is only allowed together with a count. An optimized container saves the closing marker and, with a type, the per-element markers.
Consider the following JSON value (shown formatted for readability):
@@ -181,31 +38,191 @@ Consider the following JSON value (shown formatted for readability):
| `]` | 5D | array end (`]`) |
| `}` | 7D | object end (`}`) |
## Serialization
The library uses the following mapping from JSON values types to UBJSON types according to the UBJSON specification:
| JSON value type | value/range | UBJSON type | marker |
| --------------- | ----------------------------------------- | ------------------------------------------------ | --------- |
| null | `null` | null | `Z` |
| boolean | `true` | true | `T` |
| boolean | `false` | false | `F` |
| number_integer | -9223372036854775808..-2147483649 | int64 | `L` |
| number_integer | -2147483648..-32769 | int32 | `l` |
| number_integer | -32768..-129 | int16 | `I` |
| number_integer | -128..127 | int8 | `i` |
| number_integer | 128..255 | uint8 | `U` |
| number_integer | 256..32767 | int16 | `I` |
| number_integer | 32768..2147483647 | int32 | `l` |
| number_integer | 2147483648..9223372036854775807 | int64 | `L` |
| number_unsigned | 0..127 | int8 | `i` |
| number_unsigned | 128..255 | uint8 | `U` |
| number_unsigned | 256..32767 | int16 | `I` |
| number_unsigned | 32768..2147483647 | int32 | `l` |
| number_unsigned | 2147483648..9223372036854775807 | int64 | `L` |
| number_unsigned | 9223372036854775808..18446744073709551615 | high-precision | `H` |
| number_float | *any value* | float64 (float32 if `number_float_t` is `float`) | `D` (`d`) |
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format* | array | `[` |
| object | *see notes on optimized format* | object | `{` |
Complete mapping
The mapping is **complete** in the sense that any JSON value type can be converted to a UBJSON value.
Any UBJSON output created by `to_ubjson` can be successfully parsed by `from_ubjson`.
Size constraints
The following values can **not** be converted to a UBJSON value:
- strings with more than 9223372036854775807 bytes (theoretical)
UTF-8 validation of string values and object keys
UBJSON's required string encoding is UTF-8. By default (the [`error_handler`](https://json.nlohmann.me/api/basic_json/to_ubjson/index.md) parameter left at `keep`), `to_ubjson()` writes the bytes of string values and object keys unchanged, even if they are not valid UTF-8. With `error_handler_t::strict`, it throws [`type_error.316`](https://json.nlohmann.me/home/exceptions/#jsonexceptiontype_error316) for ill-formed UTF-8 instead; `replace`/`ignore` sanitize the string. [`JSON_STRICT_BINARY_UTF8`](https://json.nlohmann.me/api/macros/json_strict_binary_utf8/index.md) makes `strict` the default.
Unused UBJSON markers
The following markers are not used in the conversion:
- `N`: no-op values are never written (they are skipped when reading).
- `C`: single-byte strings are serialized with `S` markers.
NaN/infinity handling
NaN and infinity are written as IEEE 754 float64 values (`D`) and read back unchanged. This differs from [`dump()`](https://json.nlohmann.me/api/basic_json/dump/index.md), which serializes them to `null`, and from the UBJSON specification, which encodes infinity as `null` (it does not mention NaN); other UBJSON readers may therefore not accept these values.
Optimized formats
The optimized formats for containers are supported: Parameter `use_size` adds size information to the beginning of a container and removes the closing marker. Parameter `use_type` further checks whether all elements of a container have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be used together with `use_size = true`: combining `use_type` with `use_size = false` throws [`other_error.502`](https://json.nlohmann.me/home/exceptions/#jsonexceptionother_error502) if the value contains a non-empty array, object, or binary value.
Two elements have the same type if they are written with the same type marker. For example, `[1, 2]` is written with the type `i`, but `[1, 200]` (`i`, `U`), `[true, false]` (`T`, `F`), and `[1, 2.5]` (`i`, `D`) are not; an empty container is never given a `$` annotation of its own (an array of empty arrays is still typed `[`).
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the receiving side is immediately informed on the number of elements of the container.
An array whose type marker is `Z` (null), `T` (true) or `F` (false) stores no payload at all, because the marker already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with [`out_of_range.408`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range408) when the count exceeds 1,048,576 (`1 << 20`), and `to_ubjson` writes longer arrays of these types in the counted but untyped form: the `$` type marker is dropped, the `#` count is kept, and every element is written with its own marker (`[#l 1048577 Z Z ...`), so any value it produces can be read back. Objects and untyped counted arrays are not subject to this limit.
Binary values
If the JSON data contains the binary type, the value stored is a list of integers, as suggested by the UBJSON documentation. In particular, this means that serialization and the deserialization of a JSON containing binary values into UBJSON and back will result in a different JSON object.
Example: serialize JSON values to UBJSON, with and without size/type optimization
```
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using nlohmann::json;
using namespace nlohmann::literals;
// function to print UBJSON's diagnostic format
void print_byte(uint8_t byte)
{
if (32 < byte and byte < 128)
{
std::cout << static_cast<char>(byte);
}
else
{
std::cout << static_cast<int>(byte);
}
}
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": false})"_json;
// serialize it to UBJSON
std::vector<std::uint8_t> v = json::to_ubjson(j);
// print the vector content
for (auto& byte : v)
{
print_byte(byte);
}
std::cout << '\n';
// create an array of numbers
json array = {1, 2, 3, 4, 5, 6, 7, 8};
// serialize it to UBJSON using default representation
std::vector<std::uint8_t> v_array = json::to_ubjson(array);
// serialize it to UBJSON using size optimization
std::vector<std::uint8_t> v_array_size = json::to_ubjson(array, true);
// serialize it to UBJSON using type optimization
std::vector<std::uint8_t> v_array_size_and_type = json::to_ubjson(array, true, true);
// print the vector contents
for (auto& byte : v_array)
{
print_byte(byte);
}
std::cout << '\n';
for (auto& byte : v_array_size)
{
print_byte(byte);
}
std::cout << '\n';
for (auto& byte : v_array_size_and_type)
{
print_byte(byte);
}
std::cout << '\n';
}
```
Output:
```
{i7compactTi6schemaF}
[i1i2i3i4i5i6i7i8]
[#i8i1i2i3i4i5i6i7i8
[$i#i812345678
```
## Deserialization
The library maps UBJSON types to JSON value types as follows:
| UBJSON type | JSON value type | marker |
| ----------- | --------------------------------------- | ------ |
| no-op | *no value, next value is read* | `N` |
| null | `null` | `Z` |
| false | `false` | `F` |
| true | `true` | `T` |
| float32 | number_float | `d` |
| float64 | number_float | `D` |
| uint8 | number_unsigned | `U` |
| int8 | number_integer | `i` |
| int16 | number_integer | `I` |
| int32 | number_integer | `l` |
| int64 | number_integer | `L` |
| string | string | `S` |
| char | string | `C` |
| array | array (optimized values are supported) | `[` |
| object | object (optimized values are supported) | `{` |
| UBJSON type | JSON value type | marker |
| -------------- | ------------------------------------------------ | ------ |
| no-op | *no value, next value is read* | `N` |
| null | `null` | `Z` |
| false | `false` | `F` |
| true | `true` | `T` |
| float32 | number_float | `d` |
| float64 | number_float | `D` |
| uint8 | number_unsigned | `U` |
| int8 | number_integer | `i` |
| int16 | number_integer | `I` |
| int32 | number_integer | `l` |
| int64 | number_integer | `L` |
| string | string | `S` |
| char | string | `C` |
| high-precision | number_unsigned, number_integer, or number_float | `H` |
| array | array (optimized values are supported) | `[` |
| object | object (optimized values are supported) | `{` |
High-precision numbers
The payload of `H` is a length-prefixed string that is parsed as a JSON number (surrounding whitespace is ignored). Integer text in the range of `number_unsigned_t` (non-negative) or `number_integer_t` (negative) is stored as such; any other valid number text (larger or smaller integers, fractions, and exponents such as `1e2`) is stored as `number_float`, so precision may be lost. Text that is not a valid JSON number raises [`parse_error.115`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error115), a number outside the range of `number_float_t` (`1e999`) raises [`out_of_range.406`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range406).
Complete mapping
The mapping is **complete** in the sense that any UBJSON value can be converted to a JSON value.
Every well-formed UBJSON value can be converted to a JSON value, with these exceptions: a high-precision number outside the range of `number_float_t` ([`out_of_range.406`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range406)), an optimized array of `Z`, `T`, or `F` with more than 1,048,576 elements, and a count that exceeds the capacity of the container ([`out_of_range.408`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range408)). High-precision numbers that do not fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected with a parse error.
Duplicate object keys
If an object contains the same key more than once, the last value is kept (the UBJSON specification does not address duplicate keys).
Type marker of optimized containers
The type marker after `$` must be one of `Z`, `N`, `T`, `F`, `i`, `U`, `I`, `l`, `L`, `d`, `D`, `H`, `C`, `S`, `[`, or `{`. It is checked even if the count is 0; any other marker raises [`parse_error.112`](https://json.nlohmann.me/home/exceptions/#jsonexceptionparse_error112). A counted object typed `N` (`{$N#i1 ...`) is not supported.
Ill-formed UTF-8 in string values and object keys
@@ -218,7 +235,7 @@ Example: deserialize a JSON value from UBJSON
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -232,7 +249,7 @@ int main()
json j = json::from_ubjson(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```