wt/json
1
0 Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-27 12:25:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

deploy: 94fe31eb9b

Browse Source
This commit is contained in:
nlohmann committed 2026-10-26 21:24:35 +00:00
1 parent 41c601a7d1
commit d2e264bd84
1400 files changed
+11859 -5440
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

features/binary_formats/bson.md
+117 -84
View File
@@ -10,85 +10,9 @@ representation of data types that are not part of the JSON spec. For example, BS
- [BSON Specification](http://bsonspec.org/spec.html) - the specification
## Serialization
## Byte layout
The library uses the following mapping from JSON values types to BSON types:
| JSON value type | value/range | BSON type | marker |
|-----------------|-------------------------------------------|-----------|--------|
| null | `null` | null | 0x0A |
| boolean | `true`, `false` | boolean | 0x08 |
| number_integer | -9223372036854775808..-2147483649 | int64 | 0x12 |
| number_integer | -2147483648..2147483647 | int32 | 0x10 |
| number_integer | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 0..2147483647 | int32 | 0x10 |
| number_unsigned | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | 0x11 |
| number_float | *any value* | double | 0x01 |
| string | *any value* | string | 0x02 |
| array | *any value* | document | 0x04 |
| object | *any value* | document | 0x03 |
| binary | *any value* | binary | 0x05 |
!!! warning "Incomplete mapping"
The mapping is **incomplete**, since only JSON-objects (and things contained therein) can be serialized to BSON.
Also, keys may not contain U+0000, since they are serialized a zero-terminated c-strings.
!!! warning "BSON type 0x11 interoperability"
The BSON specification defines type `0x11` as a Timestamp. This library uses marker `0x11` when serializing
`number_unsigned` values in the range `9223372036854775808..18446744073709551615`. Other BSON implementations may
therefore interpret these values as Timestamps instead of unsigned integers.
!!! info "Binary values without a subtype"
BSON requires every binary value to have a subtype. If a binary value has no subtype, this library serializes it
with the generic subtype `0x00`. After deserialization, `has_subtype()` returns `true` and `subtype()` returns `0`.
As a result, serializing and deserializing a JSON object containing such a value produces a different JSON object,
even though the binary data is unchanged.
??? example "Example: serialize a JSON value to BSON"
```cpp
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using namespace nlohmann::literals;
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": 0})"_json;
// serialize it to BSON
std::vector<std::uint8_t> v = json::to_bson(j);
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
}
std::cout << std::endl;
}
```
Output:
```c
0x1b 0x00 0x00 0x00 0x08 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0x00 0x01 0x10 0x73 0x63 0x68 0x65 0x6d 0x61 0x00 0x00 0x00 0x00 0x00 0x00
```
### Byte layout
A BSON document is a length-prefixed sequence of *elements*. The first four bytes hold the total size of the document (little-endian int32), followed by the elements, and a final `0x00` byte ends the document. Each element starts with a one-byte type, followed by its key as a NUL-terminated string and the value, whose encoding depends on the type. Arrays are documents whose keys are the decimal indices `"0"`, `"1"`, ...; nesting is expressed by embedding documents. Because every document announces its size, a reader can skip over it without parsing its content. The top level of a BSON value is always a document, which is why only JSON objects can be serialized.
Consider the following JSON value (shown formatted for readability):
@@ -119,6 +43,97 @@ Consider the following JSON value (shown formatted for readability):
<tr><td><code>}</code></td><td><span class="bl bl-t">00</span></td><td>end of the document</td></tr>
</tbody></table></div>
## Serialization
The library uses the following mapping from JSON values types to BSON types:
| JSON value type | value/range | BSON type | marker |
|-----------------|-------------------------------------------|-----------|--------|
| null | `null` | null | 0x0A |
| boolean | `true`, `false` | boolean | 0x08 |
| number_integer | -9223372036854775808..-2147483649 | int64 | 0x12 |
| number_integer | -2147483648..2147483647 | int32 | 0x10 |
| number_integer | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 0..2147483647 | int32 | 0x10 |
| number_unsigned | 2147483648..9223372036854775807 | int64 | 0x12 |
| number_unsigned | 9223372036854775808..18446744073709551615 | uint64 | 0x11 |
| number_float | *any value* | double | 0x01 |
| string | *any value* | string | 0x02 |
| array | *any value* | document | 0x04 |
| object | *any value* | document | 0x03 |
| binary | *any value* | binary | 0x05 |
!!! warning "Incomplete mapping"
The mapping is **incomplete**, since only JSON-objects (and things contained therein) can be serialized to BSON.
Also, keys may not contain U+0000, since they are serialized as zero-terminated C strings.
!!! info "Order of object members"
The members of an object are written in the order in which iteration visits them: sorted by key for
[`json`](../../api/json.md), in insertion order for [`ordered_json`](../../api/ordered_json.md).
!!! warning "BSON type 0x11 interoperability"
The BSON specification defines type `0x11` as a Timestamp. This library uses marker `0x11` when serializing
`number_unsigned` values in the range `9223372036854775808..18446744073709551615`. Other BSON implementations may
therefore interpret these values as Timestamps instead of unsigned integers.
!!! info "Binary values without a subtype"
BSON requires every binary value to have a subtype. If a binary value has no subtype, this library serializes it
with the generic subtype `0x00`. After deserialization, `has_subtype()` returns `true` and `subtype()` returns `0`.
As a result, serializing and deserializing a JSON object containing such a value produces a different JSON object,
even though the binary data is unchanged.
!!! info "Binary subtype 0x02 (old binary)"
The BSON specification defines the payload of the "binary (old)" subtype `0x02` as an int32 followed by that many
bytes, so the value is preceded by two lengths: the length of the binary element and, as the first four bytes of
the payload, the number of bytes that follow. A binary value with subtype `2` is therefore written with the outer
length 4 bytes larger than its size and the size as inner length: `binary({1, 2, 3}, 2)` becomes
`07 00 00 00 02 03 00 00 00 01 02 03`. When reading, the inner length is checked and removed, so the value
contains only the bytes that follow it. `from_bson()` throws
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) if the outer length is less than 4 or the
inner length is not the outer length minus 4. Versions before 3.13.0 did not write the inner length and returned
it as part of the payload; BSON data written by those versions with subtype 2 is not readable any more and must be
written again.
??? example "Example: serialize a JSON value to BSON"
```cpp
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using nlohmann::json;
using namespace nlohmann::literals;
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": 0})"_json;
// serialize it to BSON
std::vector<std::uint8_t> v = json::to_bson(j);
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << '\n';
}
```
Output:
```c
0x1b 0x00 0x00 0x00 0x08 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0x00 0x01 0x10 0x73 0x63 0x68 0x65 0x6d 0x61 0x00 0x00 0x00 0x00 0x00 0x00
```
## Deserialization
The library maps BSON record types to JSON value types as follows:
@@ -161,12 +176,29 @@ The library maps BSON record types to JSON value types as follows:
The BSON reader is lenient in a few areas where the BSON specification is more restrictive:
- array element keys are not checked against the required decimal sequence (`0`, `1`, `2`, ...),
- any non-zero byte is accepted as `true` for the boolean type, and
- the payload for binary subtype `0x02` is returned as-is, including its inner length prefix.
- any non-zero byte is accepted as `true` for the boolean type,
- the payloads of the binary subtypes `0x03` (UUID, old), `0x04` (UUID), and `0x05` (MD5) are not checked to have
16 bytes,
- duplicate keys within a document are accepted; the last value wins (`{"a": 1, "a": 2}` is read as `{"a": 2}`),
while a SAX handler receives every occurrence,
- a `string` value may contain embedded `0x00` bytes, and
- with `strict` set to `false`, bytes after the end of the document are ignored.
If BSON input must be validated for strict specification compliance, validate it separately before passing it to
`from_bson()`.
!!! note "Document size"
Every document starts with its size. `from_bson()` rejects a document with
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) if the size is less than 5 (the size prefix and the terminator), if an embedded document does not end before the
end of the document that contains it, if the elements do not fit into the declared size, or if the size does not
match the number of bytes read when the terminating `0x00` byte is reached. A size that is too small is detected
before an element is read whose type byte and name no longer fit; an element that starts inside the declared size
but extends past it is read, and reported to a SAX handler, before the mismatch is found at the terminator. A size
that is too large is likewise found only at the terminator. Passing `strict` as `false` skips only the check for bytes after
the end of the document ([`parse_error.110`](../../home/exceptions.md#jsonexceptionparse_error110)), not the
document size checks.
!!! warning "Ill-formed UTF-8 in string values"
The BSON specification requires `string` values (type `0x02`) to be valid UTF-8, but this is not required of a
@@ -180,8 +212,9 @@ The library maps BSON record types to JSON value types as follows:
ill-formed bytes. `to_bson()`'s own `error_handler` parameter defaults to `keep`, so such a string value or element
(key) name is written unchanged; with `strict` (the default if
[`JSON_STRICT_BINARY_UTF8`](../../api/macros/json_strict_binary_utf8.md) is enabled), it throws the same exception
instead. Element (key) names are never validated on read, since they are read byte-by-byte as a C string. `binary`
values (type `0x05`) are unaffected, since they are not required to hold text.
instead. Element (key) names are treated like string values: with `keep` they are read unchanged, with `strict`,
`replace`, or `ignore` they are checked and handled in the same way. `binary` values (type `0x05`) are unaffected,
since they are not required to hold text.
??? example "Example: deserialize a JSON value from BSON"
@@ -190,7 +223,7 @@ The library maps BSON record types to JSON value types as follows:
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using nlohmann::json;
int main()
{
@@ -205,7 +238,7 @@ The library maps BSON record types to JSON value types as follows:
json j = json::from_bson(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
std::cout << std::setw(2) << j << '\n';
}
```