diff --git a/docs/mkdocs/docs/api/basic_json/from_bson.md b/docs/mkdocs/docs/api/basic_json/from_bson.md index 6e0beafce..c04933eef 100644 --- a/docs/mkdocs/docs/api/basic_json/from_bson.md +++ b/docs/mkdocs/docs/api/basic_json/from_bson.md @@ -80,7 +80,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va - Throws [`parse_error.110`](../../home/exceptions.md#jsonexceptionparse_error110) if the given input ends prematurely or the end of the input was not reached when `strict` was set to true - Throws [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) if a parse error occurs (e.g., an - invalid string or byte array length) + invalid string or byte array length, an invalid inner length of a binary value with subtype `0x02`, or a document size + that does not match the content; `strict` does not affect the document size checks) - Throws [`parse_error.114`](../../home/exceptions.md#jsonexceptionparse_error114) if an unsupported BSON record type is encountered - Throws [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) if a string value or object key is diff --git a/docs/mkdocs/docs/api/basic_json/to_bson.md b/docs/mkdocs/docs/api/basic_json/to_bson.md index 315477e19..b3b346266 100644 --- a/docs/mkdocs/docs/api/basic_json/to_bson.md +++ b/docs/mkdocs/docs/api/basic_json/to_bson.md @@ -13,7 +13,7 @@ static void to_bson(const basic_json& j, detail::output_adapter o, ``` BSON (Binary JSON) is a binary format in which zero or more ordered key/value pairs are stored as a single entity (a -so-called document). +so-called document). The members of an object are written in the order in which iteration visits them. 1. Returns a byte vector containing the BSON serialization. 2. Writes the BSON serialization to an output adapter. diff --git a/docs/mkdocs/docs/api/byte_container_with_subtype/index.md b/docs/mkdocs/docs/api/byte_container_with_subtype/index.md index b184d8496..2999d732b 100644 --- a/docs/mkdocs/docs/api/byte_container_with_subtype/index.md +++ b/docs/mkdocs/docs/api/byte_container_with_subtype/index.md @@ -6,7 +6,7 @@ class byte_container_with_subtype : public BinaryType; ``` This type extends the template parameter `BinaryType` provided to [`basic_json`](../basic_json/index.md) with a subtype -used by BSON and MessagePack. This type exists so that the user does not have to specify a type themselves with a +used by BSON, CBOR and MessagePack. This type exists so that the user does not have to specify a type themselves with a specific naming scheme in order to override the binary type. ## Template parameters diff --git a/docs/mkdocs/docs/features/binary_formats/bson.md b/docs/mkdocs/docs/features/binary_formats/bson.md index 9c82594dd..cb763216a 100644 --- a/docs/mkdocs/docs/features/binary_formats/bson.md +++ b/docs/mkdocs/docs/features/binary_formats/bson.md @@ -66,7 +66,12 @@ The library uses the following mapping from JSON values types to BSON types: !!! warning "Incomplete mapping" The mapping is **incomplete**, since only JSON-objects (and things contained therein) can be serialized to BSON. - Also, keys may not contain U+0000, since they are serialized a zero-terminated c-strings. + Also, keys may not contain U+0000, since they are serialized as zero-terminated C strings. + +!!! info "Order of object members" + + The members of an object are written in the order in which iteration visits them: sorted by key for + [`json`](../../api/json.md), in insertion order for [`ordered_json`](../../api/ordered_json.md). !!! warning "BSON type 0x11 interoperability" @@ -81,6 +86,18 @@ The library uses the following mapping from JSON values types to BSON types: As a result, serializing and deserializing a JSON object containing such a value produces a different JSON object, even though the binary data is unchanged. +!!! info "Binary subtype 0x02 (old binary)" + + The BSON specification defines the payload of the "binary (old)" subtype `0x02` as an int32 followed by that many + bytes, so the value is preceded by two lengths: the length of the binary element and, as the first four bytes of + the payload, the number of bytes that follow. A binary value with subtype `2` is therefore written with the outer + length 4 bytes larger than its size and the size as inner length: `binary({1, 2, 3}, 2)` becomes + `07 00 00 00 02 03 00 00 00 01 02 03`. When reading, the inner length is checked and removed, so the value + contains only the bytes that follow it. `from_bson()` throws + [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) if the outer length is less than 4 or the + inner length is not the outer length minus 4. Versions before 3.13.0 did not write the inner length and returned + it as part of the payload. + ??? example "Example: serialize a JSON value to BSON" ```cpp @@ -135,12 +152,28 @@ The library maps BSON record types to JSON value types as follows: The BSON reader is lenient in a few areas where the BSON specification is more restrictive: - array element keys are not checked against the required decimal sequence (`0`, `1`, `2`, ...), - - any non-zero byte is accepted as `true` for the boolean type, and - - the payload for binary subtype `0x02` is returned as-is, including its inner length prefix. + - any non-zero byte is accepted as `true` for the boolean type, + - the payloads of the binary subtypes `0x03` (UUID, old), `0x04` (UUID), and `0x05` (MD5) are not checked to have + 16 bytes, + - duplicate keys within a document are accepted; the last value wins (`{"a": 1, "a": 2}` is read as `{"a": 2}`), + while a SAX handler receives every occurrence, + - a `string` value may contain embedded `0x00` bytes, and + - with `strict` set to `false`, bytes after the end of the document are ignored. If BSON input must be validated for strict specification compliance, validate it separately before passing it to `from_bson()`. +!!! note "Document size" + + Every document starts with its size. `from_bson()` rejects a document with + [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) if the size is less than 5 (the size prefix and the terminator), if an embedded document does not end before the + end of the document that contains it, if an element does not fit into the declared size, or if the size does not + match the number of bytes read when the terminating `0x00` byte is reached. A size that is too small is detected + before the element that exceeds it is read; a size that is too large only at the terminator, so a SAX handler has + received the content of such a document by then. Passing `strict` as `false` skips only the check for bytes after + the end of the document ([`parse_error.110`](../../home/exceptions.md#jsonexceptionparse_error110)), not the + document size checks. + !!! warning "Ill-formed UTF-8 in string values" The BSON specification requires `string` values (type `0x02`) to be valid UTF-8, but this is not required of a @@ -154,8 +187,9 @@ The library maps BSON record types to JSON value types as follows: ill-formed bytes. `to_bson()`'s own `error_handler` parameter defaults to `keep`, so such a string value or element (key) name is written unchanged; with `strict` (the default if [`JSON_STRICT_BINARY_UTF8`](../../api/macros/json_strict_binary_utf8.md) is enabled), it throws the same exception - instead. Element (key) names are never validated on read, since they are read byte-by-byte as a C string. `binary` - values (type `0x05`) are unaffected, since they are not required to hold text. + instead. Element (key) names are treated like string values: with `keep` they are read unchanged, with `strict`, + `replace`, or `ignore` they are checked and handled in the same way. `binary` values (type `0x05`) are unaffected, + since they are not required to hold text. ??? example "Example: deserialize a JSON value from BSON" diff --git a/docs/mkdocs/docs/features/binary_values.md b/docs/mkdocs/docs/features/binary_values.md index f5a87472b..e4dee45d4 100644 --- a/docs/mkdocs/docs/features/binary_values.md +++ b/docs/mkdocs/docs/features/binary_values.md @@ -227,7 +227,10 @@ array of integers. ### BSON [BSON](binary_formats/bson.md) supports binary values and subtypes. If a subtype is given, it is used and added as an -unsigned 8-bit integer. If no subtype is given, the generic binary subtype 0x00 is used. +unsigned 8-bit integer; a subtype above 255 throws +[`out_of_range.415`](../home/exceptions.md#jsonexceptionout_of_range415). If no subtype is given, the generic binary +subtype 0x00 is used. The payload of the "old binary" subtype 0x02 is preceded by a second length, which the library +writes and removes again when reading; it is not part of the payload. ??? example "Example: serialize a binary value to BSON" diff --git a/docs/mkdocs/docs/home/exceptions.md b/docs/mkdocs/docs/home/exceptions.md index 59edd944c..d9aea4420 100644 --- a/docs/mkdocs/docs/home/exceptions.md +++ b/docs/mkdocs/docs/home/exceptions.md @@ -906,13 +906,19 @@ value, or by default-constructing a [`basic_json`](../api/basic_json/index.md) w [`value_t::discarded`](../api/basic_json/value_t.md)) was passed to a binary serialization function, either directly or nested in an array or object. There is no way to represent a discarded value in CBOR, MessagePack, UBJSON, BJData, BSON, or BON8. -!!! failure "Example message" +!!! failure "Example messages" Serializing `#!json [1, 2]` to CBOR, where the second element was discarded by a parser callback: ``` [json.exception.type_error.321] cannot serialize discarded value to CBOR ``` + Serializing an object with a discarded member to BSON (a discarded value at the top level is reported as + [`type_error.317`](#jsonexceptiontype_error317) instead, since BSON requires an object there): + ``` + [json.exception.type_error.321] cannot serialize discarded value to BSON + ``` + ## Out of range This exception is thrown in case a library function is called on an input parameter that exceeds the expected range, for instance, in the case of array indices or nonexisting object keys. @@ -1155,12 +1161,16 @@ A JSON Patch `move` operation's `"from"` location is a proper prefix of its `"pa MessagePack's ext type and BSON's binary subtype are each stored in a single byte. This exception is thrown when serializing a [`byte_container_with_subtype`](../api/byte_container_with_subtype/index.md) whose subtype exceeds 255, the largest value the library can store in that byte. -!!! failure "Example message" +!!! failure "Example messages" ``` [json.exception.out_of_range.415] subtype 70000 is too large for the MessagePack ext type (max 255) ``` + ``` + [json.exception.out_of_range.415] subtype 70000 is too large for the BSON binary subtype (max 255) + ``` + !!! note This exception was added in version 3.13.0. Before that, subtypes above 255 were silently truncated modulo 256 instead of raising an error.