CBOR¶
The Concise Binary Object Representation (CBOR) is a data format whose design goals include the possibility of extremely small code sizes, fairly small message size, and extensibility without the need for version negotiation.
References
- CBOR Website - the main source on CBOR
- CBOR Playground - an interactive webpage to translate between JSON and CBOR
- RFC 8949 - the CBOR specification
Byte layout¶
CBOR encodes every data item as a head followed by optional content. The first byte of the head holds the major type in its upper three bits (unsigned integer, negative integer, byte string, text string, array, map, tag, or simple/float) and an additional information value in its lower five bits. Additional information below 24 is the argument itself (for example, a small integer or a short length); the values 24 to 27 announce that the argument follows in 1, 2, 4, or 8 big-endian bytes. Strings, arrays, and maps are prefixed with their length (or count of elements/pairs), and content follows directly. A reader of definite-length items knows how many items to expect without a terminator; indefinite-length items (additional information 31) are instead closed by the "break" byte 0xFF, which from_cbor reads but to_cbor never writes.
Consider the following JSON value (shown formatted for readability):
{
"id": 300,
"name": "json",
"pi": 3.14,
"tags": [
"a",
"b"
]
}
to_cbor encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: type/marker data.
| JSON | Bytes | Meaning |
|---|---|---|
{ | A4 | map with 4 key/value pairs |
"id": 300, | 62 69 64 19 01 2C | key: text string of length 2, then its UTF-8 bytes; value: unsigned integer, 2 bytes follow (big-endian) |
"name": "json", | 64 6E 61 6D 65 64 6A 73 6F 6E | key: text string of length 4; value: text string of length 4 |
"pi": 3.14, | 62 70 69 FB 40 09 1E B8 51 EB 85 1F | key: text string of length 2; value: double-precision float (IEEE 754, big-endian) |
"tags": [ | 64 74 61 67 73 82 | key: text string of length 4; value: array with 2 elements |
"a", | 61 61 | text string of length 1 |
"b" | 61 62 | text string of length 1 |
] | no end marker: the element count was given up front | |
} | no end marker: the member count was given up front |
Serialization¶
The library uses the following mapping from JSON values types to CBOR types according to the CBOR specification (RFC 8949):
| JSON value type | value/range | CBOR type | first byte |
|---|---|---|---|
| null | null | Null | 0xF6 |
| boolean | true | True | 0xF5 |
| boolean | false | False | 0xF4 |
| number_integer | -9223372036854775808..-4294967297 | Negative integer (8 bytes follow) | 0x3B |
| number_integer | -4294967296..-65537 | Negative integer (4 bytes follow) | 0x3A |
| number_integer | -65536..-257 | Negative integer (2 bytes follow) | 0x39 |
| number_integer | -256..-25 | Negative integer (1 byte follow) | 0x38 |
| number_integer | -24..-1 | Negative integer | 0x20..0x37 |
| number_integer | 0..23 | Integer | 0x00..0x17 |
| number_integer | 24..255 | Unsigned integer (1 byte follow) | 0x18 |
| number_integer | 256..65535 | Unsigned integer (2 bytes follow) | 0x19 |
| number_integer | 65536..4294967295 | Unsigned integer (4 bytes follow) | 0x1A |
| number_integer | 4294967296..9223372036854775807 | Unsigned integer (8 bytes follow) | 0x1B |
| number_unsigned | 0..23 | Integer | 0x00..0x17 |
| number_unsigned | 24..255 | Unsigned integer (1 byte follow) | 0x18 |
| number_unsigned | 256..65535 | Unsigned integer (2 bytes follow) | 0x19 |
| number_unsigned | 65536..4294967295 | Unsigned integer (4 bytes follow) | 0x1A |
| number_unsigned | 4294967296..18446744073709551615 | Unsigned integer (8 bytes follow) | 0x1B |
| number_float | any value representable by a float | Single-Precision Float | 0xFA |
| number_float | any value NOT representable by a float | Double-Precision Float | 0xFB |
| string | length: 0..23 | UTF-8 string | 0x60..0x77 |
| string | length: 24..255 | UTF-8 string (1 byte follow) | 0x78 |
| string | length: 256..65535 | UTF-8 string (2 bytes follow) | 0x79 |
| string | length: 65536..4294967295 | UTF-8 string (4 bytes follow) | 0x7A |
| string | length: 4294967296..18446744073709551615 | UTF-8 string (8 bytes follow) | 0x7B |
| array | size: 0..23 | array | 0x80..0x97 |
| array | size: 24..255 | array (1 byte follow) | 0x98 |
| array | size: 256..65535 | array (2 bytes follow) | 0x99 |
| array | size: 65536..4294967295 | array (4 bytes follow) | 0x9A |
| array | size: 4294967296..18446744073709551615 | array (8 bytes follow) | 0x9B |
| object | size: 0..23 | map | 0xA0..0xB7 |
| object | size: 24..255 | map (1 byte follow) | 0xB8 |
| object | size: 256..65535 | map (2 bytes follow) | 0xB9 |
| object | size: 65536..4294967295 | map (4 bytes follow) | 0xBA |
| object | size: 4294967296..18446744073709551615 | map (8 bytes follow) | 0xBB |
| binary | size: 0..23 | byte string | 0x40..0x57 |
| binary | size: 24..255 | byte string (1 byte follow) | 0x58 |
| binary | size: 256..65535 | byte string (2 bytes follow) | 0x59 |
| binary | size: 65536..4294967295 | byte string (4 bytes follow) | 0x5A |
| binary | size: 4294967296..18446744073709551615 | byte string (8 bytes follow) | 0x5B |
Binary values with subtype are mapped to tagged values (0xD8..0xDB) depending on the subtype, followed by a byte string, see "binary" cells in the table above.
Complete mapping
The mapping is complete in the sense that any JSON value type can be converted to a CBOR value, with one exception: an integer outside the range of CBOR integers (only possible with a number type wider than 64 bits) throws out_of_range.407.
Not a deterministic encoding
to_cbor does not produce the deterministic encoding of RFC 8949, Section 4.2.1, and not always the preferred serialization of Section 4.1:
- floating-point numbers are written as single precision (0xFA) when that is exact and as double precision (0xFB) otherwise; half precision (0xF9) is used only for
NaNandInfinity/-Infinity(1.0isfa 3f 80 00 00); - a binary subtype 0..23 is written as
d8 xxinstead of the one-byte tag 0xC0..0xD7 (d8 01 41 01); - object members are written in the order of
object_t(for the defaultstd::map: sorted by the key string, not by the encoded key bytes).
Integers and lengths use the shortest form, and indefinite-length items are never written. Decoders must accept all of this; the RFC does not require the preferred serialization.
NaN/infinity handling
NaN, Infinity, and -Infinity are serialized as a CBOR half-precision float (type 0xF9, 3 bytes total): NaN as 0xF9 0x7E 0x00, Infinity as 0xF9 0x7C 0x00, and -Infinity as 0xF9 0xFC 0x00. This behavior differs from the normal JSON serialization which serializes NaN or Infinity to null.
Note
Prior to version 3.8.0, NaN and Infinity were instead serialized as a CBOR double-precision float (type 0xFB, 9 bytes total).
Unused CBOR types
The following CBOR types are not used in the conversion:
- UTF-8 strings terminated by "break" (0x7F)
- arrays terminated by "break" (0x9F)
- maps terminated by "break" (0xBF)
- byte strings terminated by "break" (0x5F)
- date/time (0xC0..0xC1)
- bignum (0xC2..0xC3)
- decimal fraction (0xC4)
- bigfloat (0xC5)
- expected conversions (0xD5..0xD7)
- simple values (0xE0..0xF3, 0xF8)
- undefined (0xF7)
- break (0xFF)
Tagged items
Binary subtypes will be serialized as tagged items. See binary values for an example.
Example: serialize a JSON value to CBOR
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using nlohmann::json;
using namespace nlohmann::literals;
int main()
{
// create a JSON value
json j = R"({"compact": true, "schema": 0})"_json;
// serialize it to CBOR
std::vector<std::uint8_t> v = json::to_cbor(j);
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(byte) << " ";
}
std::cout << '\n';
}
Output:
0xa2 0x67 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0xf5 0x66 0x73 0x63 0x68 0x65 0x6d 0x61 0x00
Deserialization¶
The library maps CBOR types to JSON value types as follows:
| CBOR type | JSON value type | first byte |
|---|---|---|
| Integer | number_unsigned | 0x00..0x17 |
| Unsigned integer | number_unsigned | 0x18 |
| Unsigned integer | number_unsigned | 0x19 |
| Unsigned integer | number_unsigned | 0x1A |
| Unsigned integer | number_unsigned | 0x1B |
| Negative integer | number_integer | 0x20..0x37 |
| Negative integer | number_integer | 0x38 |
| Negative integer | number_integer | 0x39 |
| Negative integer | number_integer | 0x3A |
| Negative integer | number_integer | 0x3B |
| Byte string | binary | 0x40..0x57 |
| Byte string | binary | 0x58 |
| Byte string | binary | 0x59 |
| Byte string | binary | 0x5A |
| Byte string | binary | 0x5B |
| Byte string | binary | 0x5F |
| UTF-8 string | string | 0x60..0x77 |
| UTF-8 string | string | 0x78 |
| UTF-8 string | string | 0x79 |
| UTF-8 string | string | 0x7A |
| UTF-8 string | string | 0x7B |
| UTF-8 string | string | 0x7F |
| array | array | 0x80..0x97 |
| array | array | 0x98 |
| array | array | 0x99 |
| array | array | 0x9A |
| array | array | 0x9B |
| array | array | 0x9F |
| map | object | 0xA0..0xB7 |
| map | object | 0xB8 |
| map | object | 0xB9 |
| map | object | 0xBA |
| map | object | 0xBB |
| map | object | 0xBF |
| False | false | 0xF4 |
| True | true | 0xF5 |
| Null | null | 0xF6 |
| Half-Precision Float | number_float | 0xF9 |
| Single-Precision Float | number_float | 0xFA |
| Double-Precision Float | number_float | 0xFB |
Indefinite-length UTF-8 strings (0x7F) and byte strings (0x5F) are supported. Each chunk must be a definite-length string of the same major type, as required by RFC 8949, Section 3.2.3.
Incomplete mapping
The mapping is incomplete in the sense that not all CBOR types can be converted to a JSON value. The following CBOR types are not supported and will yield parse errors:
- simple values (0xE0..0xF3, 0xF8)
- undefined (0xF7)
Tagged items (0xC0..0xDB) are not interpreted either; see the note on tagged items below.
Negative integer overflow
CBOR negative integers (major type 1) are decoded as -1 - n. If the encoded magnitude n is too large for the result to fit into number_integer_t (std::int64_t by default), the result is stored as number_float_t, like a too small integer in JSON text. For example, -18446744073709551616 (0x3B followed by eight 0xFF bytes) is stored as -1.8446744073709552e+19.
Object keys
CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps with keys other than text strings (major type 3) are rejected with a parse_error.113 exception (or, with allow_exceptions set to false, a discarded value) naming the type of the key that was found, for instance:
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01
This applies to the SAX interface as well, as the key is read before it is passed on. This is a deliberate restriction of the library's JSON value model, not an oversight: formats built on CBOR maps with integer keys, such as COSE (RFC 9052) or CWT (RFC 8392), cannot be read with this library and need a general-purpose CBOR library instead.
A tag in front of a text string key is handled like a tag in front of a value: it is skipped with cbor_tag_handler_t::ignore or cbor_tag_handler_t::store and rejected with the default error. A tag does not turn a key of another type into a text string.
A map with duplicate keys is well-formed, but invalid under RFC 8949, Section 5.3.1. from_cbor accepts it and the last value wins; the SAX interface reports every occurrence.
Ill-formed UTF-8 in text strings
RFC 8949, Section 3.1 requires CBOR text strings (major type 3) to be valid UTF-8 (a string with ill-formed UTF-8 is "well-formed but invalid"), and Section 5.3.1 leaves it up to the decoder whether to verify this, so checking is opt-in: with the error_handler parameter left at keep (the default), from_cbor() accepts a text string (object keys included) whose bytes are not valid UTF-8 and hands them back unchanged. Passing error_handler_t::strict makes from_cbor() check and throw parse_error.113 for ill-formed UTF-8, and replace/ignore sanitize the string instead of keeping it. However, dump() still requires valid UTF-8 and throws type_error.316 for a value read with the default keep handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. to_cbor()'s own error_handler parameter defaults to keep, so such a value is written back unchanged; with strict (the default if JSON_STRICT_BINARY_UTF8 is enabled), it throws the same exception instead. Byte strings (major type 2) are unaffected, since they are not required to hold text.
The chunks of an indefinite-length text string are checked one by one, as required by RFC 8949, Section 3.2.3: a code point cannot be spread over two chunks. With error_handler_t::strict such a string throws parse_error.113; replace and ignore sanitize each chunk on its own.
Tagged items
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing cbor_tag_handler_t::ignore to function from_cbor, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing cbor_tag_handler_t::store to function from_cbor stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the tag directly in front of it is stored, and only when it has the form 0xD8..0xDB (a tag 0..23 written as 0xC0..0xD7 is skipped); for example, D8 05 C1 41 AA yields subtype null, whereas C1 D8 05 41 AA yields 5. Tags in front of a text string map key are skipped as well. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
Example: deserialize a JSON value from CBOR
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using nlohmann::json;
int main()
{
// create byte vector
std::vector<std::uint8_t> v = {0xa2, 0x67, 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x63,
0x74, 0xf5, 0x66, 0x73, 0x63, 0x68, 0x65, 0x6d,
0x61, 0x00
};
// deserialize it with CBOR
json j = json::from_cbor(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << '\n';
}
Output:
{
"compact": true,
"schema": 0
}
See also¶
- Binary formats - overview and comparison of the supported binary formats
- Binary values - how binary values and subtypes map to CBOR
to_cbor- serialize a JSON value to CBORfrom_cbor- parse a JSON value from CBORJSON_STRICT_BINARY_UTF8- reject ill-formed UTF-8 in strings- SAX interface - decode binary formats without building a value
cbor_tag_handler_t- how CBOR tags are handled when parsing