* Add images of json_documents: save() and load()
An image is a document stored so that loading it needs no
parsing: save() writes the node index, the text and the decoded
strings; the static load() reads an image written by save().
load() takes a pointer and size, a borrowed vector, or an owned
rvalue vector; the nodes are copied so they are aligned and can
be edited, while the text and decoded strings stay in the image.
image_check controls how much load() trusts the input: full
checks structure, bounds, strings and numbers, the parser's own
guarantees; bounds checks structure and bounds only; none skips
all checks, for images from a trusted source.
Layout is little-endian only ("NJVI" header, nodes, text, decoded
strings), following the idea of zero-copy formats such as
FlatBuffers and YaFF; the check follows FlatBuffers' Verifier.
New errors: parse_error.116 for a malformed image or a failed
check, type_error.320 for a discarded document or a big-endian
target.
A dedicated fuzzer and 6,000 seeded corruptions, checked under
ASan/UBSan, found and fixed two gaps: unchecked reserved header
fields, and unbounded null/boolean offsets that could make
dump() throw std::length_error.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC errors in image test and add image_check doc anchor
Move the raw string literal out of the CHECK macro (MSVC preprocessor),
cast 64-bit header fields to std::size_t (C4244 on 32-bit), and give the
image_check section in load.md a real anchor for the mkdocs strict build.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check each distinct string and float range once in the full image check
The full check scanned the contents of every string node and every float token over its own range, so many nodes pointing to one large range made load() quadratic. The structural walk now only collects the ranges; after it, each kind is sorted and checked once per distinct range. Ranges of one kind that overlap without being identical are rejected, as save() never writes them (nodes that share a value share the whole range). The cost is linear in the size of the image plus sorting the ranges.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Adapt the image tests to named documents and last-wins lookups
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test that the targets of links never reach an image
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test lookups in loaded images with duplicate and colliding keys
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Release the list of large objects once load() has built the indexes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test that invalid UTF-8 of a loaded image is not copied into an editable document
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document that editable documents refuse invalid UTF-8 copied from a loaded image
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Expect first-wins lookups in the image tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid useless casts of header fields in the image tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy 22 findings in unit-json_view_image.cpp and image.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC C4244 and C2017 in the image test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the indentation of a loop in the image test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.
- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
json_view.hpp only includes json.hpp, so including them here would duplicate
their definitions
- drop edit_storage.hpp, which edit.hpp includes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence a clang-tidy C++17 finding in the image test
bugprone-suspicious-stringview-data-usage flags data() of the source view
under C++17, although the length is passed and checked by the REQUIRE
before it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Build the image link test without exceptions
load_result() exists only with exceptions; with JSON_NOEXCEPTION the test
loads the image with the check directly (a failed check aborts).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Follow the renames from the review of the view builder
The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Rename the emit lambda in the image writer for Qt compatibility
Qt defines emit as an empty macro, so a Qt header included before
json_view.hpp broke the lambda's declaration (see #5865). The slots
variables in the image tests are renamed as well.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address the review of json_document images
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
7.6 KiB
nlohmann::basic_json_document::parse
// (1)
template<typename InputType>
static basic_json_document parse(InputType&& input,
const bool allow_exceptions = true,
const bool ignore_comments = false,
const bool ignore_trailing_commas = false);
// (2)
template<typename IteratorType>
static basic_json_document parse(IteratorType first, IteratorType last,
const bool allow_exceptions = true,
const bool ignore_comments = false,
const bool ignore_trailing_commas = false);
- Deserialize from a compatible input, borrowing or owning it depending on its value category and type (see Notes).
- Deserialize from a pair of input iterators.
Both overloads accept the same JSON text as BasicJsonType::parse(), with the same
ignore_comments/ignore_trailing_commas options, but build a basic_json_document (a flat index into
the input) instead of a tree of BasicJsonType values. They take the same inputs as BasicJsonType::parse(). Byte
inputs that stay alive can be borrowed; all others (streams, #!cpp FILE*, wide characters, non-contiguous
iterators) are read into a copy that the document owns -- see the Notes. Use parse_copy() to get an
owned copy of a borrowable input as well.
Template parameters
InputType- A compatible input, for instance:
- a
#!cpp std::string,#!cpp std::string_view, or a C-style array of single-byte characters - a pointer to a null-terminated string of single-byte characters (
#!cpp char,#!cpp signed char,#!cpp unsigned char,#!cpp std::uint8_t) - a container for which
#!cpp obj.data()and#!cpp obj.size()give contiguous single-byte access, e.g.#!cpp std::vector<char>or#!cpp std::vector<std::uint8_t> - an
#!cpp std::istreamobject - a
#!cpp FILE*pointer - wide characters (
#!cpp wchar_t,#!cpp char16_t,#!cpp char32_t), which are converted to UTF-8: a wide string object (#!cpp std::wstring,#!cpp std::u16string,#!cpp std::u32string), an array (e.g. a wide string literal), or a pointer to a null-terminated wide string
- a
IteratorType- an input iterator type, for instance a pair of pointers such as
ptrandptr + len, or a pair of#!cpp std::string::iterator; the iterators of single-byte characters are borrowed or read like the byte inputs above, those of wide characters are converted to UTF-8
Parameters
input(in)- Input to parse from.
allow_exceptions(in)- whether to throw exceptions in case of a parse error (optional,
#!cpp trueby default) ignore_comments(in)- whether comments should be ignored and treated like whitespace (
#!cpp true) or yield a parse error (#!cpp false); (optional,#!cpp falseby default) ignore_trailing_commas(in)- whether trailing commas in arrays or objects should be ignored and treated like whitespace (
#!cpp true) or yield a parse error (#!cpp false); (optional,#!cpp falseby default) first(in)- iterator to the start of a character range
last(in)- iterator to the end of a character range
Return value
The parsed document. If allow_exceptions is #!cpp false and the input is not valid JSON, the returned document is
discarded; see is_discarded.
Exceptions
Throws the same exception BasicJsonType::parse() throws for the same input and options --
the same exception id, message, and position -- because on a failing input the library's own parser is run on the
same bytes to produce the diagnostic. This replay runs only for a failing input, and costs about a lexing pass up to
the error (its SAX handler builds no values); with allow_exceptions set to #!cpp false, it is skipped and a
discarded document is returned instead. Additionally throws
out_of_range.416 if the input is 4294967280 bytes (4 GiB
minus 16 bytes) or larger, a size BasicJsonType::parse() does not reject.
Complexity
Linear in the length of the input.
Notes
Ownership. Whether the document borrows input or owns a copy of it depends on its value category and type:
input |
ownership |
|---|---|
lvalue byte container (std::string, std::vector<char>, ...), std::string_view, C string, character array |
borrowed -- input must outlive the document |
non-const rvalue #!cpp std::string |
owned, moved in without a copy |
other rvalue byte container (including a #!cpp const rvalue #!cpp std::string) |
owned, copied |
stream, #!cpp FILE*, wide characters, or anything else read through the general input adapter |
owned, read into a buffer (a stream is read to its end) |
For overload (2), a pair of pointers to single-byte integers (e.g. #!cpp const char*, #!cpp std::uint8_t*) is
borrowed. From C++20 on, so is any other contiguous iterator over single bytes, such as
#!cpp std::vector<char>::iterator or #!cpp std::string::const_iterator. Before C++20 these iterators cannot be
told apart from other class-type iterators, so their range is read into an owned buffer, as is any non-contiguous
range (e.g. of a #!cpp std::list<char>).
See owns_source to check which happened after a call, and the
feature page for the reasoning.
Numbers. As for BasicJsonType::parse(), an integer literal too large for the 64-bit
integer type becomes a floating-point value.
No lengths. An integer argument that is not a #!cpp bool where the flags are expected -- for example
#!cpp parse(ptr, len) -- does not compile (the overload is deleted). Such a call would convert len to
allow_exceptions and read ptr as a null-terminated string, past the end of a buffer that has none. To parse a
buffer of a given length, pass a pair of pointers: #!cpp parse(ptr, ptr + len). The same holds for
parse_copy, accept, and read.
Examples
??? example "Example: (1) borrowed vs. owned input, and errors identical to BasicJsonType::parse()"
```cpp
--8<-- "examples/basic_json_document__parse.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__parse.output"
```
??? example "Example: (2) parse an iterator range (no NUL terminator required)"
```cpp
--8<-- "examples/basic_json_document__parse_iterator_pair.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__parse_iterator_pair.output"
```
See also
- parse_copy - deserialize a copy of a compatible input
- accept - check whether the input is valid JSON
- read - (re-)parse into this document, reusing its memory
- owns_source - return whether the document holds its own copy of the text
- load - read a document from an image instead of parsing JSON text
BasicJsonType::parse- the corresponding function ofbasic_json
Version history
- Added in version 3.13.0.