Compare commits

...
2 Commits
Author SHA1 Message Date
Niels Lohmann 8a0679558f Apply the review's wording fixes to the json_view documentation
- Copying is disabled because of its cost, not because of two documents
  borrowing the same buffer; the unique_ptr follows from move-only.
- Inputs: streams, FILE*, and wide characters (strings, arrays, pointers)
  are all supported and owned; point to parse_copy() for owned copies.
- The error replay only runs for failing input, builds no values, and is
  skipped with allow_exceptions = false.
- read() writes copied inputs into the previous owned copy.
- Views can dangle like std::string_view, and why there is no
  shared/weak pointer design.
- The 4 GiB limit (16-byte nodes, 16-byte scanner steps), streams being
  read to the end first, which macros apply, why materialize() has no
  source positions, and wording nits (accept() vs basic_json::accept(),
  is_binary, is_discarded, see-also lists, size table, "back into").

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-16 08:14:21 +02:00
Niels Lohmann 09879b6b5d Address the review of the view builder and document
Changes promised in the replies to the review of #5620:

- read() reuses the owned copy: a copied input (an rvalue byte container,
  a stream, FILE*, wide characters, an iterator range) is appended to the
  cleared owned buffer instead of a fresh std::string being move-assigned
  over it. A moved-in std::string still replaces it.
- The error replay runs the library parser with a SAX handler that stores
  nothing and throws the reported exception, instead of
  BasicJsonType::parse(), which built a DOM up to the error.
- Pointers to and arrays of wide characters, and FILE*, take the input
  adapter path (converted to UTF-8 into an owned buffer) instead of
  failing a static_assert. Wide pointers are NUL-terminated.
- Streams are read in blocks through rdbuf() into the buffer, instead of
  character by character through the input adapter; eofbit is set at the
  end, as by the library's stream adapter.
- tape/tape_size/tape_cap are now nodes/node_count/node_capacity.
- The builder's members get descriptive, m_-prefixed names.
- The shallow frame stack is a std::array (in a wrapper with a
  user-provided constructor: -Weffc++ wants the member initialized, and
  value-initializing the 64 frames costs up to 30 % on small documents).
- The value dispatch macro has no NEXT parameter; the switch is left with
  break, and the macro is undefined after its last use.
- literal() and digits_exceed() deduce the length from the string literal;
  each literal has one helper; the two `if (!is_float)` blocks are merged.
- arena_ensure() sits right before the writes; the decode arena is a
  std::vector<char>.
- The builder records the deepest nesting, and materialize() sizes its
  stack from it once (growing it only if a subtree is deeper).
- Comment fixes (hex4, slow_string, the NUL rule, put_utf8, 0 - m, the
  arena resize, grow()) and formatting (macro backslashes, digit cases,
  one power of ten per line, value_t assertion next to is_container()).
- build_with's template parameters follow the builder's order.

Greg's idea of keeping the source offset in `off` and the arena offset in
`next` for escaped strings was measured and not adopted: string access got
7-18 % slower with a select, and 3.5-18 % slower with a branch-free field
pick (Apple M-series, clang -O3), parsing was unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-16 08:14:20 +02:00
26 changed files with 1236 additions and 533 deletions

No files matched your search

@@ -12,6 +12,11 @@ Checks whether the input is valid JSON, accepting and rejecting exactly what
function never throws an exception for invalid input, and the returned `#!cpp bool` is the only result -- no document
is returned.
The result is the same as that of `BasicJsonType::accept()` by design; the difference is how the check is made. This
function runs the document's index builder (and discards the index), while `BasicJsonType::accept()` runs the SAX
parser with a handler that ignores every event. Use this one if you go on to parse the same input with
[`parse()`](parse.md) anyway, so that both use the same code; either is fine for a plain yes/no answer.
## Template parameters
`InputType`
@@ -16,8 +16,8 @@ basic_json_document(const basic_json_document&) = delete;
2. Move constructor. Takes over `other`'s index and, if owned, its text; `other` is left as an empty document. Views
taken from `other` before the move remain valid, because the index is heap-allocated independently of the
`basic_json_document` object.
3. `basic_json_document` is move-only. Copying is disabled because it would either duplicate a potentially large index
and text, or leave two documents claiming to borrow the same buffer.
3. `basic_json_document` is move-only. Copying is disabled because of its cost: a copy would duplicate the whole
index, the decoded-string buffer, and the owned text, while a move is enough in every use we know of.
## Parameters
@@ -30,7 +30,7 @@ No-throw guarantee: the default and move constructors never throw exceptions.
## Complexity
Constant, for the default and move constructors.
Constant.
## Examples
@@ -11,13 +11,14 @@ A parsed JSON text, held as a flat index of its values
([16 bytes per value](../../home/architecture.md#node-index-of-json-views)) instead of a tree of `BasicJsonType` values.
Strings and numbers stay in the source text; only strings that contain escapes are decoded, into one buffer owned by
the document. [`basic_json_view`](../basic_json_view/index.md) is a read-only handle to one value of a
`basic_json_document`; [`materialize()`](../basic_json_view/materialize.md) turns a subtree back into the
`basic_json_document`; [`materialize()`](../basic_json_view/materialize.md) turns a subtree into the
`BasicJsonType` value that [`BasicJsonType::parse()`](../basic_json/parse.md) would have produced for it.
A document may **borrow** the text it was parsed from (the caller's buffer must then outlive the document) or **own**
it (a copy, or an rvalue `#!cpp std::string` that was moved in); see [`owns_source`](owns_source.md). `basic_json_document`
is move-only: copying a document would either duplicate a potentially large index and text, or leave two documents
claiming to borrow the same buffer, so it is disabled.
is move-only: a copy would duplicate the whole index, the decoded-string buffer, and the owned text, and a move is
enough in every use we know of. The index, the decoded strings, and the owned text live in one heap block that the
document holds through a `#!cpp std::unique_ptr`; this is also why views survive a move of the document.
## Template parameters
@@ -4,7 +4,7 @@
bool owns_source() const noexcept;
```
Returns whether the document holds its own copy of the parsed text, as opposed to borrowing the caller's buffer.
Returns whether the document holds its own copy of the parsed text or is borrowing the caller's buffer.
## Return value
@@ -21,13 +21,15 @@ static basic_json_document parse(IteratorType first, IteratorType last,
Both overloads accept the same JSON text as [`BasicJsonType::parse()`](../basic_json/parse.md), with the same
`ignore_comments`/`ignore_trailing_commas` options, but build a [`basic_json_document`](index.md) (a flat index into
the input) instead of a tree of `BasicJsonType` values. The input must be byte-oriented (see the template parameters
below): not every input type of `BasicJsonType::parse()` is supported.
the input) instead of a tree of `BasicJsonType` values. They take the same inputs as `BasicJsonType::parse()`. Byte
inputs that stay alive can be borrowed; all others (streams, `#!cpp FILE*`, wide characters, non-contiguous
iterators) are read into a copy that the document owns -- see the Notes. Use [`parse_copy()`](parse_copy.md) to get an
owned copy of a borrowable input as well.
## Template parameters
`InputType`
: A byte-oriented input, one of:
: A compatible input, for instance:
- a `#!cpp std::string`, `#!cpp std::string_view`, or a C-style array of single-byte characters
- a pointer to a null-terminated string of single-byte characters (`#!cpp char`, `#!cpp signed char`,
@@ -35,12 +37,10 @@ below): not every input type of `BasicJsonType::parse()` is supported.
- a container for which `#!cpp obj.data()` and `#!cpp obj.size()` give contiguous single-byte access, e.g.
`#!cpp std::vector<char>` or `#!cpp std::vector<std::uint8_t>`
- an `#!cpp std::istream` object
- a wide string object (`#!cpp std::wstring`, `#!cpp std::u16string`, `#!cpp std::u32string`), which is converted
to UTF-8
Other inputs are not supported: a `#!cpp FILE*`, and pointers to or arrays of wide characters (`#!cpp wchar_t`,
`#!cpp char16_t`, `#!cpp char32_t`) are rejected at compile time by a `#!cpp static_assert`. (Use
[`BasicJsonType::parse()`](../basic_json/parse.md) for these.)
- a `#!cpp FILE*` pointer
- wide characters (`#!cpp wchar_t`, `#!cpp char16_t`, `#!cpp char32_t`), which are converted to UTF-8: a wide
string object (`#!cpp std::wstring`, `#!cpp std::u16string`, `#!cpp std::u32string`), an array (e.g. a wide
string literal), or a pointer to a null-terminated wide string
`IteratorType`
: an input iterator type, for instance a pair of pointers such as `ptr` and `ptr + len`, or a pair of
@@ -78,7 +78,9 @@ discarded; see [`is_discarded`](is_discarded.md).
Throws the same exception [`BasicJsonType::parse()`](../basic_json/parse.md) throws for the same input and options --
the same exception id, message, and position -- because on a failing input the library's own parser is run on the
same bytes to produce the diagnostic. Additionally throws
same bytes to produce the diagnostic. This replay runs only for a failing input, and costs about a lexing pass up to
the error (its SAX handler builds no values); with `allow_exceptions` set to `#!cpp false`, it is skipped and a
discarded document is returned instead. Additionally throws
[`out_of_range.416`](../../home/exceptions.md#jsonexceptionout_of_range416) if the input is 4294967280 bytes (4 GiB
minus 16 bytes) or larger, a size [`BasicJsonType::parse()`](../basic_json/parse.md) does not reject.
@@ -95,7 +97,7 @@ Linear in the length of the input.
| lvalue byte container (`std::string`, `std::vector<char>`, ...), `std::string_view`, C string, character array | **borrowed** -- `input` must outlive the document |
| non-const rvalue `#!cpp std::string` | **owned**, moved in without a copy |
| other rvalue byte container (including a `#!cpp const` rvalue `#!cpp std::string`) | **owned**, copied |
| stream, wide string, or anything else read through the general input adapter | **owned**, read into a buffer (a stream is read to its end) |
| stream, `#!cpp FILE*`, wide characters, or anything else read through the general input adapter | **owned**, read into a buffer (a stream is read to its end) |
For overload (2), a pair of pointers to single-byte integers (e.g. `#!cpp const char*`, `#!cpp std::uint8_t*`) is
borrowed. From C++20 on, so is any other contiguous iterator over single bytes, such as
@@ -47,7 +47,9 @@ Every view taken from `#!cpp *this` before the call -- including the previous [`
whether or not the new parse succeeds; take fresh views from [`root()`](root.md) afterward.
`input` is borrowed or owned by the same rules as [`parse()`](parse.md#notes); a document can borrow on one call and
own on the next, since ownership is decided freshly each time.
own on the next, since ownership is decided freshly each time. An input that is copied (see the table in
[`parse()`](parse.md#notes)) is written into the document's previous owned copy, reusing its memory; a moved-in
`#!cpp std::string` replaces it.
An integer argument that is not a `#!cpp bool` where the flags are expected, such as `#!cpp read(ptr, len)`, does not
compile; see [`parse`](parse.md#notes).
@@ -39,7 +39,8 @@ auto v = doc.root();
`root()` is a cheap handle into the document's index, not a copy of anything; call it as often as needed. The
returned view is valid under the same conditions as any other view of the document -- see
[Object inspection](../basic_json_view/index.md) -- in particular, it is invalidated by the next
[`read()`](read.md) or [`shrink_to_fit()`](shrink_to_fit.md) on this document.
[`read()`](read.md) or [`shrink_to_fit()`](shrink_to_fit.md) on this document. A view can dangle, like a
`#!cpp std::string_view`; see [Ownership and lifetime](../../features/json_view.md#ownership-and-lifetime).
## Examples
@@ -38,7 +38,8 @@ Constant.
## See also
- [is_structured](is_structured.md) - return whether the type is structured
- [size](size.md), [empty](empty.md) - the number of elements, and whether there are none
- [size](size.md) - the number of elements
- [empty](empty.md) - whether there are no elements
- [`BasicJsonType::is_array`](../basic_json/is_array.md) - the corresponding function of `basic_json`
## Version history
@@ -4,8 +4,9 @@
bool is_binary() const noexcept;
```
This function always returns `#!cpp false`: a JSON text has no binary values, so a view can never refer to one. The
function exists for interface parity with [`BasicJsonType::is_binary`](../basic_json/is_binary.md).
This function always returns `#!cpp false`: a document is only ever parsed from JSON text, never from CBOR or the other
binary formats, and a JSON text has no binary values, so a view can never refer to one. The function exists for
interface parity with [`BasicJsonType::is_binary`](../basic_json/is_binary.md).
## Return value
@@ -11,7 +11,7 @@ that is itself [discarded](../basic_json_document/is_discarded.md) -- in particu
## Return value
`#!cpp true` if the view is discarded, `#!cpp false` otherwise.
`#!cpp true` if the view or its document is discarded, `#!cpp false` otherwise.
## Exception safety
@@ -38,7 +38,8 @@ Constant.
## See also
- [is_structured](is_structured.md) - return whether the type is structured
- [size](size.md), [empty](empty.md) - the number of elements, and whether there are none
- [size](size.md) - the number of elements
- [empty](empty.md) - whether there are no elements
- [`BasicJsonType::is_object`](../basic_json/is_object.md) - the corresponding function of `basic_json`
## Version history
@@ -37,7 +37,8 @@ Constant.
## See also
- [source_offset](source_offset.md) - byte offset of this value in the document's source text
- [source_offset](source_offset.md) - byte offset of this value in the source text, which differs for strings: an
escaped string has none
- [`BasicJsonType::is_string`](../basic_json/is_string.md) - the corresponding function of `basic_json`
## Version history
@@ -14,8 +14,8 @@ if the view is [discarded](is_discarded.md).
## Exception safety
Strong guarantee: if an exception is thrown, there are no changes to the view or the document it refers to (nothing
about either is mutated by this function).
Strong guarantee: there are no changes to the view or the document it refers to (nothing about either is mutated by
this function).
## Exceptions
@@ -28,12 +28,13 @@ Linear in the size of the subtree.
## Notes
`materialize()` replays the subtree through the same SAX builder [`BasicJsonType::parse()`](../basic_json/parse.md)
uses internally, so the result matches it exactly -- including, for an object, that a repeated key keeps only its
uses, so the result matches it exactly -- including, for an object, that a repeated key keeps only its
last value. The replay is iterative, so it is not limited by the call stack the way a naive recursive conversion
would be; the JSON nesting depth is limited only by available memory, as for `BasicJsonType::parse()` itself.
Unlike parsing with [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) enabled, the values
produced by `materialize()` do not carry source positions: there is no lexer run during the replay to record them.
produced by `materialize()` do not carry source positions. The index has the start offset of every value, but not the
end of arrays, objects, and escaped strings, which the positions also need.
Calling `materialize()` on the same view repeatedly builds a new, independent `BasicJsonType` value each time; it
never caches the result.
+9 -9
View File
@@ -10,15 +10,15 @@ Returns the number of elements, as [`BasicJsonType::size()`](../basic_json/size.
The return value depends on the type and is defined as follows:
| Value type | return value |
|----------------------|-----------------------------|
| null | `0` |
| discarded | `0` |
| boolean | `1` |
| string | `1` |
| number | `1` |
| object | number of key/value pairs |
| array | number of elements |
| Value type | return value |
|------------|---------------------------|
| null | `0` |
| discarded | `0` |
| boolean | `1` |
| string | `1` |
| number | `1` |
| object | number of key/value pairs |
| array | number of elements |
## Exception safety
+29 -16
View File
@@ -16,8 +16,8 @@ most of the resulting tree is thrown away almost immediately.
[`basic_json_document::parse()`](../api/basic_json_document/parse.md) parses the same JSON grammar, with the same
options, but instead of a tree it builds a flat index of the values it found: one
[16-byte entry](../home/architecture.md#node-index-of-json-views) per value (and one per object key), in document
order. Strings and numbers are not copied out of the input; they stay in the source text, and are only decoded when
actually needed (for a string, only if it contains escape sequences, into one shared buffer owned by the document).
order. Strings and numbers are referenced in place in the source text; only strings that contain escape sequences
are decoded, into one shared buffer owned by the document.
[`basic_json_view`](../api/basic_json_view/index.md) is a small, trivially copyable handle (two pointers) into that
index. It gives you the read-only, type-inspection part of the `basic_json` interface --
@@ -31,7 +31,8 @@ index. It gives you the read-only, type-inspection part of the `basic_json` inte
Include `<nlohmann/json_view.hpp>` in addition to (or instead of) `<nlohmann/json.hpp>`. Parse into a
[`json_document`](../api/json_document.md), inspect its [`root()`](../api/basic_json_document/root.md), and
`materialize()` when you need a real value:
`materialize()` when you need a real value. `materialize()` works on any view, not just the root, so you can turn just
the subtree you need into a value:
??? example "Example: parse a document, inspect its root, and materialize it"
@@ -68,15 +69,19 @@ A view is valid only while all of the following hold:
- the document is alive,
- the document has not been re-parsed since the view was taken (with [`read()`](../api/basic_json_document/read.md)
or [`parse()`](../api/basic_json_document/parse.md) into it), and has not had
[`shrink_to_fit()`](../api/basic_json_document/shrink_to_fit.md) called on it since, and
or [`parse()`](../api/basic_json_document/parse.md) into it),
- the document has not had [`shrink_to_fit()`](../api/basic_json_document/shrink_to_fit.md) called on it since, and
- if the document borrows its source text, that text is still alive.
Moving the document itself is fine and does **not** invalidate its views: the index is a separate heap allocation
that keeps its address across the move. Take a fresh view from [`root()`](../api/basic_json_document/root.md)
Moving the `basic_json_document<>` itself is fine and does **not** invalidate its views: the index is a separate heap
allocation that keeps its address across the move. Take a fresh view from [`root()`](../api/basic_json_document/root.md)
whenever any of the other conditions above was not met.
Because a view dies with its document, [`root()`](../api/basic_json_document/root.md) is not callable on a temporary
A view can dangle, exactly like a `#!cpp std::string_view`: nothing stops you from using it after its document is
gone. This is deliberate. A view is two pointers and trivially copyable; making it safe (a `#!cpp std::shared_ptr` in
the document and a `#!cpp std::weak_ptr` in each view) would make it larger, add reference counting to every copy,
and a `lock()` to every access -- exactly the costs the view exists to avoid. What is caught is the obvious case:
because a view dies with its document, [`root()`](../api/basic_json_document/root.md) is not callable on a temporary
document: `#!cpp auto v = json_document::parse(text).root();` does not compile. Give the document a name first.
??? example "Example: borrowed and owned documents, and when views become invalid"
@@ -98,14 +103,19 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
[`basic_json::accept()`](../api/basic_json/accept.md)/[`basic_json::parse()`](../api/basic_json/parse.md), with the
same `ignore_comments` and `ignore_trailing_commas` options.
- **Errors.** A failing parse throws the same exception -- the same id, message, and position -- because on failure
the library's own parser is run on the same bytes to produce the diagnostic.
`#!cpp allow_exceptions == false` gives a [discarded](../api/basic_json_document/is_discarded.md) document instead
of throwing, just as it gives a discarded value for `#!cpp basic_json::parse()`.
the library's own parser is run on the same bytes to produce the diagnostic. Valid input never pays for this, and
the replay builds no values: it costs about a lexing pass up to the error.
`#!cpp allow_exceptions == false` skips the replay and gives a
[discarded](../api/basic_json_document/is_discarded.md) document instead of throwing, just as it gives a discarded
value for `#!cpp basic_json::parse()`.
- **Number classification.** An integer literal that does not fit into the 64-bit integer type becomes a
floating-point value, exactly as it does for `#!cpp basic_json::parse()`.
- **Macros.** [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) and
[`JSON_NOEXCEPTION`](../api/macros/json_noexception.md)/[`JSON_THROW_USER`](../api/macros/json_throw_user.md)
behave the same way they do for `<nlohmann/json.hpp>`.
behave the same way they do for `<nlohmann/json.hpp>`. These are the macros that change what the parser accepts or
how it reports errors. The others either do not apply ([`JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md)
needs parent pointers in `basic_json` values) or only affect the values `materialize()` produces, through
`BasicJsonType`.
## What is different
@@ -113,12 +123,15 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
`number_unsigned_t` to both be 64 bits wide; this is a compile-time `#!cpp static_assert`.
- **A 4 GiB input limit.** An input of 4294967280 bytes (4 GiB minus 16 bytes) or more throws
[`out_of_range.416`](../home/exceptions.md#jsonexceptionout_of_range416), a limit
`#!cpp basic_json::parse()` does not have.
- **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`.
`#!cpp basic_json::parse()` does not have. It keeps an index entry at 16 bytes: offsets and counts are 32 bits
wide. The limit is 16 bytes below 2^32, because the parser reads in 16-byte steps, and a position one step past the
end must still fit.
- **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`: the stream
is read to its end into a buffer the document owns, and parsing starts after that.
- **No source positions on `materialize()`.** Even with
[`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) enabled,
[`materialize()`](../api/basic_json_view/materialize.md) does not set them: there is no lexer run during the
replay to record them.
[`materialize()`](../api/basic_json_view/materialize.md) does not set them. The index has the start offset of every
value, but not the end of arrays, objects, and escaped strings, which the positions also need.
- **Element access, iteration, `get<T>()`, JSON Pointer, `dump()`, and comparison are not (yet) provided** by
`basic_json_view`. For now, [`materialize()`](../api/basic_json_view/materialize.md) is the way to get a value you
can do those things with.
+171 -136
View File
@@ -74,9 +74,10 @@ class builder
{
public:
builder(document_data& d, const char* src, std::size_t size) noexcept
: doc(d)
, b(reinterpret_cast<const unsigned char*>(src))
, e(b + size)
: m_doc(d)
, m_begin(reinterpret_cast<const unsigned char*>(src))
, m_end(m_begin + size)
, m_shallow()
{}
/// returns false and fills `failure` on error
@@ -106,21 +107,30 @@ class builder
bool is_object;
};
document_data& doc;
const unsigned char* const b;
const unsigned char* const e;
/// 64 frames, left uninitialized: they are filled as containers open (the
/// user-provided constructor keeps the member initialization, which
/// -Weffc++ asks for, from zeroing them; that costs up to 30 % on small
/// documents)
struct frame_stack : std::array<frame, 64>
{
frame_stack() noexcept {} // NOLINT(modernize-use-equals-default,hicpp-use-equals-default,cppcoreguidelines-pro-type-member-init,hicpp-member-init): see above
};
document_data& m_doc;
const unsigned char* const m_begin;
const unsigned char* const m_end;
parse_failure m_failure{};
// the open array/object is in the cursor; enclosing ones on a stack that is
// inline for the first 64 levels
frame shallow[64]; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays): not initialized on purpose; filled as containers open
std::vector<frame> deep{};
frame_stack m_shallow;
std::vector<frame> m_deep{};
NLOHMANN_VIEW_NOINLINE bool fail(error_code c, const unsigned char* at) noexcept
{
m_failure.code = c;
m_failure.offset = static_cast<std::size_t>(at - b);
doc.tape_size = 0;
m_failure.offset = static_cast<std::size_t>(at - m_begin);
m_doc.node_count = 0;
return false;
}
@@ -149,7 +159,7 @@ class builder
/// the comment at p (*p == '/'): the position after it, or nullptr on error
NLOHMANN_VIEW_NOINLINE const unsigned char* comment(const unsigned char* p)
{
if (e - p < 2)
if (m_end - p < 2)
{
++p;
return fail_at(error_code::comment_start, p);
@@ -159,7 +169,7 @@ class builder
p += 2;
// (as in parse(), a null byte is the end of the input, so it is
// left for the caller to see)
while (p != e && *p != '\n' && *p != '\r' && !(NulIsEnd && *p == 0))
while (p != m_end && *p != '\n' && *p != '\r' && !(NulIsEnd && *p == 0))
{
++p;
}
@@ -170,11 +180,11 @@ class builder
p += 2;
for (;;)
{
if (p == e || (NulIsEnd && *p == 0))
if (p == m_end || (NulIsEnd && *p == 0))
{
return fail_at(error_code::comment_unterminated, p);
}
if (*p == '*' && p + 1 != e && p[1] == '/')
if (*p == '*' && p + 1 != m_end && p[1] == '/')
{
p += 2;
return p;
@@ -192,31 +202,32 @@ class builder
/// doubling repeatedly; returns the new node array
NLOHMANN_VIEW_NOINLINE node* grow(std::size_t n, const unsigned char* at)
{
const std::uint64_t done = static_cast<std::uint64_t>(at - b) + 1;
const std::uint64_t guess = static_cast<std::uint64_t>(n) * static_cast<std::uint64_t>(e - b + 1) / done;
const std::uint64_t grown = guess + (guess / 4) + 64; // a variable: GCC calls a cast of the sum useless where std::uint64_t is std::size_t
// (n is below 2^32: the input is smaller than 4 GiB; the sum cannot wrap)
const std::uint64_t wanted = (std::max)(grown, static_cast<std::uint64_t>(n) + (n / 2) + 64);
// (n nodes are written; at the limit there is no room for another one)
const std::uint64_t limit = document_data::max_nodes();
doc.tape_size = n;
// LCOV_EXCL_START (a node array that fills the address space)
if (NLOHMANN_VIEW_UNLIKELY(n >= limit))
{
document_data::throw_bad_alloc(); // no room for another node
document_data::throw_bad_alloc();
}
// LCOV_EXCL_STOP
const std::uint64_t done = static_cast<std::uint64_t>(at - m_begin) + 1;
const std::uint64_t guess = static_cast<std::uint64_t>(n) * static_cast<std::uint64_t>(m_end - m_begin + 1) / done;
const std::uint64_t grown = guess + (guess / 4) + 64; // a variable: GCC calls a cast of the sum useless where std::uint64_t is std::size_t
// (n is below 2^32: the input is smaller than 4 GiB; the sum cannot wrap)
const std::uint64_t wanted = (std::max)(grown, static_cast<std::uint64_t>(n) + (n / 2) + 64);
m_doc.node_count = n; // the number of nodes reserve() copies into the new array
// (a count beyond the limit is cut: the index does not grow beyond what can be addressed)
doc.reserve(static_cast<std::size_t>((std::min)(wanted, limit)));
return doc.tape;
m_doc.reserve(static_cast<std::size_t>((std::min)(wanted, limit)));
return m_doc.nodes;
}
/// four hex digits at p as a code unit (p moves past them), or -1 (p at
/// the first bad digit); one table lookup per digit and a single check,
/// the four hex digits of a unicode escape (the library's table, after
/// yyjson's read_hex_u16), or -1
/// the four hex digits of a unicode escape at p as a code unit, and p
/// moved past them; or -1, and p at the first bad digit (one table lookup
/// per digit and a single check: the library's table, after yyjson's
/// read_hex_u16)
NLOHMANN_VIEW_ALWAYS_INLINE int hex4(const unsigned char*& p) noexcept
{
if (NLOHMANN_VIEW_LIKELY(e - p >= 4))
if (NLOHMANN_VIEW_LIKELY(m_end - p >= 4))
{
const int cp = hex_codepoint(p);
if (NLOHMANN_VIEW_LIKELY(cp >= 0))
@@ -232,7 +243,7 @@ class builder
/// hex4() failed: the first bad digit (none: p)
NLOHMANN_VIEW_NOINLINE const unsigned char* hex4_error(const unsigned char* p) noexcept
{
if (e - p >= 4)
if (m_end - p >= 4)
{
while (is_hex(*p))
{
@@ -242,8 +253,9 @@ class builder
return p;
}
/// escapes present (or an error) in the string at s, scanned up to p:
/// decode into the arena
/// s points to the start of the string, p to a backslash, a control
/// character, an ill-formed UTF-8 byte, or the end: decode the string into
/// the arena
NLOHMANN_VIEW_NOINLINE decoded slow_string(const unsigned char* s, const unsigned char* p)
{
// single-character escapes; 0: invalid (and 'u', handled separately)
@@ -260,7 +272,7 @@ class builder
arena_run(s, static_cast<std::size_t>(p - s));
for (;;)
{
if (p == e)
if (p == m_end)
{
return failed(error_code::string_missing_quote, p);
}
@@ -272,20 +284,20 @@ class builder
}
if (c != '\\')
{
// (a NUL before the end of the input is a control character, as
// for json::parse, also where a NUL ends the input between values)
// (without JSON_STRICT_NUL_HANDLING, a NUL between values ends
// the input, as in json::parse; inside a string, a NUL is still
// a control character and an error)
return failed(c < 0x20 ? error_code::string_control_character : error_code::string_utf8, p);
}
++p;
if (p == e)
if (p == m_end)
{
return failed(error_code::string_missing_quote, p);
}
const unsigned char d = *p++;
arena_ensure(4);
if (d == 'u')
{
int cp = hex4(p);
int cp = hex4(p); // (moves p past the four digits)
if (NLOHMANN_VIEW_UNLIKELY(cp < 0))
{
return failed(error_code::string_unicode_hex, p);
@@ -296,12 +308,12 @@ class builder
{
return failed(error_code::string_surrogate_low, p);
}
if (e - p < 2 || p[0] != '\\' || p[1] != 'u')
if (m_end - p < 2 || p[0] != '\\' || p[1] != 'u')
{
return failed(error_code::string_surrogate_high, p);
}
p += 2;
const int lo = hex4(p);
const int lo = hex4(p); // (moves p past the four digits)
if (lo < 0)
{
return failed(error_code::string_unicode_hex, p);
@@ -312,23 +324,25 @@ class builder
}
cp = 0x10000 + ((cp - 0xD800) << 10) + (lo - 0xDC00);
}
aw = put_utf8(aw, cp);
arena_ensure(4);
m_arena_write = put_utf8(m_arena_write, cp);
}
else if (NLOHMANN_VIEW_LIKELY(d < 128 && simple_escape[d] != 0))
{
*aw++ = simple_escape[d];
arena_ensure(1);
*m_arena_write++ = simple_escape[d];
}
else
{
--p;
return failed(error_code::string_escape, p);
}
if (p != e && *p == '\\')
if (p != m_end && *p == '\\')
{
continue; // consecutive escapes ("\u00e4\u00f6"): no run in between
}
const unsigned char* const r = p;
p = scan_string_run(p, e);
p = scan_string_run(p, m_end);
arena_run(r, static_cast<std::size_t>(p - r));
}
}
@@ -348,10 +362,17 @@ class builder
return v > (std::numeric_limits<FloatType>::max)() || v < -(std::numeric_limits<FloatType>::max)();
}
/// does the magnitude digits [d, d + n) exceed the given limit (same length)?
static bool digits_exceed(const unsigned char* d, const char* limit, std::size_t n) noexcept
/// do the `digits` digits of a magnitude at d exceed `limit`, the largest
/// magnitude that fits (a string literal: N counts its terminating NUL, so
/// it has N - 1 digits)?
template<std::size_t N>
static NLOHMANN_VIEW_ALWAYS_INLINE bool digits_exceed(const unsigned char* d, std::size_t digits, const char (&limit)[N]) noexcept // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays): a string literal
{
return std::memcmp(d, limit, n) > 0;
if (NLOHMANN_VIEW_LIKELY(digits < N - 1))
{
return false;
}
return digits > N - 1 || std::memcmp(d, limit, N - 1) > 0;
}
static bool is_hex(unsigned char c) noexcept
@@ -359,19 +380,19 @@ class builder
return (c >= '0' && c <= '9') || (c >= 'A' && c <= 'F') || (c >= 'a' && c <= 'f');
}
// decode arena: the std::string in the document, written through a raw
// decode arena: the vector in the document, written through a raw
// pointer (resized ahead in large steps; trimmed when parsing succeeds)
char* aw = nullptr;
char* aend = nullptr;
char* m_arena_write = nullptr;
char* m_arena_end = nullptr;
NLOHMANN_VIEW_ALWAYS_INLINE std::size_t arena_used() const noexcept
{
return aw != nullptr ? static_cast<std::size_t>(aw - doc.arena.data()) : 0;
return m_arena_write != nullptr ? static_cast<std::size_t>(m_arena_write - m_doc.arena.data()) : 0;
}
NLOHMANN_VIEW_ALWAYS_INLINE void arena_ensure(std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(aend - aw) < n))
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(m_arena_end - m_arena_write) < n))
{
arena_grow(n);
}
@@ -380,28 +401,31 @@ class builder
NLOHMANN_VIEW_NOINLINE void arena_grow(std::size_t n)
{
const std::size_t used = arena_used();
doc.arena.resize((std::max)(doc.arena.size() * 2, used + n + 256));
aw = &doc.arena[0] + used; // NOLINT(readability-container-data-pointer): data() is const before C++17
aend = &doc.arena[0] + doc.arena.size(); // NOLINT(readability-container-data-pointer)
m_doc.arena.resize((std::max)(m_doc.arena.size() * 2, used + n + 256));
m_arena_write = m_doc.arena.data() + used;
m_arena_end = m_doc.arena.data() + m_doc.arena.size();
}
/// append the run [r, r + n) to the arena; short runs as one fixed-size
/// 16-byte move when both sides have the room (no library call)
NLOHMANN_VIEW_ALWAYS_INLINE void arena_run(const unsigned char* r, std::size_t n)
{
arena_ensure(n + 16);
if (n <= 16 && e - r >= 16)
if (n <= 16 && m_end - r >= 16)
{
std::memcpy(aw, r, 16);
arena_ensure(16);
std::memcpy(m_arena_write, r, 16);
}
else
{
std::memcpy(aw, r, n);
arena_ensure(n);
std::memcpy(m_arena_write, r, n);
}
aw += n;
m_arena_write += n;
}
/// UTF-8 encoding of cp at w (room for 4 bytes)
/// UTF-8 encoding of cp at w (room for 4 bytes); returns the position after
/// it. (A returned pointer stays in a register; writing through a
/// reference to m_arena_write would store and reload it around every byte.)
static char* put_utf8(char* w, int cp) noexcept
{
if (cp < 0x80)
@@ -446,9 +470,9 @@ class builder
{
explicit cursor(builder& owner) noexcept
: cold(owner)
, b(owner.b)
, p(owner.b)
, e(owner.e)
, b(owner.m_begin)
, p(owner.m_begin)
, e(owner.m_end)
{}
builder& cold; ///< out-of-line helpers and state that needs no registers
@@ -464,13 +488,14 @@ class builder
std::uint32_t cur_count = 0;
bool cur_is_object = false;
std::size_t depth = 0;
std::size_t max_depth = 0;
NLOHMANN_VIEW_ALWAYS_INLINE bool run()
{
cold.doc.reserve(estimate_nodes(reinterpret_cast<const char*>(b), static_cast<std::size_t>(e - b)));
base = cold.doc.tape;
cold.m_doc.reserve(estimate_nodes(reinterpret_cast<const char*>(b), static_cast<std::size_t>(e - b)));
base = cold.m_doc.nodes;
out = base;
cap = base + cold.doc.tape_cap;
cap = base + cold.m_doc.node_capacity;
if (e - p >= 3 && p[0] == 0xEF && p[1] == 0xBB && p[2] == 0xBF)
{
@@ -506,39 +531,40 @@ class builder
// value dispatch, expanded once for array elements and once for member
// values: each jump has its own history (arrays tend to hold one kind of
// value), and the continuation needs no branch on the container kind
#define NLOHMANN_VIEW_VALUE(NEXT) \
switch (cur()) \
{ \
case '"': \
if (NLOHMANN_VIEW_UNLIKELY(!string())) { return false; } \
goto NEXT; \
case '{': \
open(value_t::object); \
++p; \
goto obj_first; \
case '[': \
open(value_t::array); \
++p; \
goto arr_first; \
case '-': \
if (NLOHMANN_VIEW_UNLIKELY(!number<true>())) { return false; } \
goto NEXT; \
case '0': case '1': case '2': case '3': \
case '4': case '5': case '6': case '7': case '8': case '9': \
if (NLOHMANN_VIEW_UNLIKELY(!number<false>())) { return false; } \
goto NEXT; \
case 't': \
if (NLOHMANN_VIEW_UNLIKELY(!literal("true", 4, value_t::boolean, node_flags::is_true))) { return false; } \
goto NEXT; \
case 'f': \
if (NLOHMANN_VIEW_UNLIKELY(!literal_false())) { return false; } \
goto NEXT; \
case 'n': \
if (NLOHMANN_VIEW_UNLIKELY(!literal("null", 4, value_t::null, 0))) { return false; } \
goto NEXT; \
default: \
return fail(error_code::unexpected_value); \
// value), and the continuation needs no branch on the container kind;
// after a scalar, the switch is left to the label that follows it
#define NLOHMANN_VIEW_VALUE \
switch (cur()) \
{ \
case '"': \
if (NLOHMANN_VIEW_UNLIKELY(!string())) { return false; } \
break; \
case '{': \
open(value_t::object); \
++p; \
goto obj_first; \
case '[': \
open(value_t::array); \
++p; \
goto arr_first; \
case '-': \
if (NLOHMANN_VIEW_UNLIKELY(!number<true>())) { return false; } \
break; \
case '0': case '1': case '2': case '3': case '4': \
case '5': case '6': case '7': case '8': case '9': \
if (NLOHMANN_VIEW_UNLIKELY(!number<false>())) { return false; } \
break; \
case 't': \
if (NLOHMANN_VIEW_UNLIKELY(!literal_true())) { return false; } \
break; \
case 'f': \
if (NLOHMANN_VIEW_UNLIKELY(!literal_false())) { return false; } \
break; \
case 'n': \
if (NLOHMANN_VIEW_UNLIKELY(!literal_null())) { return false; } \
break; \
default: \
return fail(error_code::unexpected_value); \
}
arr_first:
@@ -552,7 +578,7 @@ arr_first:
goto close_container;
}
value:
NLOHMANN_VIEW_VALUE(arr_next)
NLOHMANN_VIEW_VALUE
arr_next:
++cur_count;
if (!ws())
@@ -619,7 +645,8 @@ obj_key:
{
return false;
}
NLOHMANN_VIEW_VALUE(obj_next)
NLOHMANN_VIEW_VALUE
#undef NLOHMANN_VIEW_VALUE
obj_next:
++cur_count;
if (!ws())
@@ -647,8 +674,6 @@ obj_next:
}
return fail(error_code::expected_object_end);
#undef NLOHMANN_VIEW_VALUE
close_container:
close();
if (NLOHMANN_VIEW_UNLIKELY(depth == 0))
@@ -670,8 +695,11 @@ root_done:
{
return fail(error_code::trailing_characters);
}
cold.doc.tape_size = static_cast<std::size_t>(out - base);
cold.doc.arena.resize(cold.arena_used());
cold.m_doc.node_count = static_cast<std::size_t>(out - base);
cold.m_doc.max_depth = max_depth;
// (so that size() is the number of bytes used: the arena grows ahead
// in large steps, and shrink_to_fit() keeps size() bytes)
cold.m_doc.arena.resize(cold.arena_used());
return true;
}
@@ -700,11 +728,11 @@ root_done:
case '"':
return string();
case 't':
return literal("true", 4, value_t::boolean, node_flags::is_true);
return literal_true();
case 'f':
return literal_false();
case 'n':
return literal("null", 4, value_t::null, 0);
return literal_null();
case '-':
return number<true>();
case '0':
@@ -723,9 +751,19 @@ root_done:
}
}
NLOHMANN_VIEW_ALWAYS_INLINE bool literal_true()
{
return literal("true", value_t::boolean, node_flags::is_true);
}
NLOHMANN_VIEW_ALWAYS_INLINE bool literal_false()
{
return literal("false", 5, value_t::boolean, 0);
return literal("false", value_t::boolean, 0);
}
NLOHMANN_VIEW_ALWAYS_INLINE bool literal_null()
{
return literal("null", value_t::null, 0);
}
/// skip whitespace (and comments); false on a malformed comment
@@ -798,7 +836,7 @@ indent_done:
const auto n = static_cast<std::size_t>(out - base);
base = cold.grow(n, p);
out = base + n;
cap = base + cold.doc.tape_cap;
cap = base + cold.m_doc.node_capacity;
}
node* n = out++;
#if NLOHMANN_VIEW_LITTLE_ENDIAN
@@ -830,14 +868,15 @@ indent_done:
const frame f = {cur_idx, cur_count, cur_is_object};
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
{
cold.shallow[depth - 1] = f;
cold.m_shallow[depth - 1] = f;
}
else
{
cold.deep.push_back(f);
cold.m_deep.push_back(f);
}
}
++depth;
max_depth = (std::max)(max_depth, depth);
cur_idx = idx;
cur_count = 0;
cur_is_object = k == value_t::object;
@@ -853,12 +892,12 @@ indent_done:
frame f{};
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
{
f = cold.shallow[depth - 1];
f = cold.m_shallow[depth - 1];
}
else
{
f = cold.deep.back();
cold.deep.pop_back();
f = cold.m_deep.back();
cold.m_deep.pop_back();
}
cur_idx = f.idx;
cur_count = f.count;
@@ -866,8 +905,11 @@ indent_done:
}
}
NLOHMANN_VIEW_ALWAYS_INLINE bool literal(const char* text, std::size_t n, value_t k, std::uint8_t flags)
/// the literal `text` at p (N counts the terminating NUL)
template<std::size_t N>
NLOHMANN_VIEW_ALWAYS_INLINE bool literal(const char (&text)[N], value_t k, std::uint8_t flags) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays): a string literal
{
const std::size_t n = N - 1;
if (NLOHMANN_VIEW_UNLIKELY(e - p < static_cast<std::ptrdiff_t>(n) || std::memcmp(p, text, n) != 0))
{
return fail(error_code::invalid_literal);
@@ -946,25 +988,15 @@ indent_done:
value_t kind = value_t::number_float;
if (!is_float)
{
kind = negative ? value_t::number_integer : value_t::number_unsigned;
}
if (!is_float)
{
// integers that do not fit become floats, as in parse()
if (NLOHMANN_VIEW_UNLIKELY(int_digits >= 19))
if (negative)
{
if (negative)
{
if (int_digits > 19 || (int_digits == 19 && digits_exceed(int_start, "9223372036854775808", 19)))
{
kind = value_t::number_float;
}
}
else if (int_digits > 20 || (int_digits == 20 && digits_exceed(int_start, "18446744073709551615", 20)))
{
kind = value_t::number_float;
}
kind = digits_exceed(int_start, int_digits, "9223372036854775808") ? value_t::number_float : value_t::number_integer;
}
else
{
kind = digits_exceed(int_start, int_digits, "18446744073709551615") ? value_t::number_float : value_t::number_unsigned;
}
}
// parse() rejects floats that overflow; only numbers whose magnitude
@@ -985,6 +1017,9 @@ indent_done:
// integers are converted now, while their digits are in cache
const std::uint64_t m = int_digits <= 19 ? parse_upto19(int_start, static_cast<unsigned>(int_digits), e)
: (parse_upto19(int_start, 19, e) * 10) + static_cast<std::uint64_t>(int_start[19] - '0');
// (0 - m rather than -m: unary minus on an unsigned type is
// MSVC's C4146, an error under /sdl; both give the bit pattern
// of the negative std::int64_t)
second = negative ? 0 - m : m;
}
emit(kind, 0, layout, static_cast<std::size_t>(s - b), second);
@@ -1016,7 +1051,7 @@ indent_done:
};
/// run the builder with compile-time options
template<typename FloatType, bool NulIsEnd, bool Comments, bool TrailingCommas>
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd>
inline bool build_with(document_data& d, const char* src, std::size_t size, bool sentinel, parse_failure& failure)
{
if (sentinel)
@@ -1039,11 +1074,11 @@ inline bool build(document_data& d, const char* src, std::size_t size, bool comm
{
if (comments)
{
return trailing_commas ? build_with<FloatType, NulIsEnd, true, true>(d, src, size, sentinel, failure)
: build_with<FloatType, NulIsEnd, true, false>(d, src, size, sentinel, failure);
return trailing_commas ? build_with<FloatType, true, true, NulIsEnd>(d, src, size, sentinel, failure)
: build_with<FloatType, true, false, NulIsEnd>(d, src, size, sentinel, failure);
}
return trailing_commas ? build_with<FloatType, NulIsEnd, false, true>(d, src, size, sentinel, failure)
: build_with<FloatType, NulIsEnd, false, false>(d, src, size, sentinel, failure);
return trailing_commas ? build_with<FloatType, false, true, NulIsEnd>(d, src, size, sentinel, failure)
: build_with<FloatType, false, false, NulIsEnd>(d, src, size, sentinel, failure);
}
} // namespace view
+33 -28
View File
@@ -14,6 +14,7 @@
#include <limits> // numeric_limits
#include <new> // bad_alloc, operator new, placement new
#include <string> // string
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
@@ -31,28 +32,32 @@ struct document_data
{
const char* src = nullptr;
std::size_t size = 0;
node* tape = nullptr;
std::size_t tape_size = 0;
std::size_t tape_cap = 0;
node* inline_tape = nullptr; ///< node array allocated together with this header
std::size_t inline_cap = 0;
std::string arena; ///< decoded strings that contained escapes
std::string owned; ///< owned copy of the input, if any
node* nodes = nullptr; ///< the node index (see node.hpp)
std::size_t node_count = 0;
std::size_t node_capacity = 0;
node* inline_nodes = nullptr; ///< node array allocated together with this header
std::size_t inline_capacity = 0;
std::size_t max_depth = 0; ///< deepest nesting of arrays and objects (the root container is depth 1)
std::vector<char> arena; ///< decoded strings that contained escapes (no terminator: they are handed out with their length)
std::string owned; ///< owned copy of the input, if any (a std::string, so that its NUL serves as the parser's sentinel)
std::array<const char*, 4> base = {{nullptr, nullptr, nullptr, nullptr}}; ///< string bases: source, arena (indexed by flags & node_flags::storage)
bool discarded = true;
/// one allocation for the header and room for `nodes` nodes; large
/// documents get a separate node array instead (so it can be trimmed)
static document_data* create(std::size_t nodes)
/// One allocation for the header and room for `count` nodes, so that a
/// small document needs one allocation instead of two. Large documents get
/// a separate node array instead (so it can be trimmed). When a document
/// outgrows the inline array, that array stays unused: at most 256 nodes
/// (4 KiB), and only for documents whose estimate was too small.
static document_data* create(std::size_t count)
{
nodes = nodes <= 256 ? nodes : 0;
void* mem = ::operator new (sizeof(document_data) + (nodes * sizeof(node)));
count = count <= 256 ? count : 0;
void* mem = ::operator new (sizeof(document_data) + (count * sizeof(node)));
auto* d = new (mem) document_data(); // NOLINT(cppcoreguidelines-owning-memory): owned by the returned pointer, freed by deleter
// (aligned: sizeof is a multiple of the alignment; through void*, as GCC's -Wcast-align wants)
d->inline_tape = static_cast<node*>(static_cast<void*>(static_cast<char*>(mem) + sizeof(document_data))); // NOLINT(bugprone-casting-through-void)
d->inline_cap = nodes;
d->tape = d->inline_tape;
d->tape_cap = nodes;
d->inline_nodes = static_cast<node*>(static_cast<void*>(static_cast<char*>(mem) + sizeof(document_data))); // NOLINT(bugprone-casting-through-void)
d->inline_capacity = count;
d->nodes = d->inline_nodes;
d->node_capacity = count;
return d;
}
@@ -71,8 +76,8 @@ struct document_data
/// implicit one); they cannot take default member initializers, which old
/// Clang (3.4-3.6) rejects. The other members have default member
/// initializers. noexcept: create() constructs into raw memory and could not
/// release it if this threw (the std::string default constructors do not
/// allocate).
/// release it if this threw (the std::vector and std::string default
/// constructors do not allocate).
document_data() noexcept
: arena() // NOLINT(readability-redundant-member-init)
, owned() // NOLINT(readability-redundant-member-init)
@@ -88,12 +93,12 @@ struct document_data
void release() noexcept
{
if (tape != inline_tape)
if (nodes != inline_nodes)
{
::operator delete (tape);
::operator delete (nodes);
}
tape = inline_tape;
tape_cap = inline_cap;
nodes = inline_nodes;
node_capacity = inline_capacity;
}
/// the largest node count whose size in bytes fits a std::size_t
@@ -107,12 +112,12 @@ struct document_data
NLOHMANN_VIEW_THROW(std::bad_alloc());
}
/// make room for n nodes; keeps the first tape_size nodes (throws
/// make room for n nodes; keeps the first node_count nodes (throws
/// std::bad_alloc for a count that does not fit the address space,
/// instead of wrapping around in n * sizeof(node))
void reserve(std::size_t n)
{
if (n <= tape_cap)
if (n <= node_capacity)
{
return;
}
@@ -121,13 +126,13 @@ struct document_data
throw_bad_alloc();
}
node* fresh = static_cast<node*>(::operator new (n * sizeof(node)));
if (tape_size != 0)
if (node_count != 0)
{
std::memcpy(fresh, tape, tape_size * sizeof(node));
std::memcpy(fresh, nodes, node_count * sizeof(node));
}
release();
tape = fresh;
tape_cap = n;
nodes = fresh;
node_capacity = n;
}
const char* str(const node& n) const noexcept
+87 -4
View File
@@ -40,14 +40,96 @@ namespace view
NLOHMANN_VIEW_THROW(invalid_iterator::create(id, msg, nullptr));
}
/// SAX handler of the error replay: accepts every event without storing it,
/// and throws the exception the parser reports
template<typename BasicJsonType>
struct throwing_sax
{
using number_integer_t = typename BasicJsonType::number_integer_t;
using number_unsigned_t = typename BasicJsonType::number_unsigned_t;
using number_float_t = typename BasicJsonType::number_float_t;
using string_t = typename BasicJsonType::string_t;
using binary_t = typename BasicJsonType::binary_t;
bool null()
{
return true;
}
bool boolean(bool /*unused*/)
{
return true;
}
bool number_integer(number_integer_t /*unused*/)
{
return true;
}
bool number_unsigned(number_unsigned_t /*unused*/)
{
return true;
}
bool number_float(number_float_t /*unused*/, const string_t& /*unused*/)
{
return true;
}
bool string(string_t& /*unused*/)
{
return true;
}
bool binary(binary_t& /*unused*/)
{
return true;
}
bool start_object(std::size_t /*unused*/)
{
return true;
}
bool key(string_t& /*unused*/)
{
return true;
}
bool end_object()
{
return true;
}
bool start_array(std::size_t /*unused*/)
{
return true;
}
bool end_array()
{
return true;
}
/// (a template, as in json_sax_dom_parser: the parser passes the concrete
/// exception type, which is thrown as is)
template<class Exception>
[[noreturn]] bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const Exception& ex)
{
NLOHMANN_VIEW_THROW(ex);
}
};
/*!
@brief throw the exception BasicJsonType::parse would throw for this input
The view accepts exactly the inputs parse() accepts, so on a failure the
library parser is run on the same bytes: it throws the exception parse() would
throw, with the same message, position, and "last read" token. The error path
is cold, so this costs nothing on valid input. Should parse() accept the input
nevertheless (a bug), the view's own failure is reported.
throw, with the same message, position, and "last read" token. The replay
uses a SAX handler that stores nothing, so it only lexes and parses up to the
error, without building values. The error path is cold, so this costs nothing
on valid input. Should parse() accept the input nevertheless (a bug), the
view's own failure is reported.
*/
template<typename BasicJsonType>
[[noreturn]] NLOHMANN_VIEW_NOINLINE void throw_parse_failure(const parse_failure& f, const char* src, std::size_t size,
@@ -58,7 +140,8 @@ template<typename BasicJsonType>
// (the limit is detail::view::max_input_size(): 4 GiB minus 16 bytes)
NLOHMANN_VIEW_THROW(out_of_range::create(416, "input of 4294967280 bytes or more is not supported by json_document", nullptr));
}
const BasicJsonType accepted = BasicJsonType::parse(src, src + size, nullptr, true, ignore_comments, ignore_trailing_commas);
throwing_sax<BasicJsonType> sax;
const bool accepted = BasicJsonType::sax_parse(src, src + size, &sax, input_format_t::json, true, ignore_comments, ignore_trailing_commas);
// LCOV_EXCL_START (only if parse() accepts what the view rejects: a bug)
static_cast<void>(accepted);
+78 -13
View File
@@ -8,8 +8,11 @@
#pragma once
#include <algorithm> // max
#include <cstddef> // size_t
#include <istream> // istream, streambuf
#include <string> // basic_string, char_traits, string
#include <type_traits> // decay, integral_constant, is_array, is_const, is_integral, is_lvalue_reference, is_pointer, is_same, remove_reference
#include <type_traits> // decay, integral_constant, is_array, is_const, is_integral, is_lvalue_reference, is_pointer, is_same, remove_cv, remove_extent, remove_pointer, remove_reference
#include <utility> // forward
#include <nlohmann/json.hpp>
@@ -28,19 +31,31 @@ namespace view
/// how a document takes its input
enum class input_kind
{
move_string, ///< non-const rvalue std::string: owned without a copy
c_string, ///< const char* (NUL-terminated): borrowed
char_array, ///< char array (e.g. a string literal): borrowed
borrow_range, ///< lvalue contiguous byte container, or std::string_view: borrowed
copy_range, ///< rvalue contiguous byte container (a const rvalue std::string too): copied
adapter, ///< streams, wide strings, and the rest of what the library's input adapter reads: read into a buffer
move_string, ///< non-const rvalue std::string: owned without a copy
c_string, ///< pointer to single-byte characters (NUL-terminated): borrowed
char_array, ///< array of single-byte characters (e.g. a string literal): borrowed
borrow_range, ///< lvalue contiguous byte container, or std::string_view: borrowed
copy_range, ///< rvalue contiguous byte container (a const rvalue std::string too): copied
wide_c_string, ///< pointer to wide characters (NUL-terminated): converted to UTF-8 into a buffer
adapter, ///< streams, FILE*, wide strings and arrays, and the rest of what the library's input adapter reads: read into a buffer
};
/// a single-byte character type (char, signed/unsigned char, char8_t, std::uint8_t, ...)
template<typename T>
struct is_byte_char : std::integral_constant < bool, std::is_integral<T>::value && sizeof(T) == 1 > {}; // NOLINT(bugprone-sizeof-expression): T is any pointee or element type
/// a character type the library's input adapter converts to UTF-8
template<typename T>
struct is_wide_char : std::integral_constant < bool, std::is_same<T, wchar_t>::value || std::is_same<T, char16_t>::value
|| std::is_same<T, char32_t>::value > {};
template<typename InputType>
struct classify_input
{
using R = typename std::remove_reference<InputType>::type;
using D = typename std::decay<InputType>::type;
using element = typename std::remove_cv<typename std::remove_extent<R>::type>::type;
using pointee = typename std::remove_cv<typename std::remove_pointer<D>::type>::type;
static constexpr bool is_rvalue = !std::is_lvalue_reference<InputType>::value;
static constexpr bool is_bytes = is_contiguous_byte_container<D>::value;
#if NLOHMANN_VIEW_HAS_CPP_17
@@ -49,9 +64,14 @@ struct classify_input
static constexpr bool is_string_view = false;
#endif
// NOLINTBEGIN(readability-avoid-nested-conditional-operator): a constant expression of C++11
// (arrays and pointers of other types, such as wide-character arrays and
// FILE*, go to the library's input adapter)
static constexpr input_kind value =
std::is_array<R>::value ? input_kind::char_array
: std::is_pointer<D>::value ? input_kind::c_string
(std::is_array<R>::value && is_byte_char<element>::value) ? input_kind::char_array
: std::is_array<R>::value ? input_kind::adapter
: (std::is_pointer<D>::value && is_byte_char<pointee>::value) ? input_kind::c_string
: (std::is_pointer<D>::value && is_wide_char<pointee>::value) ? input_kind::wide_c_string
: std::is_pointer<D>::value ? input_kind::adapter
: (is_rvalue && !std::is_const<R>::value && std::is_same<D, std::string>::value) ? input_kind::move_string
: (is_bytes && (!is_rvalue || is_string_view)) ? input_kind::borrow_range
: is_bytes ? input_kind::copy_range
@@ -73,11 +93,10 @@ struct is_std_string : std::false_type {};
template<typename Traits, typename Alloc>
struct is_std_string<std::basic_string<char, Traits, Alloc>> : std::true_type {};
/// drain a json input adapter (UTF-16/32 inputs arrive as UTF-8)
/// append what a json input adapter yields (UTF-16/32 inputs arrive as UTF-8)
template<typename Adapter>
std::string collect_adapter(Adapter ia)
void append_adapter(std::string& buf, Adapter ia)
{
std::string buf;
for (;;)
{
const auto ch = ia.get_character();
@@ -87,7 +106,53 @@ std::string collect_adapter(Adapter ia)
}
buf.push_back(static_cast<char>(ch));
}
return buf;
}
/// an input adapter made by the document for an input that the library's
/// input_adapter() does not take as one argument (an iterator range, a wide C
/// string), to be appended with append_adapter()
template<typename Adapter>
struct adapter_input
{
Adapter ia;
};
template<typename T>
struct is_adapter_input : std::false_type {};
template<typename Adapter>
struct is_adapter_input<adapter_input<Adapter>> : std::true_type {};
template<typename Adapter>
adapter_input<typename std::decay<Adapter>::type> make_adapter_input(Adapter&& ia)
{
return {std::forward<Adapter>(ia)};
}
/// append the rest of a stream: its characters need no conversion, so they are
/// read in blocks from the stream buffer, straight into buf (reusing its
/// capacity). As the library's stream adapter, this reads the stream buffer
/// directly and sets eofbit at the end.
inline void append_stream(std::string& buf, std::istream& is)
{
std::streambuf* const sb = is.rdbuf();
if (sb == nullptr)
{
return;
}
for (;;)
{
const std::size_t used = buf.size();
const std::size_t room = (std::max)(buf.capacity() - used, std::size_t{4096});
buf.resize(used + room);
const std::streamsize got = sb->sgetn(&buf[used], static_cast<std::streamsize>(room));
buf.resize(used + static_cast<std::size_t>(got > 0 ? got : 0));
if (got <= 0)
{
break;
}
}
is.clear(is.rdstate() | std::ios::eofbit);
}
} // namespace view
+21 -8
View File
@@ -8,6 +8,8 @@
#pragma once
#include <algorithm> // max
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t
#include <string> // string
#include <vector> // vector
@@ -43,8 +45,15 @@ BasicJsonType materialize(const document_data& d, const node* n)
BasicJsonType result;
sax_t sax(result, true);
const string_t no_token{};
// the ends of the open containers, and whether they are objects
std::vector<std::pair<const node*, bool>> open;
// the ends of the open containers, and whether they are objects; sized
// once from the document's deepest nesting (for a subtree an upper bound)
struct open_container
{
const node* end;
bool is_object;
};
std::vector<open_container> open((std::max)(d.max_depth, std::size_t{1}));
std::size_t depth = 0;
for (;;)
{
switch (static_cast<value_t>(n->kind))
@@ -61,7 +70,11 @@ BasicJsonType materialize(const document_data& d, const node* n)
{
sax.start_array(n->len);
}
open.emplace_back(document_data::child_end(n), object);
if (NLOHMANN_VIEW_UNLIKELY(depth == open.size()))
{
open.resize(depth * 2); // (deeper than the document's recorded nesting)
}
open[depth++] = open_container{document_data::child_end(n), object};
n = document_data::first_child(n);
break;
}
@@ -98,15 +111,15 @@ BasicJsonType materialize(const document_data& d, const node* n)
}
for (;;)
{
if (open.empty())
if (depth == 0)
{
return result;
}
if (n != open.back().first)
if (n != open[depth - 1].end)
{
break;
}
if (open.back().second)
if (open[depth - 1].is_object)
{
sax.end_object();
}
@@ -114,9 +127,9 @@ BasicJsonType materialize(const document_data& d, const node* n)
{
sax.end_array();
}
open.pop_back();
--depth;
}
if (open.back().second)
if (open[depth - 1].is_object)
{
// the key of the next member
string_t key(d.str(*n), n->len);
+8 -8
View File
@@ -21,14 +21,6 @@ namespace detail
namespace view
{
// the node kinds are value_t values; the tests of is_container() and of the
// number kinds depend on this numbering
static_assert(static_cast<std::uint8_t>(value_t::null) == 0 && static_cast<std::uint8_t>(value_t::object) == 1
&& static_cast<std::uint8_t>(value_t::array) == 2 && static_cast<std::uint8_t>(value_t::string) == 3
&& static_cast<std::uint8_t>(value_t::boolean) == 4 && static_cast<std::uint8_t>(value_t::number_integer) == 5
&& static_cast<std::uint8_t>(value_t::number_unsigned) == 6 && static_cast<std::uint8_t>(value_t::number_float) == 7,
"the node format depends on the numbering of value_t");
/// The largest input a document accepts, in bytes. Offsets and node counts are
/// 32 bits wide; the limit keeps 16 bytes (the width of the scanner's steps)
/// below 2^32, so that a position one step past the end of the text fits.
@@ -60,6 +52,14 @@ struct node
};
static_assert(sizeof(node) == 16, "node must stay 16 bytes");
// the node kinds are value_t values; the test below (object and array are 1
// and 2) and those of the number kinds depend on this numbering
static_assert(static_cast<std::uint8_t>(value_t::null) == 0 && static_cast<std::uint8_t>(value_t::object) == 1
&& static_cast<std::uint8_t>(value_t::array) == 2 && static_cast<std::uint8_t>(value_t::string) == 3
&& static_cast<std::uint8_t>(value_t::boolean) == 4 && static_cast<std::uint8_t>(value_t::number_integer) == 5
&& static_cast<std::uint8_t>(value_t::number_unsigned) == 6 && static_cast<std::uint8_t>(value_t::number_float) == 7,
"the node format depends on the numbering of value_t");
NLOHMANN_VIEW_ALWAYS_INLINE bool is_container(const node& n) noexcept
{
return static_cast<unsigned>(n.kind) - 1u <= 1u;
+20 -3
View File
@@ -139,9 +139,26 @@ inline std::uint64_t int_pow10(unsigned k) noexcept
static const std::array<std::uint64_t, 20> table =
{
{
1u, 10u, 100u, 1000u, 10000u, 100000u, 1000000u, 10000000u, 100000000u, 1000000000u,
10000000000u, 100000000000u, 1000000000000u, 10000000000000u, 100000000000000u, 1000000000000000u,
10000000000000000u, 100000000000000000u, 1000000000000000000u, 10000000000000000000u
1u,
10u,
100u,
1000u,
10000u,
100000u,
1000000u,
10000000u,
100000000u,
1000000000u,
10000000000u,
100000000000u,
1000000000000u,
10000000000000u,
100000000000000u,
1000000000000000u,
10000000000000000u,
100000000000000000u,
1000000000000000000u,
10000000000000000000u
}
};
return table[k];
+90 -32
View File
@@ -27,11 +27,13 @@
#include <array> // array
#include <cstddef> // nullptr_t, size_t // IWYU pragma: keep
#include <cstring> // memcpy, strlen
#include <istream> // istream
#include <iterator> // distance, input_iterator_tag, iterator_traits
#include <memory> // unique_ptr
#include <string> // string
#include <type_traits> // enable_if, integral_constant, is_base_of, is_integral, is_same, remove_cv, remove_extent
#include <type_traits> // decay, enable_if, integral_constant, is_base_of, is_integral, is_same, remove_cv, remove_extent
#include <utility> // forward, move
#include <vector> // vector
#include <nlohmann/json.hpp> // IWYU pragma: export
@@ -327,7 +329,9 @@ class basic_json_document
const bool ignore_trailing_commas = false)
{
basic_json_document d;
d.build_owned(collect(std::forward<InputType>(input)), allow_exceptions, ignore_comments, ignore_trailing_commas);
std::string buf;
append_input(buf, std::forward<InputType>(input));
d.build_owned(std::move(buf), allow_exceptions, ignore_comments, ignore_trailing_commas);
return d;
}
@@ -376,7 +380,7 @@ class basic_json_document
{
return view_type();
}
return view_type(m_data.get(), m_data->tape);
return view_type(m_data.get(), m_data->nodes);
}
/// deleted: the view of a temporary document would dangle
@@ -402,7 +406,7 @@ class basic_json_document
/// number of index nodes (values plus object keys)
std::size_t node_count() const noexcept
{
return m_data ? m_data->tape_size : 0;
return m_data ? m_data->node_count : 0;
}
/// bytes held by the document (index, decoded strings, owned text)
@@ -412,8 +416,8 @@ class basic_json_document
{
return 0;
}
return sizeof(document_data) + (m_data->inline_cap * sizeof(detail::view::node))
+ (m_data->tape != m_data->inline_tape ? m_data->tape_cap * sizeof(detail::view::node) : 0)
return sizeof(document_data) + (m_data->inline_capacity * sizeof(detail::view::node))
+ (m_data->nodes != m_data->inline_nodes ? m_data->node_capacity * sizeof(detail::view::node) : 0)
+ m_data->arena.capacity() + m_data->owned.capacity();
}
@@ -432,17 +436,17 @@ class basic_json_document
// allocate everything first, so that an exception leaves the document
// unchanged
const bool shrink_arena = d.arena.capacity() > d.arena.size();
std::string arena(shrink_arena ? d.arena : std::string());
const bool shrink_tape = d.tape != d.inline_tape && d.tape_size != d.tape_cap;
const bool into_header = d.tape_size <= d.inline_cap;
node* fresh = (shrink_tape && !into_header) ? static_cast<node*>(::operator new (d.tape_size * sizeof(node))) : d.inline_tape;
std::vector<char> arena(shrink_arena ? d.arena : std::vector<char>());
const bool shrink_nodes = d.nodes != d.inline_nodes && d.node_count != d.node_capacity;
const bool into_header = d.node_count <= d.inline_capacity;
node* fresh = (shrink_nodes && !into_header) ? static_cast<node*>(::operator new (d.node_count * sizeof(node))) : d.inline_nodes;
if (shrink_tape)
if (shrink_nodes)
{
std::memcpy(fresh, d.tape, d.tape_size * sizeof(node));
::operator delete (d.tape);
d.tape = fresh;
d.tape_cap = into_header ? d.inline_cap : d.tape_size;
std::memcpy(fresh, d.nodes, d.node_count * sizeof(node));
::operator delete (d.nodes);
d.nodes = fresh;
d.node_capacity = into_header ? d.inline_capacity : d.node_count;
}
if (shrink_arena)
{
@@ -463,7 +467,8 @@ class basic_json_document
}
}
/// parse a buffer the document takes ownership of
/// parse a buffer the document takes ownership of (it replaces the
/// previous owned copy)
void build_owned(std::string&& buf, bool allow_exceptions, bool comments, bool trailing_commas)
{
ensure_data(buf.data(), buf.size());
@@ -471,6 +476,29 @@ class basic_json_document
build(m_data->owned.data(), m_data->owned.size(), allow_exceptions, comments, trailing_commas, true, true);
}
/// parse a copy of the input; the copy is appended to the cleared owned
/// buffer, so that its capacity is reused
template<typename T>
void build_copy(T&& input, bool allow_exceptions, bool comments, bool trailing_commas)
{
if (!m_data)
{
std::string buf;
append_input(buf, std::forward<T>(input));
build_owned(std::move(buf), allow_exceptions, comments, trailing_commas);
return;
}
document_data& d = *m_data;
// (the previous text is gone from here on: should reading the input
// throw, the document is discarded)
d.discarded = true;
d.src = "";
d.size = 0;
d.owned.clear();
append_input(d.owned, std::forward<T>(input));
build(d.owned.data(), d.owned.size(), allow_exceptions, comments, trailing_commas, true, true);
}
/// sentinel: src[size] is readable and 0 (std::string, C strings)
void build(const char* src, std::size_t size, bool allow_exceptions, bool comments, bool trailing_commas, bool owned, bool sentinel)
{
@@ -482,7 +510,7 @@ class basic_json_document
}
d.src = src;
d.size = size;
d.tape_size = 0;
d.node_count = 0;
d.arena.clear();
d.discarded = true;
detail::view::parse_failure failure;
@@ -518,7 +546,6 @@ class basic_json_document
template<typename CharT>
void read_kind(CharT* s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::c_string> /*unused*/)
{
static_assert(sizeof(CharT) == 1 && std::is_integral<typename std::remove_cv<CharT>::type>::value, "json_view parses byte (char-like) input");
if (s == nullptr)
{
build("", 0, ae, c, tc, false, true);
@@ -534,7 +561,6 @@ class basic_json_document
void read_kind(Array& a, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::char_array> /*unused*/)
{
using CharT = typename std::remove_cv<typename std::remove_extent<Array>::type>::type;
static_assert(sizeof(CharT) == 1 && std::is_integral<CharT>::value, "json_view parses byte (char-like) input");
const std::size_t n = std::extent<Array>::value;
// a trailing NUL (string literals) is not part of the text, as for
// parse(), and serves as sentinel
@@ -554,13 +580,25 @@ class basic_json_document
template<typename T>
void read_kind(const T& s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::copy_range> /*unused*/)
{
build_owned(std::string(reinterpret_cast<const char*>(s.data()), static_cast<std::size_t>(s.size())), ae, c, tc); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
build_copy(s, ae, c, tc);
}
template<typename CharT>
void read_kind(CharT* s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::wide_c_string> /*unused*/)
{
if (s == nullptr)
{
build("", 0, ae, c, tc, false, true);
return;
}
// NUL-terminated, as for the single-byte C strings
build_copy(detail::view::make_adapter_input(detail::input_adapter(s, s + std::char_traits<typename std::remove_cv<CharT>::type>::length(s))), ae, c, tc);
}
template<typename T>
void read_kind(T&& input, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::adapter> /*unused*/)
{
build_owned(detail::view::collect_adapter(detail::input_adapter(std::forward<T>(input))), ae, c, tc);
build_copy(std::forward<T>(input), ae, c, tc);
}
template<typename IteratorType>
@@ -584,25 +622,45 @@ class basic_json_document
template<typename IteratorType>
void read_range_impl(IteratorType first, IteratorType last, bool ae, bool c, bool tc, std::false_type /*other*/)
{
build_owned(detail::view::collect_adapter(detail::input_adapter(first, last)), ae, c, tc);
build_copy(detail::view::make_adapter_input(detail::input_adapter(first, last)), ae, c, tc);
}
/// append the text of an input to buf: a contiguous byte input as one
/// block, a stream in blocks, an input adapter (made here for any other
/// input) character by character
template<typename T>
static void append_input(std::string& buf, T&& input)
{
using D = typename std::decay<T>::type;
// NOLINTBEGIN(readability-avoid-nested-conditional-operator): a constant expression of C++11
append_input_impl(buf, std::forward<T>(input), std::integral_constant < int, detail::is_contiguous_byte_container<D>::value ? 0
: std::is_base_of<std::istream, D>::value ? 1
: detail::view::is_adapter_input<D>::value ? 2 : 3 > {});
// NOLINTEND(readability-avoid-nested-conditional-operator)
}
template<typename T>
static std::string collect(T&& input)
static void append_input_impl(std::string& buf, const T& s, std::integral_constant<int, 0> /*contiguous bytes*/)
{
return collect_impl(std::forward<T>(input), std::integral_constant<bool, detail::is_contiguous_byte_container<typename std::decay<T>::type>::value> {});
buf.append(reinterpret_cast<const char*>(s.data()), static_cast<std::size_t>(s.size())); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
template<typename Stream>
static void append_input_impl(std::string& buf, Stream&& is, std::integral_constant<int, 1> /*stream*/) // NOLINT(cppcoreguidelines-missing-std-forward): read in place, as by the library's stream adapter
{
detail::view::append_stream(buf, is);
}
template<typename Adapter>
static void append_input_impl(std::string& buf, detail::view::adapter_input<Adapter>&& input, std::integral_constant<int, 2> /*input adapter*/)
{
detail::view::append_adapter(buf, std::move(input.ia));
}
template<typename T>
static std::string collect_impl(const T& s, std::true_type /*contiguous*/)
static void append_input_impl(std::string& buf, T&& input, std::integral_constant<int, 3> /*other*/)
{
return {reinterpret_cast<const char*>(s.data()), static_cast<std::size_t>(s.size())}; // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
template<typename T>
static std::string collect_impl(T&& s, std::false_type /*other*/)
{
return detail::view::collect_adapter(detail::input_adapter(std::forward<T>(s)));
detail::view::append_adapter(buf, detail::input_adapter(std::forward<T>(input)));
}
std::unique_ptr<document_data, document_data::deleter> m_data{}; // NOLINT(readability-redundant-member-init)
+508 -232
View File
File diff suppressed because it is too large. Load diff
+109
View File
@@ -17,6 +17,9 @@ using nlohmann::ordered_json_document;
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <ios>
#include <istream>
#include <list>
#include <map>
#include <random>
@@ -396,6 +399,112 @@ TEST_CASE("json_view")
CHECK(materialized<json_document>(wide) == json::parse(wide));
CHECK(json_document::parse(static_cast<const char*>(nullptr), false).is_discarded());
CHECK(json_document::parse("", false).is_discarded());
// wide-character arrays and pointers are converted to UTF-8, as by
// parse(); the pointers are NUL-terminated
CHECK(materialized<json_document>(L"[\"ä€\", 1]") == json::parse(L"[\"ä€\", 1]"));
CHECK(materialized<json_document>(u"[\"ä€\", 1]") == json::parse(u"[\"ä€\", 1]"));
CHECK(materialized<json_document>(U"[\"ä€\", 1]") == json::parse(U"[\"ä€\", 1]"));
const wchar_t* const wide_pointer = wide.c_str();
const json_document from_wide_pointer = json_document::parse(wide_pointer);
CHECK(from_wide_pointer.owns_source());
CHECK(from_wide_pointer.root().materialize() == json::parse(wide));
const std::u16string wide16 = u"{\"\U0001F600\": [true]}";
CHECK(materialized<json_document>(wide16.c_str()) == json::parse(wide16));
const std::u32string wide32 = U"{\"\U0001F600\": [true]}";
CHECK(materialized<json_document>(wide32.c_str()) == json::parse(wide32));
CHECK(json_document::parse(static_cast<const wchar_t*>(nullptr), false).is_discarded());
// FILE*: read to its end
std::FILE* file = std::tmpfile(); // NOLINT(cppcoreguidelines-owning-memory)
if (file != nullptr)
{
CHECK(std::fputs(text.c_str(), file) >= 0);
CHECK(std::fseek(file, 0, SEEK_SET) == 0);
const json_document from_file = json_document::parse(file);
CHECK(from_file.owns_source());
CHECK(from_file.root().materialize() == expected);
CHECK(std::fclose(file) == 0); // NOLINT(cppcoreguidelines-owning-memory)
}
}
SECTION("streams")
{
// longer than one block of the bulk read
std::string text = "[";
for (int i = 0; i < 5000; ++i)
{
text += (i != 0 ? ",\"" : "\"") + std::to_string(i) + "\"";
}
text += ']';
std::istringstream stream(text);
const json_document from_stream = json_document::parse(stream);
CHECK(from_stream.source() == text);
CHECK(from_stream.root().materialize() == json::parse(text));
// the stream is read to its end, and eofbit set, as by parse()
CHECK(stream.eof());
CHECK(!stream.bad());
std::istringstream empty_stream;
CHECK(json_document::parse(empty_stream, false).is_discarded());
CHECK(empty_stream.eof());
std::istringstream rvalue_source("[1]");
CHECK(json_document::parse_copy(std::move(rvalue_source)).owns_source());
// a stream without a buffer is an empty input
std::istream no_buffer(nullptr);
CHECK(json_document::parse(no_buffer, false).is_discarded());
#if !defined(JSON_NOEXCEPTION)
// an exception while reading the input leaves a discarded document
json_document d = json_document::parse(std::vector<char>(text.begin(), text.end()));
std::istringstream throwing("[1]");
throwing.exceptions(std::ios::eofbit);
CHECK_THROWS_AS(d.read(throwing), std::ios_base::failure&);
CHECK(d.is_discarded());
CHECK(d.root().is_discarded());
d.read("[2]");
CHECK(d.root().materialize() == json::parse("[2]"));
#endif
}
SECTION("read() reuses the owned copy")
{
std::string big = "[";
for (int i = 0; i < 1000; ++i)
{
big += (i != 0 ? ",\"" : "\"") + std::to_string(i) + "\"";
}
big += ']';
const std::string small = "{\"a\": [1, 2]}";
// copied inputs are written into the existing buffer
json_document d = json_document::parse(std::vector<char>(big.begin(), big.end()));
REQUIRE(d.owns_source());
const char* const buffer = d.source().data();
d.read(std::vector<char>(small.begin(), small.end()));
CHECK(d.owns_source());
CHECK(d.source().data() == buffer);
CHECK(d.root().materialize() == json::parse(small));
std::istringstream stream(small);
d.read(stream);
CHECK(d.source().data() == buffer);
CHECK(d.root().materialize() == json::parse(small));
d.read(std::wstring(L"[3]"));
CHECK(d.source().data() == buffer);
CHECK(d.root().materialize() == json::parse("[3]"));
// a moved-in string replaces it (long enough to be on the heap)
std::string moved = big;
const char* const moved_buffer = moved.data();
d.read(std::move(moved));
CHECK(d.source().data() == moved_buffer);
CHECK(d.root().materialize() == json::parse(big));
// a borrowed input leaves nothing owned
d.read(big);
CHECK(!d.owns_source());
CHECK(d.source().data() == big.data());
}
SECTION("integer arguments do not compile")
+27 -14
View File
@@ -113,9 +113,9 @@ json value_of(const document_data& d, const node*& n)
json value_of(const built& b)
{
const node* n = b.data->tape;
const node* n = b.data->nodes;
json v = value_of(*b.data, n);
CHECK(n == b.data->tape + b.data->tape_size);
CHECK(n == b.data->nodes + b.data->node_count);
return v;
}
@@ -340,6 +340,17 @@ TEST_CASE("json_view builder")
SECTION("deep nesting")
{
// the deepest nesting is recorded (materialize() sizes its stack from it)
CHECK(build("1", false, false, false).data->max_depth == 0);
CHECK(build("[]", false, false, false).data->max_depth == 1);
CHECK(build(R"([[1],{"a":[[]]},[]])", false, false, false).data->max_depth == 4);
// a stack that turns out too small still grows
const std::string nested = R"([[1],{"a":[[{"b":[]}]]},[]])";
const built shallow = build(nested, false, false, false);
REQUIRE(shallow.ok);
shallow.data->max_depth = 1;
CHECK(nlohmann::detail::view::materialize<json>(*shallow.data, shallow.data->nodes) == json::parse(nested));
// the open containers beyond 64 levels live on the heap
for (const std::size_t depth :
{
@@ -349,8 +360,9 @@ TEST_CASE("json_view builder")
const std::string arrays = std::string(depth, '[') + std::string(depth, ']');
const built b = build(arrays, false, false, false);
REQUIRE(b.ok);
CHECK(b.data->tape_size == depth);
CHECK(b.data->tape[0].next == depth);
CHECK(b.data->node_count == depth);
CHECK(b.data->nodes[0].next == depth);
CHECK(b.data->max_depth == depth);
std::string objects;
for (std::size_t i = 0; i < depth; ++i)
{
@@ -359,7 +371,8 @@ TEST_CASE("json_view builder")
objects += '1' + std::string(depth, '}');
const built o = build(objects, false, false, false);
REQUIRE(o.ok);
CHECK(o.data->tape_size == (2 * depth) + 1);
CHECK(o.data->node_count == (2 * depth) + 1);
CHECK(o.data->max_depth == depth);
CHECK(!build(std::string(depth, '[') + std::string(depth - 1, ']'), false, false, false).ok);
}
}
@@ -454,7 +467,7 @@ TEST_CASE("json_view node integer bits")
CAPTURE(c.text)
const built b = build(c.text, false, false, true);
REQUIRE(b.ok);
const node& n = b.data->tape[1];
const node& n = b.data->nodes[1];
CHECK(integer_bits(n) == c.bits);
CHECK(n.len == static_cast<std::uint32_t>(c.bits));
CHECK(n.next == static_cast<std::uint32_t>(c.bits >> 32));
@@ -469,23 +482,23 @@ TEST_CASE("json_view node array size limit")
// for absurd counts everywhere)
std::unique_ptr<document_data, document_data::deleter> d(document_data::create(0));
d->reserve(8);
REQUIRE(d->tape_cap >= 8);
d->tape_size = 2;
const std::size_t cap = d->tape_cap;
node* const tape = d->tape;
REQUIRE(d->node_capacity >= 8);
d->node_count = 2;
const std::size_t cap = d->node_capacity;
node* const nodes = d->nodes;
#if !defined(JSON_NOEXCEPTION)
const std::size_t too_many = document_data::max_nodes() + 1;
CHECK_THROWS_AS(d->reserve(too_many), std::bad_alloc&);
CHECK_THROWS_AS(d->reserve((std::numeric_limits<std::size_t>::max)()), std::bad_alloc&);
// the array is unchanged
CHECK(d->tape == tape);
CHECK(d->tape_cap == cap);
CHECK(d->tape_size == 2);
CHECK(d->nodes == nodes);
CHECK(d->node_capacity == cap);
CHECK(d->node_count == 2);
#endif
// the largest count that fits is not refused by the check (nothing is
// allocated for a count that is already there)
d->reserve(cap);
CHECK(d->tape == tape);
CHECK(d->nodes == nodes);
}