Parsing from a FILE* reads one character at a time with std::fgetc, which locks and unlocks the FILE object for every character. file_input_adapter now locks the file once when it is created and unlocks it when it is destroyed - also when parsing throws - and reads with the unlocked stdio functions in between: flockfile/getc_unlocked/funlockfile with glibc, Apple, Android, and the BSDs, and _lock_file/_fgetc_nolock/_unlock_file with the Microsoft C runtime. Other C libraries keep std::fgetc. The lock is recursive, so fread() in the binary readers and stdio calls of a SAX handler or parser callback on the same thread keep working. On an Apple M1 Max, parsing from a FILE* is 2.9 to 4.0 times as fast (twitter.json 14.3 -> 4.1 ms, citm_catalog.json 35.4 -> 8.8 ms); with glibc, whose locking is cheaper, 1.1 to 1.3 times. The values and the position of the file after parsing are unchanged. Other threads that use the same FILE object now wait until parsing is done instead of interleaving their reads with the parser's. JSON_DISABLE_UNLOCKED_FILE_IO restores std::fgetc, for programs in which a SAX handler waits for another thread that uses the same FILE. The new tests check from a second thread that the lock is released after success and after a parse error, and run again with the macro. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
7.0 KiB
nlohmann::basic_json::sax_parse
// (1)
template <typename InputType, typename SAX>
static bool sax_parse(InputType&& i,
SAX* sax,
input_format_t format = input_format_t::json,
const bool strict = true,
const bool ignore_comments = false,
const bool ignore_trailing_commas = false,
const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error);
// (2)
template<class IteratorType, class SAX, class SentinelType = IteratorType>
static bool sax_parse(IteratorType first, SentinelType last,
SAX* sax,
input_format_t format = input_format_t::json,
const bool strict = true,
const bool ignore_comments = false,
const bool ignore_trailing_commas = false,
const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error);
Read from input and generate SAX events
-
Read from a compatible input.
-
Read from a pair of character iterators, or an iterator and a sentinel of a different type (C++20 ranges support).
The value_type of the iterator must be an integral type with a size of 1, 2, or 4 bytes, which will be interpreted respectively as UTF-8, UTF-16, and UTF-32. If
SentinelTypediffers fromIteratorType, it must be comparable to the iterator type withoperator!=.
The SAX event lister must follow the interface of json_sax.
Template parameters
InputType- A compatible input, for instance:
- an
std::istreamobject - a
FILEpointer - a C-style array of characters
- a pointer to a null-terminated string of single byte characters
- a container
objfor whichbegin(obj)andend(obj)produce a valid pair of iterators (as found via ADL or member functions, with semantics compatible tostd::beginandstd::end)
- an
IteratorType- a compatible iterator type for overload (2); a pair of character iterators whose
value_typeis an integral type with a size of 1, 2, or 4 bytes (interpreted respectively as UTF-8, UTF-16, and UTF-32) SentinelType- defaults to
IteratorType; may be a different type comparable toIteratorTypeviaoperator!=, for overload (2), for instance.- a custom sentinel type for C++20 ranges
std::default_sentinel_t, whenIteratorTypeisstd::counted_iterator
SAX- a class fulfilling the SAX event listener interface; see
json_sax
Parameters
i(in)- Input to parse from
sax(in)- SAX event listener (must not be null)
format(in)- the format to parse (JSON, BJData, BON8, BSON, CBOR, MessagePack, or UBJSON) (optional,
input_format_t::jsonby default), seeinput_format_tfor more information strict(in)- whether the input has to be consumed completely (optional,
#!cpp trueby default); when#!cpp falseand the input is a#!cpp std::istream, the character that terminates a number is consumed unlessJSON_PRECISE_STREAM_POSITIONis defined to1; seeoperator>> ignore_comments(in)- whether comments should be ignored and treated like whitespace (
#!cpp true) or yield a parse error (#!cpp false); (optional,#!cpp falseby default) ignore_trailing_commas(in)- whether trailing commas in arrays or objects should be ignored and treated like whitespace (
#!cpp true) or yield a parse error (#!cpp false); (optional,#!cpp falseby default) tag_handler(in)- how to handle CBOR tags; see
cbor_tag_handler_t. Ignored for formats other than CBOR (optional,cbor_tag_handler_t::errorby default). first(in)- iterator to the start of a character range
last(in)- iterator to the end of a character range, or a sentinel value that compares equal to the end iterator with
operator!=
Return value
return value of the last processed SAX event
Exception safety
Strong guarantee: if an exception is thrown, there are no changes in the JSON value.
Exceptions
- Throws
parse_error.101in case of an unexpected token, or empty input like a nullFILE*orchar*pointer.
Complexity
Linear in the length of the input. The parser is a predictive LL(1) parser. The complexity can be higher if the SAX
consumer sax has a super-linear complexity.
Notes
A UTF-8 byte order mark is silently ignored.
When reading from a #!c FILE*, the file is locked while it is parsed. The lock is recursive, so the SAX handler may
use the same file with stdio functions, but it must not wait for another thread that uses the same FILE object; see
JSON_DISABLE_UNLOCKED_FILE_IO.
Examples
??? example
The example below demonstrates the `sax_parse()` function reading from string and processing the events with a
user-defined SAX event consumer.
```cpp
--8<-- "examples/sax_parse.cpp"
```
Output:
```json
--8<-- "examples/sax_parse.output"
```
See also
- parse - deserialize from a compatible input
- accept - check if the input is valid JSON
JSON_DISABLE_STREAM_BULK_SCAN- read astd::istreamone character at a timeJSON_DISABLE_UNLOCKED_FILE_IO- read aFILE*with the locking stdio functions
Version history
- Added in version 3.2.0.
- Ignoring comments via
ignore_commentsadded in version 3.9.0. - Added
ignore_trailing_commasin version 3.13.0. - Added
tag_handlerin version 3.13.0. - Extended container support (1) to include types with lvalue-only ADL
begin/end(matchingstd::begin/std::endsemantics) in version 3.13.0. - Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0.
JSON_PRECISE_STREAM_POSITIONadded in version 3.13.0 to optionally leave a#!cpp std::istreampositioned right after the parsed value whenstrictis#!cpp false.- Parsing from an
std::istreamscans the characters in its stream buffer in bulk in version 3.13.0 (JSON_DISABLE_STREAM_BULK_SCAN). - Parsing from a
FILE*locks the file once instead of once per character in version 3.13.0 (JSON_DISABLE_UNLOCKED_FILE_IO).
!!! warning "Deprecation"
Overload (2) replaces calls to `sax_parse` with a pair of iterators as their first parameter which has been
deprecated in version 3.8.0. This overload will be removed in version 4.0.0. Please replace all calls like
`#!cpp sax_parse({ptr, ptr+len});` with `#!cpp sax_parse(ptr, ptr+len);`.
See the [migration guide](../../integration/migration_guide.md#parsing) for how to update existing code.