Read the character following a number without consuming it, instead of
consuming it and putting it back. input_stream_adapter now peeks with
sgetc() and only steps over the character when the next one is requested
or when the adapter is destroyed, so releasing it cannot fail - no
putback position is required from the streambuf.
Suggested by gregmarr in #5344.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Four CI failures, all in the new test code:
- GCC (-Werror=useless-cast): drop the `json(...)` wrapper around
`json::parse(...)`, which already returns a `json`.
- GCC (-Werror=unused-result): assign the discarded `json::parse()`
result to a dummy, the idiom used elsewhere in the test suite, and
catch `json::parse_error&` for consistency.
- clang-tidy (google-default-arguments): remove the default argument
from the `pbackfail()` override; `sungetc()` supplies the base
declaration's default.
- MSVC (bad allocation): `no_putback_streambuf::underflow()` set a
one-character get area without advancing `m_pos`, so an implementation
whose `istream::get` peeks before it bumps re-read the same character
forever. Keep no get area at all: `underflow()` peeks, `uflow()`
consumes, and `sungetc()` still always lands in `pbackfail()`, which
is what the test needs.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
operator>> is documented to leave the stream positioned right after the
parsed value, so that concatenated JSON values can be read back to back.
That did not hold for numbers: a number is only terminated by the
character following it, and lexer::scan_number() reads that character
and calls unget() -- which is simulated and rewinds only the lexer's own
bookkeeping. input_stream_adapter consumes via sbumpc() with no matching
sungetc(), so the terminating character stayed consumed and the next
extraction started one byte too late ('1true' left the stream at 'rue').
Propagating unget() to the adapter directly does not work: next_unget
makes the following get() replay the cached character, so the terminator
would be delivered twice. Instead, restore the still-pending character
once at the end of a non-strict parse, where the input is handed back to
the caller:
- input_stream_adapter gains unget_character() (sungetc()) and advertises
it via supports_unget, detected the same way as supports_seek.
- lexer::restore_pending_unget() turns a pending simulated unget of a
real (non-EOF) character into a real one and clears next_unget so the
character is not also replayed. It is a no-op for adapters that cannot
unget, and reports failure when sungetc() fails, in which case the
input is left as it was before.
- parser calls it on the three non-strict paths, i.e. for operator>> and
sax_parse(strict = false).
Strict parse()/accept() are unaffected: they require the input to end
after the value, so the character is consumed by the end-of-input check
anyway. Parse error messages and reported positions are unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Migrate ci_icpc/ci_test_compilers_gcc_old/ci_infer off custom json-ci image
Replaces the last three consumers of ghcr.io/nlohmann/json-ci with official
images: ci_icpc now uses Intel's own intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04
(the last release with classic icc/icpc before Intel dropped it in oneAPI
2024.0), ci_test_compilers_gcc_old installs old GCCs on official ubuntu:20.04
via the same PPA/archive setup the custom image used (working around
actions/checkout's incompatibility with official gcc:4/5/6 images), and
ci_infer runs directly on ubuntu-latest, fetching Facebook's official Infer
release tarball inline instead of a maintained image. No job in
ubuntu.yml references the custom image anymore.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Update quality-assurance compiler table for CI image migration
Reflects the ci_icpc/ci_test_compilers_gcc_old container migration: the
old-GCC jobs (4.8/4.9/5/6) now compile inside official ubuntu:20.04 rather
than the custom Focal-based json-ci image (same OS, just now attributed to
the official image), and ci_icpc now uses Intel's official
intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04, bumping the reported ICC
version from 2021.5.0 to 2021.10.0 and the OS from Ubuntu 20.04.3 to 22.04.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI failures from the json-ci image migration
- ci_icpc: the official intel/oneapi-hpckit image has no CMake preinstalled
(the custom image bundled one); add the missing lukka/get-cmake step.
- ci_test_compilers_gcc_old: official ubuntu:20.04 has no build tool, so
CMake's default Unix Makefiles generator failed with "CMAKE_MAKE_PROGRAM
is not set"; install make alongside the PPA-provided g++.
- ci_infer: Infer v1.1.0's bundled Clang frontend can't parse GCC 14's
headers (ubuntu-latest's default toolchain), failing with parse errors in
<bits/unicode.h>; bump to the latest release, v1.3.0, whose newer bundled
frontend understands them (release asset also renamed upstream from
infer-linux64-v1.1.0.tar.xz to infer-linux-x86_64-v1.3.0.tar.xz).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_test_compilers_gcc_old: g++-6 missing from xenial-only archives
My inline PPA/archive replication only added the xenial main/universe
suites, but g++-6 isn't available there ("has no installation candidate").
The original custom Dockerfile also pulled from bionic main/universe and
xenial-updates main/universe; add those back to match.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_test_compilers_gcc_old: install git for CMake's FetchContent tests
Official ubuntu:20.04 ships no git at all (actions/checkout only succeeded
via its API-download fallback). The cmake_fetch_content(2) tests invoke
CMake's own ExternalProject_Add, which needs a real git binary and failed
with "could not find git for clone of json-populate". Install git alongside
the other build prerequisites.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_icpc: drop redundant setvars.sh sourcing
Unlike the old custom image, the official intel/oneapi-hpckit image already
has the oneAPI environment (icc/icpc on PATH) baked in at the container
level. Explicitly re-sourcing setvars.sh in the Build step failed with
"setvars.sh has already been run. Skipping re-execution." (exit code 3,
aborting the step under `sh -e`). Drop the now-unnecessary sourcing.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_icpc: exclude classic ICC from the std::span regression test
Bumping to Intel's official intel/oneapi-hpckit:2023.2.1 image (see previous
commit) also bumped classic icc/icpc from 2021.5.0 to 2021.10.0. The newer
version's __has_include(<span>) now returns true, but it still can't
actually compile std::span/std::as_bytes usage:
error: namespace "std" has no member "as_bytes"
error: namespace "std" has no member "span"
Exclude __ICC/__INTEL_COMPILER the same way _LIBCPP_VERSION is already
excluded for issue #4490.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix amalgamation/style check: indent comment per astyle
Verified with the pinned astyle 3.4.13 (make install_astyle) locally;
no further diff.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_icpc: skip UTF-8 u8-literal comparison test on classic ICC
test-deserialization_cpp20 failed:
ERROR: CHECK( j2["emoji"] == "😀" ) is NOT correct!
check_utf8() only guards against MSVC's ANSI-codepage quirk (its docstring
example), but classic ICC has an analogous problem: it doesn't encode a
narrow string literal containing non-ASCII source characters as UTF-8,
so comparing a decoded u8R"(...)" literal against a narrow literal with
the same characters fails. Extend the existing guard.
Verified with the pinned astyle 3.4.13; no diff.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add implementation to retrieve start and end positions of json during parse
* Add more unit tests and add start/stop parsing for arrays
* Add raw value for all types
* Add more tests and fix compiler warning
* Amalgamate
* Fix CLang GCC warnings
* Fix error in build
* Style using astyle 3.1
* Fix whitespace changes
* revert
* more whitespace reverts
* Address PR comments
* Fix failing issues
* More whitespace reverts
* Address remaining PR comments
* Address comments
* Switch to using custom base class instead of default basic_json
* Adding a basic using for a json using the new base class. Also address PR comments and fix CI failures
* Address decltype comments
* Diagnostic positions macro (#4)
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Fix missed include deletion
* Add docs and address other PR comments (#5)
* Add docs and address other PR comments
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address new PR comments and fix CI tests for documentation
* Update documentation based on feedback (#6)
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address std::size_t and other comments
* Fix new CI issues
* Fix lcov
* Improve lcov case with update to handle_diagnostic_positions call for discarded values
* Fix indentation of LCOV_EXCL_STOP comments
* fix amalgamation astyle issue
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Possible fix for #4485
Throw's an exception when i is nullptr,
also added a testcase for this scenario though most likely in the wrong test file.cpp
* quick cleanup
* Fix compile issues
* moved tests around, changed exceptions, removed a possibly unneeded include
* add back include <memory> for testing something
* Ninja doesn't like not having a \n, at end of file, adding it back
* update input_adapter file to deal with empty/null file ptr.
* ran make pretty
* added test for inputadapter
* ran make amalgamate
* Update tests/src/unit-deserialization.cpp
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Update tests/src/unit-deserialization.cpp
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Update input adapters.hpp with new includes
* fix unabigious use of _, (there was a double declare)
* did the amalagamate
* rm duplicate includes
* make amalgamate again
* reorder
* amalgamate
* moved it above
* amalgamate
---------
Co-authored-by: Jordan <jordan-hoang@users.noreply.github.com>
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Move UDLs into nlohmann::literals::json_literals namespace
* Add 'using namespace' to unit tests
* Add 'using namespace' to examples
* Add 'using namespace' to README
* Move UDL mkdocs pages out of basic_json/
* Update documentation
* Update docset index
* Add JSON_GlobalUDLs CMake option
* Add unit test
* Build examples without global UDLs
* Add CI target
When parsing a string containing two JSON values using iterators, after
parsing, iterator_input_adapter should have advanced to the first
character of the second value.
Add a unit test to check that this is true.
Niels Lohmann