wt/json
1
0 Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-21 11:29:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

Write json_view's dump() without a call or conversion per token (#5633)

Browse Source 
* Write json_view's dump() without a call or conversion per token

The default dump() (no indentation, no ensure_ascii) gets its own
writer that makes the same walk and produces the same output:

- the write position stays in a local variable instead of a
  member, so the compiler keeps it in a register across stores
  through aliasing char pointers;
- strings and number tokens are copied with fixed-size 32-byte
  moves wherever enough source bytes remain, instead of one
  memcpy call per token;
- the innermost open container lives in local variables; a stack
  that starts as a local array of 32 entries holds the rest;
- unedited documents are walked through the node array in order,
  and integer tokens are read from the source directly.

On top of that, float tokens of at most 15 significant digits are
written straight from their digits via zmij::to_shortest() and
write_shortest(), without converting to a double and back: such
decimals are farther apart than a double's rounding interval, so
the token's digits are the double's shortest digits. Tokens of
16+ digits, or edited values, still go through decimal_to_float().
The view's own NEON write_decimal() is removed in favor of the
shared writer, and the dump output now grows in 64 KiB steps
instead of being resized to its estimate at once.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC C4127 and Infer finding in the view serializer

Test the template options through enabled() so that conditions combined
with them are not constant, and mark the nav::value() null dereference
as a false positive like materialize.hpp does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix out-of-bounds read when dumping a float token of an unchecked image

A float node of an image loaded with image_check::bounds can hold a
token whose bytes are not digits, so its value need not have the digit
count recorded in the node. write_double_at() passed that count to
write_short_decimal(), which indexes its table of powers of ten by it:
an assertion failure in debug builds, an out-of-bounds read in release
builds. Use the counted overload only if the value has exactly that many
digits, otherwise count them.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tidy the view serializer: doxygen block placement, enabled() for a constant

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Use a named document in the dump test: root() of a temporary does not compile

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test write_short_decimal and powers_of_ten_16 against to_chars

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy 22 findings in unit-to_chars.cpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Write doubles from their bits only where double is IEEE 754 binary64

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Split the value and pointer edit tests off unit-json_view_edit.cpp

The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view_edit.cpp has 35409 sections since the fast
dump instantiates more code for editable documents, so its test cases "views
and values", "deeply nested values", "pointers below a null value", and
"strings of other documents are checked" move into
unit-json_view_edit_values.cpp. The helper exception_of_call() that both
files use moves into json_view_test_helpers.hpp.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Move the dump tests of this branch to unit-json_view_dump.cpp

The dump and comparison tests moved to their own file in the dump pull
request; the tests of the fast dump() join them there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review: one powers-of-ten table, literal sizes from the type

- dtoa_impl::powers_of_ten() (10^0 to 10^19, one entry per line)
  replaces powers_of_ten_16() and the table of int_pow10()
- put_literal() takes the length of a string literal from its type
- small_stack holds the heap vector by reference
- the growth step of the output buffer is written in hex

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Explain why the growth of the dump's stack is static

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann authored and GitHub committed 2026-10-21 10:21:30 +02:00
1 parent 4971200b54
commit 4155e2938f
12 files changed
+1743 -395
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tests/src/json_view_test_helpers.hpp
+17
View File
@@ -15,6 +15,7 @@
#include <nlohmann/json_view.hpp>
#include <algorithm> // any_of
#include <functional> // function
#include <random> // mt19937
#include <string> // string
@@ -86,4 +87,20 @@ inline bool has_duplicate_keys(const nlohmann::ordered_json_view& v)
return e.is_structured() && has_duplicate_keys(e);
});
}
#if !defined(JSON_NOEXCEPTION)
// the exception a call throws, or "" if it throws none
inline std::string exception_of_call(const std::function<void()>& f)
{
try
{
f();
}
catch (const nlohmann::json::exception& e)
{
return e.what();
}
return "";
}
#endif
} // namespace json_view_test
tests/src/unit-json_view_dump.cpp
+52
View File
@@ -94,6 +94,58 @@ TEST_CASE("json_view dump")
CHECK(d.root().dump() == json::parse(text).dump());
CHECK(d.root().dump() == "[1.5,100.0,0,-0.0,1.2345678901234568e+29,18446744073709551615,-9223372036854775808,0.1,1e-07,5e-324]");
CHECK(d.root().dump(-1, ' ', false, json_view::number_format::source) == "[1.50,1E2,-0,-0.0,123456789012345678901234567890,18446744073709551615,-9223372036854775808,0.1,1e-7,5e-324]");
// also indented, and with ensure_ascii
CHECK(d.root().dump(0, ' ', false, json_view::number_format::source) == "[\n1.50,\n1E2,\n-0,\n-0.0,\n123456789012345678901234567890,\n18446744073709551615,\n-9223372036854775808,\n0.1,\n1e-7,\n5e-324\n]");
CHECK(d.root().dump(-1, ' ', true, json_view::number_format::source) == "[1.50,1E2,-0,-0.0,123456789012345678901234567890,18446744073709551615,-9223372036854775808,0.1,1e-7,5e-324]");
// float tokens of up to 17 significant digits in every spelling: those
// of at most 15 digits are written from their digits, the others
// through the conversion; both as dump() writes them
{
std::mt19937_64 tokens(1170); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
// a number below n; the remainder is a std::uint64_t, which is
// std::size_t on some platforms and wider on others
const auto draw = [&tokens](std::size_t n)
{
const std::uint64_t r = tokens() % n;
return static_cast<std::size_t>(r);
};
std::string many_tokens = "[";
for (int i = 0; i < 20000; ++i)
{
const std::size_t length = 1 + draw(17);
std::string digits(1, static_cast<char>('1' + draw(9)));
for (std::size_t k = 1; k < length; ++k)
{
digits += static_cast<char>('0' + draw(10));
}
digits += std::string(draw(4), '0'); // trailing zeros
std::string token = draw(3) == 0 ? "-" : "";
const std::size_t point = draw(digits.size() + 1);
if (point == 0)
{
token += "0." + std::string(draw(5), '0') + digits;
}
else
{
token += digits.substr(0, point) + (point < digits.size() ? "." + digits.substr(point) : "");
}
// an exponent that keeps the value between about 1e-320 and 1e300
const int exponent = static_cast<int>(draw(600)) - 300 - static_cast<int>(point);
if (draw(4) != 0)
{
token += (draw(2) == 0 ? "e" : "E") + std::string(exponent >= 0 && draw(2) == 0 ? "+" : "") + std::to_string(exponent);
}
else if (point == digits.size())
{
token += ".0"; // (a float, not an integer)
}
many_tokens += (i != 0 ? "," : "") + token;
}
many_tokens += ']';
const json_document many_doc = json_document::parse(many_tokens);
CHECK(many_doc.root().dump() == json::parse(many_tokens).dump());
}
// random doubles, written as parse() and dump() would
std::mt19937_64 rng(1170); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
tests/src/unit-json_view_edit.cpp
+6 -329
View File
@@ -19,13 +19,18 @@ using nlohmann::ordered_json_editable_document;
using nlohmann::ordered_json_editable_view;
using ptr_t = ordered_json::json_pointer;
#include "json_view_test_helpers.hpp"
#if !defined(JSON_NOEXCEPTION)
using json_view_test::exception_of_call;
#endif
#include <array>
#include <cmath>
#include <cstring>
#include <cstdint>
#include <functional>
#include <iterator>
#include <limits>
#include <map>
#include <memory>
#include <random>
#include <set>
@@ -512,25 +517,6 @@ TEST_CASE("json_view edits: differential")
}
}
namespace
{
#if !defined(JSON_NOEXCEPTION)
// the exception a call throws, or "" if it throws none
std::string exception_of_call(const std::function<void()>& f)
{
try
{
f();
}
catch (const json::exception& e)
{
return e.what();
}
return "";
}
#endif
} // namespace
TEST_CASE("json_view edits: errors")
{
SECTION("an empty document")
@@ -586,316 +572,7 @@ TEST_CASE("json_view edits: errors")
static_cast<void>(other);
}
TEST_CASE("json_view edits: views and values")
{
SECTION("a value that is no longer part of the document")
{
json_editable_document d = json_editable_document::parse("[[[1,2]]]");
const json_editable_view inner = d.root()[0][0];
d.set(d.root()[0], json::array({7}));
d.set(inner, 5);
CHECK(d.root().dump() == "[[7]]");
CHECK(inner.get<int>() == 5);
}
SECTION("views keep referring to their value")
{
json_editable_document d = json_editable_document::parse(R"({"a": [10, 20, 30], "b": {"c": "text"}})");
const json_editable_view a = d.root()["a"];
const json_editable_view twenty = a[1];
const json_editable_view c = d.root()["b"]["c"];
d.insert(a, 0, 5);
d.push_back(a, 40);
CHECK(twenty.get<int>() == 20);
CHECK(a[2].get<int>() == 20);
d.erase(a, 2);
CHECK(twenty.get<int>() == 20); // an erased value keeps its last value
d.set(c, 7);
CHECK(c.get<int>() == 7); // a held view sees an assignment
d.set(d.root()["b"], json::array({1, 2}));
CHECK(d.root()["b"].dump() == "[1,2]");
CHECK(d.root().dump() == R"({"a":[5,10,30,40],"b":[1,2]})");
CHECK(d.root()["a"][0].source_offset() == static_cast<std::size_t>(-1)); // a new value
CHECK(d.root()["a"][1].source_offset() != static_cast<std::size_t>(-1));
}
SECTION("strings stay valid while more edits come")
{
json_editable_document d = json_editable_document::parse("[]");
const auto first = d.push_back(d.root(), std::string(100, 'x')).get_string();
for (int i = 0; i < 1000; ++i)
{
d.push_back(d.root(), std::string(static_cast<std::size_t>(i % 50), 'y'));
}
CHECK(std::string(first.data(), first.size()) == std::string(100, 'x'));
CHECK(d.root().size() == 1001);
}
SECTION("numbers")
{
json_editable_document d = json_editable_document::parse(R"([1.50, 1E2, 3])");
d.set(d.root(), 2, 0.1);
d.push_back(d.root(), std::numeric_limits<double>::quiet_NaN());
d.push_back(d.root(), -std::numeric_limits<double>::infinity());
d.push_back(d.root(), (std::numeric_limits<std::uint64_t>::max)());
d.push_back(d.root(), (std::numeric_limits<std::int64_t>::min)());
CHECK(d.root().dump() == "[1.5,100.0,0.1,null,null,18446744073709551615,-9223372036854775808]");
CHECK(d.root().dump(-1, ' ', false, json_editable_view::number_format::source) == "[1.50,1E2,0.1,null,null,18446744073709551615,-9223372036854775808]");
CHECK(std::isnan(d.root()[3].get<double>()));
CHECK(std::isinf(d.root()[4].get<double>()));
CHECK(d.root()[2].number_token() == "0.1");
CHECK(d.root()[5].get<std::uint64_t>() == 18446744073709551615u);
CHECK(d.root()[6].number_token() == "-9223372036854775808");
CHECK(d.root().materialize().dump() == json::parse(R"([1.5, 100.0, 0.1, null, null, 18446744073709551615, -9223372036854775808])").dump());
}
SECTION("nulls become containers, and the root can be replaced")
{
json_editable_document d = json_editable_document::parse("[null, null]");
d.set(d.root()[0], "k", 1);
d.push_back(d.root()[1], true);
CHECK(d.root().dump() == R"([{"k":1},[true]])");
d.set(d.root(), "scalar");
CHECK(d.root().dump() == R"("scalar")");
d.set(d.root(), json{{"x", {1, 2}}});
d.set(json::json_pointer("/x/-"), 3);
d.set(json::json_pointer("/x/3"), 4); // the size of the array appends too
d.set(json::json_pointer("/y"), false);
CHECK(d.root().dump() == R"({"x":[1,2,3,4],"y":false})");
CHECK(d.erase(json::json_pointer("/x/0")) == 1);
CHECK(d.erase(json::json_pointer("/y")) == 1);
CHECK(d.erase(json::json_pointer("/nothing")) == 0);
CHECK(d.root().dump() == R"({"x":[2,3,4]})");
}
SECTION("duplicate keys")
{
json_editable_document d = json_editable_document::parse(R"({"a": 1, "b": 2, "a": 3})");
d.set(d.root(), "a", 4); // the first member is assigned, the others dropped
CHECK(d.root().dump() == R"({"a":4,"b":2})");
d = json_editable_document::parse(R"({"a": 1, "b": 2, "a": 3})");
CHECK(d.erase(d.root(), "a") == 2);
CHECK(d.root().dump() == R"({"b":2})");
}
SECTION("values from other documents")
{
const json_document source = json_document::parse(R"({"list": [1, "two", {"three": 3.5}], "text": "a\nb"})");
json_editable_document edited = json_editable_document::parse("[0]");
edited.set(edited.root(), 0, json{{"inner", {1, 2}}});
json_editable_document d = json_editable_document::parse("{}");
d.set(d.root(), "copy", source.root()["list"]);
d.set(d.root(), "text", source.root()["text"]);
d.set(d.root(), "edited", edited.root()[0]);
d.set(d.root(), "self", d.root()["copy"]);
// (a raw string with a backslash must not be a macro argument: MSVC C2017)
const std::string expected = R"({"copy":[1,"two",{"three":3.5}],"text":"a\nb","edited":{"inner":[1,2]},"self":[1,"two",{"three":3.5}]})";
CHECK(d.root().dump() == expected);
CHECK(d.root()["copy"] == source.root()["list"]);
CHECK(source.root()["list"] == d.root()["self"]);
CHECK(d.root() != source.root());
}
SECTION("large objects")
{
std::string text = "{";
for (int i = 0; i < 200; ++i)
{
text += (i != 0 ? ",\"k" : "\"k") + std::to_string(i) + "\":" + std::to_string(i);
}
text += '}';
json_editable_document d = json_editable_document::parse(text);
d.set(d.root(), "k7", "seven"); // assigned in place: the index stays in use
CHECK(d.root()["k7"].get_string() == "seven");
d.set(d.root(), "new", 1); // appended: the members move, the lookup is linear
CHECK(d.root()["new"].get<int>() == 1);
CHECK(d.root()["k199"].get<int>() == 199);
d.erase(d.root(), "k0");
CHECK(!d.root().contains("k0"));
CHECK(d.root().size() == 200);
}
SECTION("reuse and memory")
{
json_editable_document d = json_editable_document::parse("[1, 2, 3]");
const std::size_t before = d.memory_usage();
for (int i = 0; i < 100; ++i)
{
d.push_back(d.root(), "some text");
}
CHECK(d.memory_usage() > before);
const json_editable_view first = d.root()[0];
d.shrink_to_fit(); // (with edits, the index stays in place)
CHECK(first.get<int>() == 1);
d.read(std::string("[true]"));
CHECK(d.root().dump() == "[true]");
d.push_back(d.root(), false);
CHECK(d.root().dump() == "[true,false]");
}
}
TEST_CASE("json_view edits: deeply nested values")
{
// copying a value into a document must not recurse per nesting level
const std::size_t depth = 100000;
const std::string brackets = std::string(depth, '[') + std::string(depth, ']');
std::string braces;
for (std::size_t i = 0; i < depth; ++i)
{
braces += "{\"a\":";
}
braces += '1';
braces += std::string(depth, '}');
SECTION("a view of a read-only document")
{
const json_document source = json_document::parse(brackets);
json_editable_document d = json_editable_document::parse("[]");
d.push_back(d.root(), source.root());
CHECK(d.root().dump() == "[" + brackets + "]");
}
SECTION("a view of an editable document")
{
const json_editable_document source = json_editable_document::parse(braces);
json_editable_document d = json_editable_document::parse("{}");
d.set(d.root(), "deep", source.root());
CHECK(d.root().dump() == "{\"deep\":" + braces + "}");
}
SECTION("a view of an edited document (values behind links)")
{
const json_document source = json_document::parse(brackets);
json_editable_document edited = json_editable_document::parse("[[]]");
edited.push_back(edited.root()[0], source.root());
edited.push_back(edited.root(), source.root());
json_editable_document d = json_editable_document::parse("null");
d.set(d.root(), edited.root());
CHECK(d.root().dump() == "[[" + brackets + "]," + brackets + "]");
}
SECTION("a basic_json value")
{
json deep = json::array();
json* inner = &deep;
for (std::size_t i = 1; i < depth; ++i)
{
inner->push_back(json::array());
inner = &inner->back();
}
json_editable_document d = json_editable_document::parse("[]");
d.push_back(d.root(), deep);
CHECK(d.root().dump() == "[" + brackets + "]");
}
SECTION("a basic_json value with objects")
{
json deep = 1;
for (std::size_t i = 0; i < depth; ++i)
{
json outer = json::object();
outer["a"] = std::move(deep);
deep = std::move(outer);
}
json_editable_document d = json_editable_document::parse("{}");
d.set(d.root(), "deep", deep);
CHECK(d.root().dump() == "{\"deep\":" + braces + "}");
}
}
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("json_view edits: pointers below a null value")
{
// a null value on the way becomes what basic_json makes of it: an array
// for "-" and for digits, an object otherwise
struct test_case
{
const char* document;
const char* pointer;
};
const std::array<test_case, 16> cases =
{
{
{R"({"a":null})", "/a/0"},
{R"({"a":null})", "/a/-"},
{R"({"a":null})", "/a/3"},
{R"({"a":null})", "/a/x"},
{R"({"a":null})", "/a/+1"},
{R"({"a":null})", "/a/01"},
{R"({"a":null})", "/a/"},
{R"({"a":{"b":null}})", "/a/b/1"},
{R"({"a":{"b":null}})", "/a/b/-"},
{R"({"a":[null]})", "/a/0/0"},
{R"({"a":[null,null]})", "/a/1/k"},
{R"([null])", "/0"},
{"null", "/0"},
{"null", "/-"},
{"null", "/k"},
{"null", ""},
}
};
for (const test_case& c : cases)
{
CAPTURE(c.document)
CAPTURE(c.pointer)
json expected = json::parse(c.document);
const std::string error = exception_of_call([&]
{
expected[json::json_pointer(c.pointer)] = 1;
});
json_editable_document d = json_editable_document::parse(c.document);
if (error.empty())
{
d.set(json::json_pointer(c.pointer), 1);
CHECK(d.root().dump() == expected.dump());
CHECK(d.root().materialize() == expected);
}
else
{
// the same error, and the document is not changed
CHECK(exception_of_call([&] { d.set(json::json_pointer(c.pointer), 1); }) == error);
CHECK(d.root().dump() == json::parse(c.document).dump());
}
}
}
TEST_CASE("json_view edits: strings of other documents are checked")
{
// A document borrows the text it was parsed from, and sees later changes
// of the text: a way to get ill-formed UTF-8 into a view. Copying it into
// an editable document is an error, as for any other string.
std::string text = R"({"key":"abc","list":["abc"]})";
const json_document source = json_document::parse(text);
const auto message_of = [](const std::string & bad)
{
return exception_of_call([&]
{
const std::string dumped = json(bad).dump();
static_cast<void>(dumped);
});
};
json_editable_document d = json_editable_document::parse("[1]");
d.push_back(d.root(), source.root());
CHECK(d.root().dump() == R"([1,{"key":"abc","list":["abc"]}])");
text[text.find("abc") + 1] = '\xC3'; // "a\xC3c"
text[text.rfind("abc") + 1] = '\xC3';
const std::string bad_value = message_of(std::string("a\xC3" "c"));
CHECK(!bad_value.empty());
CHECK(exception_of_call([&] { d.push_back(d.root(), source.root()["key"]); }) == bad_value);
CHECK(exception_of_call([&] { d.set(d.root()[0], source.root()["list"][0]); }) == bad_value);
CHECK(exception_of_call([&] { d.set(d.root(), 0, source.root()["list"]); }) == bad_value);
CHECK(exception_of_call([&] { d.set(d.root(), 0, source.root()); }) == bad_value);
text[text.find("key") + 1] = '\xC3'; // a key is checked as well
const std::string bad_key = message_of(std::string("k\xC3" "y"));
CHECK(exception_of_call([&] { d.set(d.root(), 0, source.root()); }) == bad_key);
CHECK(exception_of_call([&] { d.push_back(d.root(), source.root()); }) == bad_key);
// nothing of the failed edits is visible
CHECK(d.root().dump() == R"([1,{"key":"abc","list":["abc"]}])");
}
TEST_CASE("json_view edits: the size of a text arena")
{
using nlohmann::detail::view::text_capacity;
tests/src/unit-json_view_edit_values.cpp
+357
View File
@@ -0,0 +1,357 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json_view.hpp>
using nlohmann::json;
using nlohmann::json_document;
using nlohmann::json_editable_document;
using nlohmann::json_editable_view;
#include "json_view_test_helpers.hpp"
#if !defined(JSON_NOEXCEPTION)
using json_view_test::exception_of_call;
#endif
#include <array>
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <limits>
#include <map>
#include <string>
#include <utility>
#include <vector>
// These tests were split off unit-json_view_edit.cpp, whose object file got too
// large for the MinGW linker (see json_view_test_helpers.hpp).
TEST_CASE("json_view edits: views and values")
{
SECTION("a value that is no longer part of the document")
{
json_editable_document d = json_editable_document::parse("[[[1,2]]]");
const json_editable_view inner = d.root()[0][0];
d.set(d.root()[0], json::array({7}));
d.set(inner, 5);
CHECK(d.root().dump() == "[[7]]");
CHECK(inner.get<int>() == 5);
}
SECTION("views keep referring to their value")
{
json_editable_document d = json_editable_document::parse(R"({"a": [10, 20, 30], "b": {"c": "text"}})");
const json_editable_view a = d.root()["a"];
const json_editable_view twenty = a[1];
const json_editable_view c = d.root()["b"]["c"];
d.insert(a, 0, 5);
d.push_back(a, 40);
CHECK(twenty.get<int>() == 20);
CHECK(a[2].get<int>() == 20);
d.erase(a, 2);
CHECK(twenty.get<int>() == 20); // an erased value keeps its last value
d.set(c, 7);
CHECK(c.get<int>() == 7); // a held view sees an assignment
d.set(d.root()["b"], json::array({1, 2}));
CHECK(d.root()["b"].dump() == "[1,2]");
CHECK(d.root().dump() == R"({"a":[5,10,30,40],"b":[1,2]})");
CHECK(d.root()["a"][0].source_offset() == static_cast<std::size_t>(-1)); // a new value
CHECK(d.root()["a"][1].source_offset() != static_cast<std::size_t>(-1));
}
SECTION("strings stay valid while more edits come")
{
json_editable_document d = json_editable_document::parse("[]");
const auto first = d.push_back(d.root(), std::string(100, 'x')).get_string();
for (int i = 0; i < 1000; ++i)
{
d.push_back(d.root(), std::string(static_cast<std::size_t>(i % 50), 'y'));
}
CHECK(std::string(first.data(), first.size()) == std::string(100, 'x'));
CHECK(d.root().size() == 1001);
}
SECTION("numbers")
{
json_editable_document d = json_editable_document::parse(R"([1.50, 1E2, 3])");
d.set(d.root(), 2, 0.1);
d.push_back(d.root(), std::numeric_limits<double>::quiet_NaN());
d.push_back(d.root(), -std::numeric_limits<double>::infinity());
d.push_back(d.root(), (std::numeric_limits<std::uint64_t>::max)());
d.push_back(d.root(), (std::numeric_limits<std::int64_t>::min)());
CHECK(d.root().dump() == "[1.5,100.0,0.1,null,null,18446744073709551615,-9223372036854775808]");
CHECK(d.root().dump(-1, ' ', false, json_editable_view::number_format::source) == "[1.50,1E2,0.1,null,null,18446744073709551615,-9223372036854775808]");
CHECK(std::isnan(d.root()[3].get<double>()));
CHECK(std::isinf(d.root()[4].get<double>()));
CHECK(d.root()[2].number_token() == "0.1");
CHECK(d.root()[5].get<std::uint64_t>() == 18446744073709551615u);
CHECK(d.root()[6].number_token() == "-9223372036854775808");
CHECK(d.root().materialize().dump() == json::parse(R"([1.5, 100.0, 0.1, null, null, 18446744073709551615, -9223372036854775808])").dump());
}
SECTION("numbers of other float types")
{
// doubles have their own path to the output; other float types are
// written as basic_json writes them, non-finite values as null
using json_float = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
using document_float = nlohmann::basic_json_document<json_float, true>;
document_float d = document_float::parse("[1.5]");
d.push_back(d.root(), std::numeric_limits<float>::quiet_NaN());
d.push_back(d.root(), -std::numeric_limits<float>::infinity());
CHECK(d.root().dump() == "[1.5,null,null]");
CHECK(d.root().dump(2) == json_float::parse("[1.5, null, null]").dump(2));
}
SECTION("nulls become containers, and the root can be replaced")
{
json_editable_document d = json_editable_document::parse("[null, null]");
d.set(d.root()[0], "k", 1);
d.push_back(d.root()[1], true);
CHECK(d.root().dump() == R"([{"k":1},[true]])");
d.set(d.root(), "scalar");
CHECK(d.root().dump() == R"("scalar")");
d.set(d.root(), json{{"x", {1, 2}}});
d.set(json::json_pointer("/x/-"), 3);
d.set(json::json_pointer("/x/3"), 4); // the size of the array appends too
d.set(json::json_pointer("/y"), false);
CHECK(d.root().dump() == R"({"x":[1,2,3,4],"y":false})");
CHECK(d.erase(json::json_pointer("/x/0")) == 1);
CHECK(d.erase(json::json_pointer("/y")) == 1);
CHECK(d.erase(json::json_pointer("/nothing")) == 0);
CHECK(d.root().dump() == R"({"x":[2,3,4]})");
}
SECTION("duplicate keys")
{
json_editable_document d = json_editable_document::parse(R"({"a": 1, "b": 2, "a": 3})");
d.set(d.root(), "a", 4); // the first member is assigned, the others dropped
CHECK(d.root().dump() == R"({"a":4,"b":2})");
d = json_editable_document::parse(R"({"a": 1, "b": 2, "a": 3})");
CHECK(d.erase(d.root(), "a") == 2);
CHECK(d.root().dump() == R"({"b":2})");
}
SECTION("values from other documents")
{
const json_document source = json_document::parse(R"({"list": [1, "two", {"three": 3.5}], "text": "a\nb"})");
json_editable_document edited = json_editable_document::parse("[0]");
edited.set(edited.root(), 0, json{{"inner", {1, 2}}});
json_editable_document d = json_editable_document::parse("{}");
d.set(d.root(), "copy", source.root()["list"]);
d.set(d.root(), "text", source.root()["text"]);
d.set(d.root(), "edited", edited.root()[0]);
d.set(d.root(), "self", d.root()["copy"]);
// (a raw string with a backslash must not be a macro argument: MSVC C2017)
const std::string expected = R"({"copy":[1,"two",{"three":3.5}],"text":"a\nb","edited":{"inner":[1,2]},"self":[1,"two",{"three":3.5}]})";
CHECK(d.root().dump() == expected);
CHECK(d.root()["copy"] == source.root()["list"]);
CHECK(source.root()["list"] == d.root()["self"]);
CHECK(d.root() != source.root());
}
SECTION("large objects")
{
std::string text = "{";
for (int i = 0; i < 200; ++i)
{
text += (i != 0 ? ",\"k" : "\"k") + std::to_string(i) + "\":" + std::to_string(i);
}
text += '}';
json_editable_document d = json_editable_document::parse(text);
d.set(d.root(), "k7", "seven"); // assigned in place: the index stays in use
CHECK(d.root()["k7"].get_string() == "seven");
d.set(d.root(), "new", 1); // appended: the members move, the lookup is linear
CHECK(d.root()["new"].get<int>() == 1);
CHECK(d.root()["k199"].get<int>() == 199);
d.erase(d.root(), "k0");
CHECK(!d.root().contains("k0"));
CHECK(d.root().size() == 200);
}
SECTION("reuse and memory")
{
json_editable_document d = json_editable_document::parse("[1, 2, 3]");
const std::size_t before = d.memory_usage();
for (int i = 0; i < 100; ++i)
{
d.push_back(d.root(), "some text");
}
CHECK(d.memory_usage() > before);
const json_editable_view first = d.root()[0];
d.shrink_to_fit(); // (with edits, the index stays in place)
CHECK(first.get<int>() == 1);
d.read(std::string("[true]"));
CHECK(d.root().dump() == "[true]");
d.push_back(d.root(), false);
CHECK(d.root().dump() == "[true,false]");
}
}
TEST_CASE("json_view edits: deeply nested values")
{
// copying a value into a document must not recurse per nesting level
const std::size_t depth = 100000;
const std::string brackets = std::string(depth, '[') + std::string(depth, ']');
std::string braces;
for (std::size_t i = 0; i < depth; ++i)
{
braces += "{\"a\":";
}
braces += '1';
braces += std::string(depth, '}');
SECTION("a view of a read-only document")
{
const json_document source = json_document::parse(brackets);
json_editable_document d = json_editable_document::parse("[]");
d.push_back(d.root(), source.root());
CHECK(d.root().dump() == "[" + brackets + "]");
}
SECTION("a view of an editable document")
{
const json_editable_document source = json_editable_document::parse(braces);
json_editable_document d = json_editable_document::parse("{}");
d.set(d.root(), "deep", source.root());
CHECK(d.root().dump() == "{\"deep\":" + braces + "}");
}
SECTION("a view of an edited document (values behind links)")
{
const json_document source = json_document::parse(brackets);
json_editable_document edited = json_editable_document::parse("[[]]");
edited.push_back(edited.root()[0], source.root());
edited.push_back(edited.root(), source.root());
json_editable_document d = json_editable_document::parse("null");
d.set(d.root(), edited.root());
CHECK(d.root().dump() == "[[" + brackets + "]," + brackets + "]");
}
SECTION("a basic_json value")
{
json deep = json::array();
json* inner = &deep;
for (std::size_t i = 1; i < depth; ++i)
{
inner->push_back(json::array());
inner = &inner->back();
}
json_editable_document d = json_editable_document::parse("[]");
d.push_back(d.root(), deep);
CHECK(d.root().dump() == "[" + brackets + "]");
}
SECTION("a basic_json value with objects")
{
json deep = 1;
for (std::size_t i = 0; i < depth; ++i)
{
json outer = json::object();
outer["a"] = std::move(deep);
deep = std::move(outer);
}
json_editable_document d = json_editable_document::parse("{}");
d.set(d.root(), "deep", deep);
CHECK(d.root().dump() == "{\"deep\":" + braces + "}");
}
}
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("json_view edits: pointers below a null value")
{
// a null value on the way becomes what basic_json makes of it: an array
// for "-" and for digits, an object otherwise
struct test_case
{
const char* document;
const char* pointer;
};
const std::array<test_case, 16> cases =
{
{
{R"({"a":null})", "/a/0"},
{R"({"a":null})", "/a/-"},
{R"({"a":null})", "/a/3"},
{R"({"a":null})", "/a/x"},
{R"({"a":null})", "/a/+1"},
{R"({"a":null})", "/a/01"},
{R"({"a":null})", "/a/"},
{R"({"a":{"b":null}})", "/a/b/1"},
{R"({"a":{"b":null}})", "/a/b/-"},
{R"({"a":[null]})", "/a/0/0"},
{R"({"a":[null,null]})", "/a/1/k"},
{R"([null])", "/0"},
{"null", "/0"},
{"null", "/-"},
{"null", "/k"},
{"null", ""},
}
};
for (const test_case& c : cases)
{
CAPTURE(c.document)
CAPTURE(c.pointer)
json expected = json::parse(c.document);
const std::string error = exception_of_call([&]
{
expected[json::json_pointer(c.pointer)] = 1;
});
json_editable_document d = json_editable_document::parse(c.document);
if (error.empty())
{
d.set(json::json_pointer(c.pointer), 1);
CHECK(d.root().dump() == expected.dump());
CHECK(d.root().materialize() == expected);
}
else
{
// the same error, and the document is not changed
CHECK(exception_of_call([&] { d.set(json::json_pointer(c.pointer), 1); }) == error);
CHECK(d.root().dump() == json::parse(c.document).dump());
}
}
}
TEST_CASE("json_view edits: strings of other documents are checked")
{
// A document borrows the text it was parsed from, and sees later changes
// of the text: a way to get ill-formed UTF-8 into a view. Copying it into
// an editable document is an error, as for any other string.
std::string text = R"({"key":"abc","list":["abc"]})";
const json_document source = json_document::parse(text);
const auto message_of = [](const std::string & bad)
{
return exception_of_call([&]
{
const std::string dumped = json(bad).dump();
static_cast<void>(dumped);
});
};
json_editable_document d = json_editable_document::parse("[1]");
d.push_back(d.root(), source.root());
CHECK(d.root().dump() == R"([1,{"key":"abc","list":["abc"]}])");
text[text.find("abc") + 1] = '\xC3'; // "a\xC3c"
text[text.rfind("abc") + 1] = '\xC3';
const std::string bad_value = message_of(std::string("a\xC3" "c"));
CHECK(!bad_value.empty());
CHECK(exception_of_call([&] { d.push_back(d.root(), source.root()["key"]); }) == bad_value);
CHECK(exception_of_call([&] { d.set(d.root()[0], source.root()["list"][0]); }) == bad_value);
CHECK(exception_of_call([&] { d.set(d.root(), 0, source.root()["list"]); }) == bad_value);
CHECK(exception_of_call([&] { d.set(d.root(), 0, source.root()); }) == bad_value);
text[text.find("key") + 1] = '\xC3'; // a key is checked as well
const std::string bad_key = message_of(std::string("k\xC3" "y"));
CHECK(exception_of_call([&] { d.set(d.root(), 0, source.root()); }) == bad_key);
CHECK(exception_of_call([&] { d.push_back(d.root(), source.root()); }) == bad_key);
// nothing of the failed edits is visible
CHECK(d.root().dump() == R"([1,{"key":"abc","list":["abc"]}])");
}
#endif
tests/src/unit-json_view_image.cpp
+25
View File
@@ -1036,6 +1036,31 @@ TEST_CASE("json_view images: check")
}
}
SECTION("float tokens with fewer digits than the layout records")
{
// The bytes of the token are not digits (they read as zeros or as
// other values), so the value has fewer (or more) digits than the
// layout says: dump() must still write a number.
for (const auto& source : std::vector<std::pair<std::string, std::string>>
{
{"123456789012345678.5", std::string("@") + std::string(16, '0') + "1.1"}, // 19 digits
{"1234.5", "@001.1"}, // 5 digits
{"1234.5", "9??.??"} // more than 5 digits
})
{
CAPTURE(source.second)
const std::vector<std::uint8_t> img = json_document::parse("[" + source.first + "]").save();
const node n = node_at(img, 1);
REQUIRE(source.second.size() == n.len);
std::vector<std::uint8_t> b = img;
std::memcpy(b.data() + text_at(img) + n.off, source.second.data(), n.len);
const json_document d = json_document::load(b, image_check::bounds);
const std::string dumped = d.root().dump();
CAPTURE(dumped)
CHECK(json::parse(dumped)[0].is_number());
}
}
SECTION("nodes that share a range")
{
// [big string, then n strings made to point to the big string]: every
tests/src/unit-to_chars.cpp
+82
View File
@@ -846,3 +846,85 @@ TEST_CASE("choice of the conversion")
DOCTEST_MSVC_SUPPRESS_WARNING_POP
}
}
TEST_CASE("short decimals")
{
// write_short_decimal() writes digits * 10^exp for the digits of a double
// that need no conversion (at most 15, the first not 0): as to_chars()
// writes the (positive) double that has these digits
const auto written = [](std::uint64_t digits, int exp) -> std::string
{
std::array<char, 64> buf{}; // (up to 41 bytes are written)
char* const end = nlohmann::detail::dtoa_impl::write_short_decimal(buf.data(), digits, exp);
return {buf.data(), end};
};
const auto written_counted = [](std::uint64_t digits, int count, int exp) -> std::string
{
std::array<char, 64> buf{};
char* const end = nlohmann::detail::dtoa_impl::write_short_decimal(buf.data(), digits, count, exp);
return {buf.data(), end};
};
const auto expected = [](std::uint64_t digits, int exp) -> std::string
{
const double value = std::strtod((std::to_string(digits) + "e" + std::to_string(exp)).c_str(), nullptr);
std::array<char, 64> buf{};
char* const end = nlohmann::detail::to_chars(buf.data(), buf.data() + 32, value);
return {buf.data(), end};
};
SECTION("powers of ten")
{
const auto& powers = nlohmann::detail::dtoa_impl::powers_of_ten();
CHECK(powers.front() == 1u);
for (std::size_t i = 1; i < powers.size(); ++i)
{
CHECK(powers[i] == powers[i - 1] * 10u);
}
CHECK(powers.back() == 10000000000000000000u);
}
SECTION("examples")
{
CHECK(written(1, 0) == "1.0");
CHECK(written(15, -1) == "1.5");
CHECK(written(125, -2) == "1.25");
CHECK(written(1, 22) == "1e+22");
CHECK(written(123456789012345, -2) == "1234567890123.45");
CHECK(written(999999999999999, -15) == "0.999999999999999");
CHECK(written(5, -324) == "5e-324");
CHECK(written_counted(1, 1, 0) == "1.0");
CHECK(written_counted(125, 3, -2) == "1.25");
CHECK(written_counted(100, 3, -2) == "1.0");
CHECK(written_counted(999999999999999, 15, -15) == "0.999999999999999");
}
SECTION("random digits, exponents and trailing zeros")
{
std::mt19937_64 rng(1170); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
for (int i = 0; i < 100000; ++i)
{
// 1 to 15 digits, the first not 0, and up to 14 of them trailing zeros
std::uint64_t count = 1 + (rng() % 15u);
std::uint64_t power = 1;
for (std::uint64_t k = 1; k < count; ++k)
{
power *= 10;
}
std::uint64_t digits = power + (rng() % (9 * power));
const std::uint64_t zeros = (rng() % 3u == 0) ? (rng() % count) : 0;
for (std::uint64_t k = 0; k < zeros; ++k)
{
digits = (digits / 10) * 10;
}
// (a value between 1e-300 and 1e300)
const int exp = static_cast<int>(rng() % 560u) - 300 - static_cast<int>(count);
CAPTURE(digits)
CAPTURE(count)
CAPTURE(exp)
const std::string want = expected(digits, exp);
CHECK(written(digits, exp) == want);
CHECK(written_counted(digits, static_cast<int>(count), exp) == want);
}
}
}