Compare json_view values without a vector per container and a basic_json per number

operator== of views (view/view, view/json, and editable documents) built two vectors of members for every object it entered, sorted them with stable_sort for every object, and converted each number to a basic_json value. It now: - keeps the members of all open objects in one vector (reserved with geometric growth, so that nested objects do not reallocate), and orders them by insertion sort for up to 16 members, or not at all when the keys ascend already; - compares numbers of one type directly, and builds a basic_json only for numbers of different types; - compares short texts (keys, numbers, most strings) with two overlapping loads (bytes_equal(), in node.hpp); - first walks both documents in lockstep, in the order of their text, which finds most pairs of equal values without the work of the comparison. The walk only enters an object if the comparison would visit its members in the same order (the object type keeps the order, the keys ascend, or all values are scalars), so a difference is never found later than before, and it records the containers it found different, so that the comparison does not walk them again; - rejects values of different types and arrays of different sizes before any of this, and, for an object type that keeps the order, objects whose first (up to four) members have different keys, which ends a comparison that differs near the start without preparing the members. Results are unchanged: numbers compare by value across types, duplicate keys resolve as parse() resolves them, and objects compare in document order for ordered_json and by key otherwise. Instructions of one comparison of a document with an equal copy, macOS arm64 (develop -> now), json_document / ordered_json_document / editable: twitter 16.1M -> 7.0M 17.7M -> 1.5M 16.6M -> 7.3M citm 45.0M -> 3.4M 47.0M -> 3.2M 45.7M -> 4.3M canada 111.4M -> 13.7M 111.4M -> 13.7M 113.3M -> 15.0M jeopardy 26.2M -> 3.7M 27.3M -> 3.7M 26.6M -> 12.1M and of a view with a basic_json value: twitter 11.4M -> 5.9M 11.2M -> 7.3M 11.6M -> 5.9M citm 33.9M -> 9.7M 32.6M -> 9.1M 34.6M -> 9.8M canada 92.9M -> 32.1M 92.9M -> 32.1M 93.9M -> 32.8M and Linux instruction counts (GCC 13 / Clang 18) of the same comparisons: document with an equal copy json_document ordered_json_document editable twitter -49 % / -52 % -89 % / -89 % -48 % / -50 % citm -90 % / -89 % -91 % / -90 % -88 % / -87 % jeopardy (4,000 questions) -85 % / -85 % -86 % / -86 % -46 % / -49 % view with a basic_json value twitter -41 % / -45 % -28 % / -35 % -45 % / -44 % citm -63 % / -65 % -63 % / -66 % -65 % / -64 % jeopardy -39 % / -43 % -14 % / -24 % -44 % / -43 % A sweep that places one difference at about 2,100 positions of eight documents (a token at an even spacing, the first and last tokens, the first, middle and last member of the root removed, one appended, the document wrapped) in five kinds of comparison (view/view of read-only and of editable documents, view/json, json_document, editable/view) finds the same results as before and no placement slower on the Mac (Clang); on Linux no placement is slower than 3 % (Clang and GCC). Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 11:36:14 +00:00 · 2026-10-22 09:30:42 +02:00
1 parent 2dc1e54856
commit fa8134ae85
@@ -120,6 +120,74 @@ NLOHMANN_VIEW_ALWAYS_INLINE std::uint32_t number_length(const node& n) noexcept
           : (n.extra & 0xFFu) + (n.kind == static_cast<std::uint8_t>(value_t::number_integer) ? 1u : 0u);
 }

+/// whether n bytes are equal; the short texts of JSON (keys, numbers, most
+/// strings) are compared by two overlapping loads instead of a library call
+NLOHMANN_VIEW_ALWAYS_INLINE bool bytes_equal(const char* x, const char* y, std::size_t n) noexcept
+{
+    const auto load8 = [](const char* p)
+    {
+        std::uint64_t v = 0;
+        std::memcpy(&v, p, sizeof(v));
+        return v;
+    };
+    const auto load4 = [](const char* p)
+    {
+        std::uint32_t v = 0;
+        std::memcpy(&v, p, sizeof(v));
+        return v;
+    };
+    if (n >= 8)
+    {
+        std::size_t i = 0;
+        for (; i + 8 < n; i += 8)
+        {
+            if (load8(x + i) != load8(y + i))
+            {
+                return false;
+            }
+        }
+        return load8(x + n - 8) == load8(y + n - 8);
+    }
+    if (n >= 4)
+    {
+        return load4(x) == load4(y) && load4(x + n - 4) == load4(y + n - 4);
+    }
+    for (std::size_t i = 0; i < n; ++i)
+    {
+        if (x[i] != y[i])
+        {
+            return false;
+        }
+    }
+    return true;
+}
+
+/// whether the n1 bytes at x are before the n2 bytes at y (memcmp() order, then the shorter first);
+/// the first difference is found eight bytes at a time
+NLOHMANN_VIEW_ALWAYS_INLINE bool bytes_less(const char* x, std::size_t n1, const char* y, std::size_t n2) noexcept
+{
+    const std::size_t n = n1 < n2 ? n1 : n2;
+    std::size_t i = 0;
+    for (; i + 8 <= n; i += 8)
+    {
+        const std::uint64_t u = read_eight_bytes(x + i);
+        const std::uint64_t v = read_eight_bytes(y + i);
+        if (u != v)
+        {
+            const auto shift = static_cast<unsigned>(count_trailing_zeros(u ^ v)) & ~7u; // (the first byte that differs)
+            return ((u >> shift) & 0xFFu) < ((v >> shift) & 0xFFu);
+        }
+    }
+    for (; i < n; ++i)
+    {
+        if (x[i] != y[i])
+        {
+            return static_cast<unsigned char>(x[i]) < static_cast<unsigned char>(y[i]);
+        }
+    }
+    return n1 < n2;
+}
+
 /// estimated number of nodes for an input of `size` bytes (one node per ~12
 /// bytes covers typical documents without regrowth)
 inline std::size_t estimate_nodes(std::size_t size) noexcept
@@ -723,8 +723,12 @@ class basic_json_view
    template<typename, bool> friend class basic_json_document;
    template<typename, bool> friend class basic_json_view;
    template<typename, typename> friend class detail::view::editor;
+    template<typename, typename> friend class detail::view::view_side;
    friend iterator;

+    /// whether the view is one of an editable document
+    static constexpr bool is_editable = Editable;
+
    basic_json_view(const document_data* d, const node* n) noexcept
        : m_doc(d), m_node(n)
    {}
@@ -345,6 +345,33 @@ TEST_CASE("json_view comparison")
        }
    }

+    SECTION("objects whose first keys differ or repeat")
+    {
+        const char* const texts[] =
+        {
+            R"({"a":1,"b":2})", R"({"a":1,"c":2})", R"({"b":1,"a":2})", R"({"a":1,"a":2,"b":3})", R"({"a":2,"b":3})",
+            R"({"a":1,"b":1,"a":2})", R"({"a":2,"b":1})", R"({"a":1,"a":2})", R"({"b":1})", R"({"a":1,"b":2,"c":3,"d":4,"e":5})",
+            R"({"a":1,"b":2,"c":3,"d":4,"f":5})", R"({"a":1,"b":2,"c":3,"a":4,"e":5})", R"({"a":4,"b":2,"c":3,"e":5})",
+            R"({"\u0061":1,"b":2})", R"({})", R"([{"a":1,"b":2}])", R"([{"a":1,"c":2}])"
+        };
+        for (const char* const x : texts)
+        {
+            for (const char* const y : texts)
+            {
+                CAPTURE(x)
+                CAPTURE(y)
+                const ordered_json_document dx = ordered_json_document::parse(x);
+                const ordered_json_document dy = ordered_json_document::parse(y);
+                const ordered_json_editable_document ex = ordered_json_editable_document::parse(x);
+                const ordered_json jx = ordered_json::parse(x);
+                const ordered_json jy = ordered_json::parse(y);
+                CHECK((dx.root() == dy.root()) == (jx == jy));
+                CHECK((dx.root() == jy) == (jx == jy));
+                CHECK((ex.root() == dy.root()) == (jx == jy));
+            }
+        }
+    }
+
    SECTION("numbers, duplicate keys, member order")
    {
        const auto same = [](const char* x, const char* y)
@@ -385,6 +412,129 @@ TEST_CASE("json_view comparison")
        CHECK(!(null_document.root() == discarded));
    }

+    SECTION("values that are the same text, or the same value written differently, at any depth")
+    {
+        // nested arrays and objects around depths where the walk over equal text gives up
+        for (const int depth :
+                {
+                    1, 5, 47, 48, 49, 50, 60, 300
+                })
+        {
+            CAPTURE(depth)
+            std::string open;
+            std::string close;
+            for (int i = 0; i < depth; ++i)
+            {
+                open += i % 2 == 0 ? R"({"k":[1,"s",{"x":true},)" : R"([null,2.5,{"y":)";
+                close = (i % 2 == 0 ? "]}" : "}]") + close;
+            }
+            // the same text, one number changed, and one number written differently
+            const std::string text = open + "7" + close;
+            const std::string changed = open + "8" + close;
+            const std::string spelled = open + "7.0" + close;
+            const json_document a = json_document::parse(text);
+            const json_document b = json_document::parse(text);
+            const json_document c = json_document::parse(changed);
+            const json_document d = json_document::parse(spelled);
+            CHECK(a.root() == b.root());
+            CHECK(a.root() != c.root());
+            CHECK(c.root() != a.root());
+            CHECK(a.root() == d.root());
+            CHECK(d.root() == a.root());
+            CHECK(a.root() == json::parse(text));
+            CHECK(a.root() != json::parse(changed));
+            CHECK(json::parse(spelled) == a.root());
+            const nlohmann::json_editable_document e = nlohmann::json_editable_document::parse(text);
+            CHECK(a.root() == e.root());
+            CHECK(e.root() == a.root());
+            CHECK(c.root() != e.root());
+            CHECK(e.root() == e.root());
+            const ordered_json_document oa = ordered_json_document::parse(text);
+            const ordered_json_document oc = ordered_json_document::parse(changed);
+            CHECK(oa.root() != oc.root());
+            CHECK(oa.root() == ordered_json::parse(text));
+            CHECK(oc.root() != ordered_json::parse(text));
+        }
+
+        // not the same text, but the same value
+        const auto same = [](const char* x, const char* y)
+        {
+            const json_document dx = json_document::parse(x);
+            const json_document dy = json_document::parse(y);
+            const bool equal = dx.root() == dy.root();
+            CHECK(equal == (json::parse(x) == json::parse(y)));
+            CHECK(equal == (dx.root() == json::parse(y)));
+            return equal;
+        };
+        CHECK(same(R"({"a":1,"a":2})", R"({"a":3,"a":2})"));
+        CHECK(same(R"({"a":1,"b":[1,{"c":2,"c":3}]})", R"({"b":[1,{"c":9,"c":3}],"a":1})"));
+        CHECK(!same(R"({"a":1,"a":2})", R"({"a":2,"a":3})"));
+        CHECK(same("[1.0,-0,0,1e2]", "[1.00,0,-0,100.0]"));
+        CHECK(same(R"(["\u00e9","\n"])", "[\"\xc3\xa9\",\"\\n\"]"));
+        CHECK(!same(R"({"a":[1,2,{"b":null}]})", R"({"a":[1,2,{"b":false}]})"));
+        CHECK(!same(R"({"a":[1,2,{"b":null}]})", R"({"a":[1,2,{"c":null}]})"));
+        CHECK(!same(R"({"a":[1,2,{"b":null}]})", R"({"a":[1,2,{"b":null,"c":1}]})"));
+
+        // edited documents: moved sequences, links, and values of new nodes
+        nlohmann::json_editable_document edited = nlohmann::json_editable_document::parse(R"({"a":[1,2,3],"b":{"c":"d"},"e":0.5})");
+        const json_document plain = json_document::parse(R"({"a":[1,2,3,4],"b":{"c":"x","y":null},"e":0.5,"z":[]})");
+        edited.push_back(edited.root()["a"], 4);
+        edited.set(edited.root()["b"], "c", "x");
+        edited.set(edited.root()["b"], "y", nullptr);
+        edited.set(edited.root(), "z", json::array());
+        CHECK(edited.root() == plain.root());
+        CHECK(plain.root() == edited.root());
+        edited.erase(edited.root()["a"], 0);
+        CHECK(edited.root() != plain.root());
+        CHECK(edited.root()["a"] == json::parse("[2,3,4]"));
+        const json_document tail = json_document::parse("[2,3,4]");
+        CHECK(edited.root()["a"] == tail.root());
+    }
+
+    SECTION("large documents, with a difference at any place")
+    {
+        const auto make = [](std::size_t count, std::size_t changed, bool swap_members)
+        {
+            std::string text = R"({"b":1,"list":[)";
+            for (std::size_t i = 0; i < count; ++i)
+            {
+                text += (i != 0 ? "," : "");
+                const std::string id = std::to_string(i == changed ? i + 1 : i);
+                text += swap_members && i % 3 == 0 ? R"({"v":[1.5,"s",{"k":)" + id + R"(}],"id":)" + id + "}" : R"({"id":)" + id + R"(,"v":[1.5,"s",{"k":)" + id + "}]}";
+            }
+            return text + R"(],"a":{"y":2,"x":3}})";
+        };
+        const std::size_t count = 3000;
+        for (const std::size_t changed :
+                {
+                    count, std::size_t{0}, std::size_t{1}, count / 2, count - 1
+                })
+        {
+            for (const bool swap :
+                    {
+                        false, true
+                    })
+            {
+                CAPTURE(changed)
+                CAPTURE(swap)
+                const std::string text_a = make(count, count, false);
+                const std::string text_b = make(count, changed, swap);
+                const json_document a = json_document::parse(text_a);
+                const json_document b = json_document::parse(text_b);
+                const bool equal = changed == count;
+                CHECK((a.root() == b.root()) == equal);
+                CHECK((b.root() == a.root()) == equal);
+                CHECK((a.root() == json::parse(text_b)) == equal);
+                CHECK((json::parse(text_b) == a.root()) == equal);
+                CHECK((a.root()["list"] == b.root()["list"]) == equal);
+                const ordered_json_document oa = ordered_json_document::parse(text_a);
+                const ordered_json_document ob = ordered_json_document::parse(text_b);
+                CHECK((oa.root() == ob.root()) == (equal && !swap));
+                CHECK((oa.root() == ordered_json::parse(text_b)) == (equal && !swap));
+            }
+        }
+    }
+
    SECTION("deep nesting")
    {
        const std::string deep = std::string(100000, '[') + std::string(100000, ']');