How nlohmann/json 3.13.0 got 3× faster¶

Niels Lohmann · October 28, 2026

Since 2016, the README of this library has said this about speed:

There are certainly faster JSON libraries out there. However, if your goal is to speed up your development by adding JSON support with a single header, then this library is the way to go.

That sentence is still true, but the gap is a lot smaller now. Swap the 3.12.0 header for the 3.13.0 one, recompile, and the 182 benchmarks of the library's own suite run 3.3× as fast on an Apple M1 Max (geometric mean; 3.1× on an Intel MacBook Pro from 2016). Not a line of your code changes. Parsing typical documents got 3.5–5.1× as fast, and writing them 2.9–6.6×. The new version also adds json_document, a second way to hold a parsed document, which competes with the fastest C++ and Rust JSON libraries; How it compares has the numbers, including where it loses. The price is compile time: two small example programs take 36–68% longer to compile. A precompiled header or a C++20 module wins back most or all of that in debug builds, and 41% to all of it in optimized builds (The price).

This post is about how. Hardly any of the techniques are new: we borrowed them from simdjson, yyjson, fast_float, Żmij, serde_json, Glaze, SQLite, Boost.JSON, gnulib, or a textbook, and we name the source at each step. What is worth writing about is the combination, and the conditions it had to work under. It is a header-only C++11 library whose 3.x API has been stable for almost nine years, and whose value type is a template over its string, number, object, array, and allocator types. Input comes from strings, streams, FILE*, iterator pairs, wide strings, and user-defined adapters. Output had to stay the same byte for byte, and error messages had to keep pointing at the same byte. The few deliberate changes of output are listed in Appendix F.

Every number comes from a benchmark we ran for this post or from the pull request that made the change. The appendix describes the test files and the setup, has the full tables behind every chart, and lists the pull requests behind each theme.

Fast paths are where bugs hide: they run only for some inputs, so a mistake in one can go unnoticed until the right input comes along. So every fast path was tested against the slow path it replaces, on millions of generated and fuzzed inputs. Both had to return the same value, or fail with the same error message at the same byte. Appendix J lists the checks.

The short version¶

The test files are the three of Milo Yip's nativejson-benchmark that JSON libraries have been measured with for a decade, twitter.json (strings, much of it Japanese), citm_catalog.json (many small objects, indented) and canada.json (floating-point numbers), plus the 56 MB jeopardy.json. Appendix A says what makes each of them hard.

Speedup of 3.13.0 over 3.12.0 for a selection of workloads: the old time divided by the new one, Google Benchmark suite on an Apple M1 Max, median of 12 runs. More is better; the vertical line at 1× is 3.12.0. The times are in Appendix B.

The rows above are chosen to show the range. All 182 benchmarks of the suite:

How many of the suite's 182 benchmarks got how much faster. The two below 1× copy citm_catalog.json and twitter.json without destroying the copy (0.95× and 0.99×). Every row is in Appendix B.

The median benchmark got 2.9× as fast. On an Intel MacBook Pro (Skylake), the median is 2.9× as well, but parsing floating-point numbers gained more (canada.json 6.1× instead of 4.1×) and writing integers less (dump() 1.4–2.3× instead of 2.2–3.7×); Appendix B has both columns. Copying and comparing values gained the least, and the largest gains come from fixing algorithms rather than tuning them (theme 7).

Eight themes did most of the work:

Where the time went: a timeline¶

The suite compares two points. To see which change did what, we compiled one small benchmark against the header of each of the 358 commits on develop that changed json.hpp or json_view.hpp from 3.12.0 to November 7, and of 3.13.0, and ran all of them interleaved. Four of the eight workloads we followed:

• parse() of citm_catalog.json: the lexer's structure and whitespace handling, and building many small objects;
• parse() of canada.json: converting text to floating-point numbers;
• dump() of twitter.json: writing strings;
• from_cbor() of twitter.json: reading a binary format.

Each panel shows the speedup over 3.12.0 after each commit, on a logarithmic scale. The badges above the steps name the theme below that explains them. Moving code around changes a build's speed by a few percent on its own, so only the larger steps mean something.

Speedup over 3.12.0 after each of the 358 commits that changed the headers up to November 7, from 3.12.0 (left) to 3.13.0 (right). More is better; each point is the median of 4 runs of one build, taken in random order (with 4 runs, the mean of the two middle ones). The badges are theme numbers; "U" is a UTF-8 check that came and went. The other four workloads, and every labeled step with its pull request and size, are in Appendix C.

A few things stand out. A percentage for a step is relative to the time just before that step, not to the time of 3.12.0:

• There is no single big step. Parsing citm_catalog.json got faster in about six visible steps. The largest removed a buffer that only error messages read, which alone saved 35% of the time. The next ones added the fast paths for contiguous input, derived positions only on error, and skipped indentation eight bytes at a time; most other changes moved it by a few percent.
• Writing had a few large steps. dump() of twitter.json lost 70% of its time in one change (a write buffer and bulk string copies), and dump() of canada.json 65% in another (Żmij).
• Regressions happen in between. A UTF-8 check of every text string in the binary readers, byte by byte, made from_cbor() of twitter.json 37% slower for 66 commits, until the readers were made lenient again by default, as in 3.12.0, with the check as an option. That decision was about the format specifications, which don't require a decoder to check, not about speed. The cost went with it. It never shipped in a release.
• The input decides how big a step is. Removing the error-message buffer measured 8% on the benchmark of its pull request; on the indented citm_catalog.json it saved 35%, because every space of the indentation went through that buffer.

1 Specialize for each input source¶

This is the function the 3.12.0 lexer called for every byte of input:

char_int_type get()
{
    ++position.chars_read_total;
    ++position.chars_read_current_line;

    if (next_unget)
    {
        // just reset the next_unget variable and work with current
        next_unget = false;
    }
    else
    {
        current = ia.get_character();
    }

    if (JSON_HEDLEY_LIKELY(current != char_traits<char_type>::eof()))
    {
        token_string.push_back(char_traits<char_type>::to_char_type(current));
    }

    if (current == '\n')
    {
        ++position.lines_read;
        position.chars_read_current_line = 0;
    }

    return current;
}

The 3.12.0 lexer's get(), called for every byte of input.

Two counters in memory, a branch for a pending unget(), a call into the input adapter, a push_back() into a buffer that only an error message reads, and a branch for line breaks. The counters are members of the lexer, so the compiler has to load, add, and store them for every byte. A string or a number then copied each byte a second time into their token buffer. That is a clean design: one interface works for a std::string, an std::istream, a FILE*, a pair of iterators over a std::list<char>, or a user's own adapter. But it makes every input pay for the most limited one.

3.13.0 has a dedicated reader for each kind of input and picks it at compile time:

flowchart LR
    src[/"input"/] --> q{{"what is it?"}}
    q -->|"std::string, std::vector,<br>their iterators, pointers,<br>contiguous ranges"| c["contiguous:<br>scan the whole buffer"]
    q -->|"std::istream"| w["window:<br>scan the streambuf's get area"]
    q -->|"FILE*"| f["window:<br>scan the FILE's buffer"]
    q -->|"other iterators,<br>user adapters"| b["byte at a time"]

The four kinds of input the lexer distinguishes in 3.13.0. Everything on the fast paths falls back to the byte path on anything unusual, so the result and the error messages stay the same. The FILE* window needs glibc or Apple's C library; elsewhere, a FILE* is read one character at a time with one lock per parse.

Detecting the input in C++11¶

An input adapter can declare what it supports as static members (supports_seek, supports_bulk_scan, and a few more). The lexer detects them with the detection idiom, so a user's adapter that declares nothing keeps the byte path:

template<typename A> using detect_supports_seek = decltype(A::supports_seek);
template<typename A>
constexpr bool input_adapter_supports_seek(std::true_type) { return A::supports_seek; }
template<typename A>
constexpr bool input_adapter_supports_seek(std::false_type) { return false; }

static constexpr bool seekable = input_adapter_supports_seek<InputAdapterType>(
    is_detected<detect_supports_seek, InputAdapterType> {});
static constexpr bool lazy_position = seekable;

Detecting a static member of the input adapter with the detection idiom: an adapter that doesn't declare supports_seek is not seekable.

Without if constexpr, the flags select overloads by tag dispatch, f(std::integral_constant<bool, lazy_position>{}), which also keeps MSVC from warning about constant conditions. get() still looks like the old one, but its position and token bookkeeping are now calls. For an adapter that can seek, their std::true_type overloads are empty and cost nothing, because the position and the text of a token are derived only when an error message needs them (theme 4). For a std::string, what remains is the branch on next_unget and the adapter's bounds check and pointer increment. For the most common calls, not even that remains: they don't use the lexer at all (One loop instead of two).

Contiguous input¶

Containers with data() and size() over single-byte elements now become a pointer range in every C++ standard, not only with C++20's contiguous iterators. data() and size() alone would be duck typing, so the trait also requires the container's value_type to be the element type behind data(). A type whose size() counts something else keeps the iterator path: slower, but correct. On a pointer range, the strings, the numbers and the indentation after a line break are scanned in bulk (themes 2 and 3). Every fast path defers to the byte path on anything malformed, so error messages and byte positions stay the same, with one deliberate correction (the column after a number at the end of a line). Appendix J describes how that was checked.

A pair of iterators is harder. Before C++20, s.begin() and s.end() of a std::string can't be told from the iterators of any other random-access container, so json::parse(s.begin(), s.end()) was read one character at a time, while json::parse(s) got all the bulk scans. The library now matches the iterator types of std::string and of a std::vector of one-byte integers exactly, and turns the pair into the pointer range it covers. That made parse(s.begin(), s.end()) 1.7–2.0× as fast on the M1, and accept() 2.7–3.1×.

One loop instead of two¶

Even on a pointer range, the parser and the lexer still talked through an interface: the parser asks for a token, the lexer finds it and hands it over, and the parser dispatches on its type. The lexer's state, the current character, a pending unget(), and where the token starts, lives in members, so it goes through memory on every token. After all of the above, json::accept() of twitter.json still took 2.8× as long as json_document::accept() (theme 8), which reads the same bytes.

yyjson's reader shows the alternative: one function that holds the input in a local pointer and in which the grammar, the whitespace and the tokens are one loop. After a :, it tests for the characters that can start a value right there, instead of returning a token type and dispatching on it. 3.13.0 adds such a parser for input in one block of memory, contiguous_parser. Its stack of open arrays and objects is a 64-bit word with one bit per level, so the kind of the enclosing container is a shift away and stays in a register; every 64 levels, the word is pushed to a vector. It doesn't build values itself: it drives the library's own SAX handlers, so parse() builds the same json with the same code as before. It is used by parse() without a callback, by accept(), and by sax_parse() with the user's own handler, each with the default options; everything else keeps the old parser.

The hard part was the errors. A second parser would have to produce the same exception, message and position for every malformed input, a large surface for small differences. So it doesn't report errors at all. It stops at the token the old parser would reject, before it reports that token to the handler, and the old parser runs again on the same input to report the error. parse() gives it a handler that only reports, and sax_parse() one that forwards only parse_error() to the user's handler, so that no event arrives twice. An error at the end of a large input costs parse() about 1.2× as much as before; valid input costs nothing extra.

On the M1, accept() got 1.6–2.0× as fast, sax_parse() 1.4–1.5×, and parse() 10–16% faster, about the same with GCC and Clang on x86-64. In accept(), which needs no values, the parser also skips the digits of a number without computing its value, except for the rare float whose exponent may reach the largest double: another 16% on canada.json. The cost is about 1,000 lines of header, and a translation unit that parses a string and dumps it compiles 3% slower at -O0 and 9% at -O2.

Streams¶

An std::istream already has a buffer: the get area of its streambuf, gptr() to egptr(). The lexer now scans that window in bulk and keeps the current token as a pointer into it. Those are protected members of std::streambuf, and reaching them takes neither a cast nor a derived object:

struct streambuf_access : std::streambuf
{
    static const char* get_ptr(std::streambuf* s) { return (s->*(&streambuf_access::gptr))(); }
    static const char* get_end(std::streambuf* s) { return (s->*(&streambuf_access::egptr))(); }
    static void bump(std::streambuf* s, int n)    { (s->*(&streambuf_access::gbump))(n); }
};

Reaching the protected get area of any std::streambuf through pointers to members, without a cast.

A derived class may name a protected member of its base in a pointer to member, as long as it qualifies the name with its own class. The resulting pointer has the type char* (std::streambuf::*)() const, and access is checked when the name is used, not when the pointer is called, so it works on any std::streambuf*. The common alternative, static_cast<streambuf_access*>(sb)->gptr(), is undefined behavior, because the object is not a streambuf_access.

Before the stream refills its get area, the adapter appends the part of the current token that is in it to a carry buffer; only an error message ever reads the token text back. Nothing is read ahead: the adapter advances gptr() by exactly what the lexer consumes, so the stream is left where it was before. On an M1 Max, parsing from an std::ifstream took 1.8–2.5× as long as reading the file into a string and parsing that, in the pull request's measurement before the change, and 9–16% longer after it. Since then, parsing a string got faster still, because the fused parser of One loop instead of two needs input in one block of memory, so a stream now takes 25–38% longer than the string again (the file read with one read(), parsing twitter.json, citm_catalog.json, and canada.json).

FILE*¶

std::fgetc locks the FILE on every call. 3.13.0 locks it once per parse (flockfile, then getc_unlocked; _lock_file and _fgetc_nolock with the Microsoft runtime). flockfile is recursive, so the binary readers' fread() and a SAX handler's own ftell() keep working. Whereas parsing citm_catalog.json from a FILE* got 4.4× as fast on macOS in the pull request's measurement, with glibc it got only 1.1–1.3× as fast, because glibc's per-call locking is already cheap.

That still left a call per character, while a stream had its get area scanned in bulk. But a FILE has a buffer too, and getc_unlocked() is a macro that reads it: glibc's compares _IO_read_ptr with _IO_read_end, Apple's counts down _r and advances _p. The adapter now reads those same fields and offers the lexer the rest of the buffer as a window, as for a stream. It consumes exactly the characters getc_unlocked() would have returned, so the file is left where it was before, and when the C library is about to refill the buffer, the part of the current token in it is kept for error messages. The idea is gnulib's freadptr(), which knows these fields for many C libraries. Parsing from a FILE* got another 2.0–2.6× faster on the M1 and 1.7–2.2× with glibc on x86-64, and accept() 2.9–4.4×; it is now about as fast as parsing from an std::ifstream. uClibc defines __GLIBC__ but has other fields, so it is excluded, and other C libraries keep reading one character at a time.

Two details: MemorySanitizer builds keep fgetc, because glibc's inline getc_unlocked reads a buffer that uninstrumented glibc filled. And a SAX handler that waits for another thread using the same FILE now deadlocks, as the documentation says. Both paths can be switched off, with JSON_DISABLE_STREAM_BULK_SCAN and JSON_DISABLE_UNLOCKED_FILE_IO; the second turns the FILE window off as well, as it needs the lock for the whole parse.

2 Work on words, not bytes¶

The core of a JSON lexer is "copy bytes until something interesting happens". Theme 1 gave it ranges of bytes. This theme reads them a word at a time.

Eight bytes in a word¶

In a string, the interesting bytes are the closing quote, a backslash, a control character, and anything above 0x7F (UTF-8). This test checks eight bytes at once with ordinary 64-bit arithmetic:

// return a word whose high bit is set in every byte of v that needs
// individual handling; zero if all 8 bytes can be copied as they are
inline std::uint64_t swar_string_special(std::uint64_t v) noexcept
{
    constexpr std::uint64_t ones = 0x0101010101010101ull;
    constexpr std::uint64_t high = 0x8080808080808080ull;
    const std::uint64_t q = v ^ 0x2222222222222222ull; // '"'
    const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\'
    const std::uint64_t has_quote     = (q - ones) & ~q & high;
    const std::uint64_t has_backslash = (b - ones) & ~b & high;
    const std::uint64_t has_control   = (v - 0x2020202020202020ull) & ~v & high; // < 0x20
    const std::uint64_t has_non_ascii = v & high;                                // >= 0x80
    return has_quote | has_backslash | has_control | has_non_ascii;
}

The eight-byte test of the string scanner, in portable C++11.

XOR with the quote turns every quote byte into zero, and (x - 0x01…01) & ~x & 0x80…80 sets the high bit of the bytes that were zero: the "determine if a word has a zero byte" trick from Sean Eron Anderson's Bit Twiddling Hacks. The subtraction's borrow runs only upwards, so it can flag a byte after a real hit, but never one before it: the lowest flag is always right, and the number of trailing zero bits divided by eight is the index of the first special byte.

The code above is the readable form, and it does more work than it needs to. The & ~x of the zero-byte test only keeps bytes from 0x80 out of the result, and has_non_ascii flags exactly those bytes anyway. So all three & ~x terms can go, and the whole test becomes one expression:

return ((q - ones) | (b - ones) | (v - 0x2020202020202020ull) | v) & high;

The same test without the redundant & ~x terms.

Clang finds this reduction on its own; GCC doesn't, and emitted 24 instructions per word instead of 16. With GCC on x86-64, writing it out made validating twitter.json and citm_catalog.json 5–7% faster and dump() 2–3%. The masks of both forms are the same for every pair of adjacent byte values and for 1.3 billion random words.

swar_string_special() on the last eight bytes of "twitter",: one word in, one flag out, and the count of trailing zero bits says how many bytes can be copied before the quote.

Two details make this portable C++11. The word is loaded with std::memcpy, which compilers turn into one load and which, unlike a reinterpret_cast, doesn't break the aliasing rules. And the load always produces the bytes in little-endian order, so the first byte of the text is the lowest byte of the word everywhere; a big-endian target assembles the word from bytes, which compilers fold into a load and a byte swap. Every loop checks that eight bytes remain before it loads, and finishes the remaining one to seven bytes one at a time, so nothing is read past the input.

The same kind of word test now runs in several places:

• Strings in the lexer, including runs of well-formed multi-byte UTF-8, which are validated one sequence after the other without going back to the byte loop.
• Numbers, eight digits per step (theme 3).
• Indentation. After a newline, word ^ 0x2020…20 is zero for eight spaces; the first non-zero byte ends the indentation, and its trailing-zero count says how many spaces of the last word to skip as well. The idea is from Glaze's skip_matching_ws, which compares with the previous line's indentation; comparing with spaces needs no state and works for any indentation width. A version that skipped only whole words validated 5–12% slower and gave nothing on two-space indentation, where the runs are often shorter than a word. With four-space indentation, validating citm_catalog.json got 22–30% faster.
• dump(), for runs of characters that need no escaping, with and without ensure_ascii.
• The binary formats, for the zero byte that ends a BSON key, and for the UTF-8 check of strings with error_handler_t::strict. On the M1, reading canada.json from BSON got 14% faster, and writing twitter.json as UTF-8-checked CBOR 14%.

16 bytes with SSE2 and NEON¶

Where a vector unit is part of the baseline instruction set (SSE2 on x86-64, NEON on 64-bit Arm), the string paths take 16 bytes per step. Three details are worth repeating. A signed comparison byte < 0x20 catches control characters and every byte from 0x80, because those are negative as int8_t. Flipping bit 1 first folds the quote into that comparison too: c ^ 2 maps 0x00–0x1F onto themselves and the quote (0x22) onto 0x20, keeps the bytes from 0x80 negative, and maps every other ASCII byte to 33 or more. So (int8_t)(c ^ 2) < 33 finds three of the four classes of bytes, and only the backslash needs a comparison of its own (Daniel Lemire, "Detect control characters, quotes and backslashes efficiently using SWAR", 2025; ruby/json uses the same trick in its SIMD scan). Neither GCC nor Clang finds this one on its own. And NEON has no equivalent of x86's pmovmskb, which turns a vector of comparison results into a bit mask. The usual replacement, by Danila Kutenin, shifts each 16-bit lane right by four and narrows it to eight bits, which leaves four bits per input byte in a 64-bit integer:

while (e - p >= 16)
{
    const uint8x16_t in = vld1q_u8(p);
    // the backslash, and (c ^ 2) < 33 as int8_t for the quote, the control
    // characters and the bytes from 0x80
    const uint8x16_t special = vorrq_u8(vceqq_u8(in, vdupq_n_u8('\\')),
                                        vcltq_s8(vreinterpretq_s8_u8(veorq_u8(in, vdupq_n_u8(2))),
                                                 vdupq_n_s8(33)));
    // each byte of special is 0x00 or 0xFF; shifting each 16-bit lane right by 4 and
    // keeping its low byte leaves 4 bits per input byte
    const uint8x8_t nibbles = vshrn_n_u16(vreinterpretq_u16_u8(special), 4);
    const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(nibbles), 0);
    if (bits != 0)
    {
        return p + (count_trailing_zeros(bits) >> 2u);   // four bits per byte
    }
    p += 16;
}

The NEON string scan: two comparisons for four classes of bytes, and a narrowing shift in place of pmovmskb.

On twitter.json, the vector scan made json::parse() 5% faster with Clang and 9% with GCC on an M1 Max; on long strings, 19–24%. It did not win from the first byte. On the M1, starting with NEON made short strings 3–4% slower and the rest of the lexer bigger. So ARM runs four 8-byte words first and keeps the vector loop out of line, while x86-64 runs one word first. Inlining the vector code into the lexer cost 5% of validating jeopardy.json on the M1, although no string of that file is long enough to reach it: the scan function, which runs once per token, saved more registers and needed a stack slot.

json_document, the new document type of json_view.hpp, also skips the digits after a decimal point 16 bytes at a time with SSE2: a compare and a bit mask instead of a branch per digit, about 33% of the instructions for a 12-digit fraction. Parsing canada.json got 22% faster on x86-64, a file of 10,000 floats 28–32%.

One row got slower: with GCC 16, citm_catalog.json took 8% longer with the same number of instructions, because more of its hot jumps now crossed a 32-byte boundary. The microcode fix for Intel's JCC erratum makes that slow on our server. Assembling with -Wa,-mbranches-within-32B-boundaries shrinks the loss to about 4%, but a header-only library can't choose the assembler flags of the programs that include it, so we left the code as it is, and all numbers in this post are measured without that flag.

Validating UTF-8 16 bytes at a time¶

Validating UTF-8 16 bytes at a time uses simdjson's "lookup4" algorithm (Keiser and Lemire, 2021): three 16-entry tables, indexed by nibbles of a byte and of the byte before it, each give the set of errors that nibble allows, and a pair of bytes is invalid if all three sets share an error. It needs pshufb, which is SSSE3 and not part of baseline x86-64. The function is compiled with __attribute__((target("ssse3"))), out of line, and called only if cpuid says so. The result of that query is cached in a constant-initialized static std::atomic<int>, so that neither a guarded local static nor a global constructor is needed; threads that ask at the same time all store the same answer. On an error, the code backs up to the start of the sequence and lets the byte path produce the error message. JSON_NO_SIMD turns all vector code off.

Short strings¶

Short strings are a different problem. Most keys and many values are short: citm_catalog.json averages 8 bytes per string and jeopardy.json 14. For them, setting up a scan costs more than the scan. The first step took the idea of SQLite's jsonAppendString(): check each byte of a string of up to 32 bytes against a 256-entry table of bytes that can be written unchanged, and copy the string in one go if they all pass. 32 bytes was the best threshold on the M1; 16 gained less, and 64 lost again. The second step replaced the table, where SSE2 or NEON is available, with overlapping loads, the trick that memcpy implementations such as glibc's use for short copies:

// 16 <= n <= 32: two loads that overlap in the middle cover the whole string
const uint8x16_t a = vld1q_u8(p);
const uint8x16_t b = vld1q_u8(p + n - 16);
const uint8x16_t special = /* '"', '\\', < 0x20 or 0x7F, in a or in b */;
if (vmaxvq_u8(special) != 0)
{
    return false;          // nothing has been written yet; take the general path
}
vst1q_u8(d, a);
vst1q_u8(d + n - 16, b);   // the two stores overlap too, with the same bytes

Writing a string of 16–32 bytes with two overlapping loads and stores (NEON).

The code above on profile_link_color, a key of twitter.json.

On twitter.json, this cut the instructions of dump() by 21%. Shorter strings use the same trick with two overlapping 8-, 4-, or 2-byte loads, so no memcpy with a length known only at run time is left.

Escapes from a table¶

When the scanner stops at a quote, a backslash, or a control character, the writer copies its escape sequence from a table of 8-byte rows, padded with zeros, with the length in the last byte:

{{'\\', 'u', '0', '0', '0', '0', 0, 6}},   // 0x00
{{'\\', 'n', 0, 0, 0, 0, 0, 2}},           // 0x0A
{{'\\', '"', 0, 0, 0, 0, 0, 2}},           // 0x22

std::memcpy(write_buffer.data() + write_buffer_pos, escape.data(), 8);   // always 8 bytes
write_buffer_pos += escape[7];                                         // keep only the real length

Three rows of the escape table, and how a row is written: always 8 bytes, then the real length.

One 8-byte store and one add, with no branch on the length; the padding lies behind the write position and is overwritten by the next write. The writer makes sure that 8 bytes are free before it copies. The idea is from orjson's QUOTE_TAB, which orjson took from sonic-rs. Although dumping 100,000 log lines full of \n, \t, and quotes got 23% faster on an M1 Max, most of that came from writing straight into the write buffer instead of an intermediate one. The table alone gained 2–7%.

3 Own the number conversions¶

3.12.0 converted every floating-point number with strtod and every integer with strtoull or strtoll, after copying the token into a buffer. strtod is correct, but general: it consults the locale for the decimal point, sets errno, and handles any length of input. std::from_chars would avoid the locale, but it doesn't exist before C++17, and libc++ has its floating-point overloads without defining __cpp_lib_to_chars, so macOS builds could not detect it.

canada.json, a GeoJSON file of 111,126 numbers, shows what that costs. 108,528 of its numbers have 16 or 17 significant digits, like -65.613616999999977. The classic shortcut is Clinger's fast path (Clinger, 1990): write the number as its digits w times a power of ten, convert w to a double, and multiply or divide it by that power of ten. If both are exact as double, that is, w is at most 2⁵³ (9,007,199,254,740,992) and the power at most 10²², the operation rounds only once, and the result is the correctly rounded double. For most of canada.json's numbers, w doesn't fit into 53 bits.

3.13.0 converts float and double numbers itself, without the C library and without the locale:

flowchart TB
    t["number token"] --> d{{"at most 19<br>significant digits?"}}
    d -->|"yes"| c{{"digits and power of ten<br>exact as double?"}}
    c -->|"yes"| cl["Clinger's fast path:<br>one multiplication or division"]
    c -->|"no"| el["Eisel-Lemire:<br>128-bit integer arithmetic"]
    d -->|"no"| el2["Eisel-Lemire on the first 19 digits"]
    el2 --> amb{{"w and w+1 round<br>to the same double?"}}
    amb -->|"yes"| done(["done"])
    amb -->|"no"| big["compare with the exact midpoint<br>(big-integer arithmetic)"]

How 3.13.0 converts a decimal token to a double. Of canada.json's 111,080 floating-point numbers, 9,742 take Clinger's fast path and 101,338 Eisel-Lemire; none has more than 19 digits.

Eight digits at a time¶

On contiguous input and stream windows, the lexer accumulates the value while it validates the token, so the digits are read once. The core is fast_float's eight-digit conversion: three multiplications instead of eight.

// v: eight ASCII digits, the first digit in the lowest byte
std::uint32_t parse_eight_digits(std::uint64_t v) noexcept
{
    v = ((v & 0x0F0F0F0F0F0F0F0Fu) * 2561u) >> 8u;            // 2561 = 10 * 2^8 + 1
    v = ((v & 0x00FF00FF00FF00FFu) * 6553601u) >> 16u;        // 6553601 = 100 * 2^16 + 1
    v = ((v & 0x0000FFFF0000FFFFu) * 42949672960001u) >> 32u; // 10000 * 2^32 + 1
    return static_cast<std::uint32_t>(v);
}

fast_float's conversion of eight ASCII digits to their value, with three multiplications.

parse_eight_digits() on "12345678": multiplying by 10·2⁸ + 1 adds ten times each byte to the byte above it, and the shift and the next mask keep every second lane.

Checking that all eight bytes are digits takes one more expression. A digit is a byte from 0x30 to 0x39. In hexadecimal, its upper half is 3 and its lower half is at most 9. Adding 6 tests the lower half: 0x39 + 6 = 0x3F still has the upper half 3, but 0x3A + 6 = 0x40 carries into it. So a byte is a digit exactly if its upper half is 3 both before and after adding 6, and with masks, that is one comparison for all eight bytes. An integer of up to 18 digits always fits into an int64_t, and 19 digits into a uint64_t, so they need no overflow check.

Clinger's fast path¶

Clinger's fast path is guarded by more than the two conditions above:

#if !defined(FLT_EVAL_METHOD) || FLT_EVAL_METHOD == 0
if (exponent >= -22 && exponent <= 22 && w <= (std::uint64_t{1} << 53))
{
    double value = static_cast<double>(w);
    value = exponent < 0 ? value / exact_power_of_ten(-exponent)
                         : value * exact_power_of_ten(exponent);
    return negative ? -value : value;
}
#endif

Clinger's fast path: one correctly rounded multiplication or division, only without x87 extended precision.

With x87 arithmetic (FLT_EVAL_METHOD != 0), the intermediate result has extended precision and is rounded twice, so those targets skip the fast path and take the integer-only route below.

Eisel-Lemire¶

Eisel-Lemire, adapted from fast_float's compute_float, handles everything else with integers only: it multiplies the digits with a 128-bit approximation of 5^q from a table and takes the mantissa from the top of the product. For up to 19 digits, that always suffices (Mushtak and Lemire, 2023). Longer numbers, rare in JSON, are bracketed by their first 19 digits and that plus one; only if the two round differently are all digits compared with the exact midpoint. Appendix I has the details. Because it is integer arithmetic, the result no longer depends on the C locale. (JSON always uses ., but strtod expects the locale's decimal separator, so 3.12.0 replaced the . before calling it, which failed for separators longer than one byte.) Appendix J describes how the conversion was checked against strtod.

Writing: Żmij¶

For output, 3.12.0 used Grisu2, which is fast but sometimes writes a 17th digit where 16 suffice. 3.13.0 ports Żmij by Victor Zverovich to C++11. It writes the shortest decimal that reads back as the same double, and the closest one if there are several. Its digit conversion, by Xiang JunBo, is the parser's divide-by-multiplying idea run backwards on all lanes of a word at once (Appendix I shows it), and with SSE2 or NEON, Żmij's vector code (the NEON version by Dougall Johnson, the SSE version by Tobias Schlüter) converts all 16 digits in one vector register.

One stall turned up on the way. To make room for the decimal point, the code first wrote all 16 digits, as two 8-byte stores, and then moved the digits after the point one byte to the right: it loaded them and stored them one byte further. That load came right after the two stores and read part of each. A CPU can hand a load the data of one store that hasn't reached the cache yet, but it can't combine two, so the load had to wait until both stores were written. The commit measured that wait as about 25% of the time to write a double. The vector version avoids memory: it keeps the digits in a register, moves the ones after the point up by one byte there, puts '.' into the gap, and stores all 16 bytes once.

In the pull request's measurement, dump() of canada.json got about 3× as fast, on an M1 Max as well as on an x86-64 server with GCC and Clang. The changed output is listed in Appendix F: 0.14% of random doubles are written differently, all of them with the same or fewer digits, and all of them read back as the same value.

4 Don't do unnecessary work¶

A lot of what the 3.12.0 parser did was correct, cheap per call, and read by nobody in the common case.

The token buffer¶

The old get() of theme 1 pushed every character into token_string, a buffer that only the "last read" part of an error message uses. For seekable input, the lexer now remembers the offset where the token started and copies the text from the input if an error happens. This is the 35% step of the timeline: in the indented citm_catalog.json, every space of the indentation went through the buffer. For streams, the carry buffer of theme 1 does the same job; the per-character copy had been about 20% of what was left of a stream parse.

accept() went one step further: it doesn't copy strings and keys at all. The lexer and the parser take a template parameter that drops values, so accept() gets its own instantiation without the copies. That made validating twitter.json 11–13% faster on x86-64, and 19 files of simdjson's examples 12% by geometric mean.

Positions only on error¶

For input that can be seeked, the lexer no longer counts lines and columns. The byte offset is the distance from the start, and lines and columns are recomputed only when an error message needs them:

std::size_t count_consumed_lines(std::size_t last_index, std::size_t& line_start) const
{
    const auto to = std::next(begin, last_index);
    using reverse_it = std::reverse_iterator<IteratorType>;
    const auto last_newline = std::find(reverse_it(to), reverse_it(begin), '\n');
    line_start = std::distance(begin, last_newline.base());   // column: offset - line_start
    return std::count(begin, last_newline.base(), '\n');      // line: newlines before it
}

Line and column of an error, computed from the byte offset only when an error message needs them.

That is linear in the input, but only an error pays for it. It is what serde_json's slice reader does (position_of_index() with memrchr and a newline count). A pending unget() is a flag, and the offset subtracts one for it, so unget() does no position work at all. Measuring the two parts of the old get() separately on an M1 showed that the newline branch cost more than the counters: removing only the branch saved 16% of validating an indented twitter.json, removing only the increments 4%. Parsing citm_catalog.json got 12% faster.

The text of float tokens¶

The lexer copied every float token into a buffer so that a SAX handler's number_float(val, s) could see the original text: an out-of-line std::string::append per float, about 10% of parsing canada.json. The handlers behind parse() and accept() never look at it. The parser now recognizes them by their exact type:

template<typename SAX> struct sax_ignores_number_text : std::false_type {};
template<typename B, typename IA, bool Collect, typename L>
struct sax_ignores_number_text<json_sax_dom_parser<B, IA, Collect, L>> : std::true_type {};
template<typename B>
struct sax_ignores_number_text<json_sax_acceptor<B>> : std::true_type {};
// json_sax_dom_callback_parser likewise

// at the call site
sax->number_float(res, number_float_text(sax));   // an empty string for the handlers above

Recognizing the library's own SAX handlers, which never read the text of a float token.

A class derived from json_sax_dom_parser does not match the specialization, so it keeps getting the exact text. For every other handler, the lexer only records a pointer to the token and its length, and copies on the first get_string(). That is safe for contiguous input, which has to outlive the parse anyway; in a stream's window, the next refill could overwrite the text, so there the copy is made at once. The first version put the "is there pending text?" check into the get_string() that keys and strings use too, and lost 2–3% on string-heavy input. Keys and strings now call a separate function without the check. Parsing canada.json got 9–11% faster.

A copy that should have been a move¶

from_cbor() and the other binary readers ended like this:

return res ? result : basic_json(value_t::discarded);   // 3.12.0

The end of the 3.12.0 binary readers: the conditional operator copies result.

result is a local, but the conditional operator's operand is an lvalue expression, and the implicit move on return applies only to a returned name. So every successfully parsed value was deep-copied once and then destroyed. 3.13.0 assigns discarded to result on failure and returns result by name. Decoding a 2 MB CBOR document went from 26.99 ms to 14.65 ms. The copy was also a stack-overflow risk: the copy constructor recursed once per level, for a value the reader had already accepted.

ordered_json growth¶

ordered_json stores its members in a std::vector<std::pair<const std::string, json>>. Because the key is const, the pair's "move" constructor copies the key and can throw. When std::vector grows, it uses std::move_if_noexcept to keep the strong exception guarantee, so for this element type it copies every element, including the whole subtree behind every value. The library now grows the storage itself, in two phases:

// phase 1, may throw but only touches tmp:
// copy the keys, default-construct the values, add the new element
Container tmp(this->get_allocator());
tmp.reserve((std::max)(size_type{1}, 2 * this->size()));
for (const auto& element : *this)
    tmp.emplace_back(std::piecewise_construct, std::forward_as_tuple(element.first),
                     std::forward_as_tuple());
tmp.emplace_back(std::forward<Args>(args)...);
// phase 2, cannot throw: move-assign the values, adopt the storage
auto it = tmp.begin();
for (auto& element : *this) { it->second = std::move(element.second); ++it; }
Container::swap(tmp);

How ordered_map grows its storage in two phases, without copying the values.

The new element is constructed before any value is moved, so j.emplace("new", std::move(j["old"])) still works, and the strong guarantee holds without a try, which matters because the library also builds with -fno-exceptions. Parsing twitter.json into ordered_json needs 33% of the allocations and, in the pull request's measurement, about 50% of the time. Unfortunately, compilers with the EDG front end (Intel's icpc, NVIDIA's nvc++) consider that pair's move constructor noexcept. There, the vector already "moves", the new path is not taken, and a key copy that throws during growth calls std::terminate.

Containers built one element at a time¶

The DOM parser used to push_back every element into its array, and insert every member into its object as soon as it was parsed. Now all open arrays share one stack of elements; when an array of up to 16 elements ends, it is constructed at its exact size from a range of move iterators, one allocation. ordered_json objects collect up to 32 members the same way, with a linear duplicate-key check over that short slice, and larger objects get a side table of hashes for that check from 64 members on. An ordered_json object with very many members went from 43 million instructions to 1.9 million, and ordered_json now parses 1–15% faster than json in our suite, since json's std::map still allocates a node per member. The binary readers know their sizes up front and use a separate handler without the stack; sharing it had made them 5–12% slower with GCC.

The binary readers also stopped taking every element through their general dispatch. When an array opens, the CBOR and MessagePack readers read the numbers, booleans, and nulls that follow in one loop, and the UBJSON and BJData readers do the same for arrays with a type marker. Reading an array of a million integers from CBOR or MessagePack got 42–47% faster, and canada.json 10%. Arrays of one element skip the loop: in 1,000 arrays nested in each other, each entered it only to find another array, which cost 6–8%. Now the loop costs nested arrays nothing measurable, and costs 100,000 arrays [1,2] at most 1.5%.

Smaller ones¶

The parser checks for the expected : or , before it falls into the lexer's general token switch (−7% on citm_catalog.json, a contribution by @mvanslobbe), and to_cbor() no longer creates a temporary basic_json for every object key it writes (−55% on twitter.json, by @alexprabhat99). And the binary readers wrote every byte of a string twice: they grew the string with resize(), which fills it with zeros, and then copied the bytes over the zeros. The zero fill alone was 5% of the instructions of from_cbor(). A string that the input holds in one block is now appended from it directly, after its claimed length has been checked against the input, so a huge claimed length still allocates nothing. Reading twitter.json and citm_catalog.json from CBOR and MessagePack got 5–9% faster on the M1, 3–9% on x86-64.

5 Read the assembly¶

A good part of 3.13.0 came from reading what the compiler made of the hot loops, with Clang's -Rpass-missed=inline, profiles, and instruction counts. None of the following shows up in the source as a problem.

A stack canary on every token¶

The function that dispatches on the first byte of a token built the expected text of true, false, and null in local arrays:

case 't':   // 3.12.0
{
    std::array<char_type, 4> true_literal = {{'t', 'r', 'u', 'e'}};
    return scan_literal(true_literal.data(), true_literal.size(), token_type::literal_true);
}

The 3.12.0 lexer's dispatch for true: a local array, and with it a stack canary.

With -fstack-protector-strong, which Ubuntu's GCC enables by default and many distributions use for their packages, any local array gives a function a stack canary: the prologue loads a secret (on x86-64 Linux from thread-local storage) and stores it next to the locals, and every return compares it. Even worse, that happens on every call, also for the [, , and " tokens that never touch the array. The canary's load alone was 11% of the function's samples. (Plain -fstack-protector only protects character arrays of at least 8 bytes, so it didn't trigger here.) Now the literal is a string literal whose length is a template parameter, compared on contiguous input with one memcmp of constant size, and the dispatch has no canary and no array on the stack:

case 't': return scan_literal("rue", token_type::literal_true);

template<std::size_t N>
JSON_HEDLEY_NEVER_INLINE token_type scan_literal(const char (&rest)[N], token_type return_type);

The 3.13.0 dispatch: a string literal whose length is a template parameter.

scan_literal() is deliberately out of line: inlined, it made Clang's dispatch grow from 233 to 278 instructions, and slower. Together with the next change, validating canada.json got 10.5% faster with GCC on x86-64. The same change replaced the parser's std::vector<bool> nesting stack, whose push_back GCC called out of line, with std::vector<std::uint8_t>; a bit vector saves memory that a stack of nesting levels doesn't need.

Exceptions that block inlining¶

A function that can throw builds the message, allocates the exception, and needs a landing pad for every temporary, even if the throw never happens:

if (JSON_HEDLEY_LIKELY(ptr != nullptr)) { return *ptr; }   // 3.12.0, get_ref()
JSON_THROW(type_error::create(303, detail::concat(
    "incompatible ReferenceType for get_ref, actual type is ", obj.type_name()), &obj));

The throw in 3.12.0's get_ref(): building the message makes the function too big to inline.

That made get_ref(), operator[], at() and the iterator operators too big to inline, and gave callers large frames (192 bytes for get_ref() with GCC). 3.13.0 moves construction and throw into helpers marked [[noreturn]] and never inlined, so the hot function keeps a test and a call:

if (JSON_HEDLEY_LIKELY(ptr != nullptr)) { return *ptr; }
throw_incompatible_reference_type(obj);   // JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE

The same function in 3.13.0: a test and a call of a helper that is never inlined.

That fixed 27 throw sites. Iterator arithmetic got 6× as fast with Clang, and std::sort over a json array of 10,000 integers in user code 6–9% faster.

The value's own lifecycle had the same problem: the destructor, the copy constructor, and operator== contained the code for every type, so they were too big to inline even for a number. Clang reported about 380 call sites where they were not inlined. Now the destructor and the copy constructor start with an inline test whether the value owns memory, which Clang compiles to one bit test (1 << type & 0x10E), operator== with one for two scalars of the same type, and the rest is out of line. j == 5 went from 3.55 ns to 0.65 ns with Clang. It is not free: a test file of extern "C" wrappers around the library's functions, which we compile to check what gets inlined, grew by 13% at -O2, and with GCC, operator== on large documents got about 7% slower.

Aliasing¶

C++ allows a char to access any object, so a store through a char pointer may change any object whose address the compiler can't rule out. In a writer that keeps its write position pos in a member, buffer[pos++] = c therefore costs more than it looks: the compiler can't prove that the store of c didn't overwrite pos itself, so it writes pos to memory before every character and loads it again after. A local variable whose address never escapes can't be reached by any store, and stays in a register:

std::size_t pos = write_buffer_pos;   // a local: no char store can reach it
// ... write_buffer[pos++] = digit; ...
write_buffer_pos = pos;               // published once at the end

Keeping the write position in a local, so that the compiler can keep it in a register.

On a 4 MiB binary value, dump() went from 18.0 ms to 7.4 ms with this change alone. json_document's writer and the binary writers keep their cursors in locals for the same reason, and publish them before any call that could see them.

Loop shape beats branch count¶

A change that removed one branch per whitespace byte made scanning long runs of indentation 1.75–3.2× slower with Clang and GCC. It had rewritten do { get(); } while (is_whitespace()); into get(); while (is_whitespace()) { get_ignoring_pending_unget(); }, and the assembly showed why it lost: neither compiler kept the input's read and end pointers in registers any more, and both reloaded them from memory on every character. Restoring the do-while shape, get(); if (is_whitespace()) { do { ... } while (is_whitespace()); }, fixed it. We don't know why the while loop lost the registers, only that the do-while shape gets them back.

Inlining that disappears¶

Inlining is a budget per function. Once the string writer had grown, GCC 13 and 16 stopped inlining two tiny, unchanged helpers into the pretty-printing loop, and indented output got up to 13% slower. They are now forced inline. A change in one place silently made an unrelated loop slower; only a benchmark of indented output noticed.

Branches the compiler chose¶

json_document writes most floats of canada.json by deciding, from where the double lies relative to its source text, which of two or three candidate digit strings to write. The conditions of those selections depend on the digits, so they are as good as random:

const bool up = from > 37 * one;
const bool shorter = biased_distance(from - (up ? 10 * one : 0)) < radius;
const std::uint64_t a = w - r10 + (up ? 10u : 0u);
return shorter ? a : b;

Selecting a candidate digit string: GCC 13 compiled the conditional operators as jumps.

Clang compiled the ?: as conditional moves; GCC 13 compiled them as jumps. With nothing to tell it that the conditions are unpredictable, a branch that skips some work looks cheaper than computing both sides. Callgrind's branch simulation counted 220,000 mispredictions per dump of canada.json with GCC, against 97,000 with Clang. Written as masks, which have no branch form to reconstruct, the selections stay branch-free with GCC and Clang:

// all ones or zero
std::uint64_t mask_if(bool b) { return std::uint64_t{0} - static_cast<std::uint64_t>(b); }
std::uint64_t select_if(bool b, std::uint64_t a, std::uint64_t c)
{
    return c ^ ((a ^ c) & mask_if(b));
}

const bool shorter = biased_distance(from - (mask_if(up) & (10 * one))) < radius;
const std::uint64_t a = w - r10 + (mask_if(up) & 10u);
return select_if(shorter, a, b);

The same selections written as masks, which stay branch-free with GCC and Clang.

With GCC on x86-64, the mispredictions went down to 98,000, and the dump of canada.json got about 15% faster with 5% more instructions. With Clang, the time stayed the same. We found this one while writing this post.

Code size as a cost¶

get<std::array<int, 1000>>() used a pack expansion, {{ j.at(Idx).get<T>()... }}. A pack expansion is unrolled before the optimizer decides anything, so once at() had become small enough to inline, Clang emitted one function of about 28,000 instructions (with 3.12.0, about 8,000): a thousand inlined copies of the bounds and type checks. Its time moved from 657 to 957 ns just by aligning the code differently. A loop needs 49 instructions.

6 Write through a cursor¶

In 3.12.0, every dump() created its output adapter with std::make_shared, built a 512-character indentation string, and the serializer wrote every structural character through a virtual call:

o->write_characters("{\n", 2);
o->write_characters(indent_string.c_str(), new_indent);
o->write_character('\"');

How the 3.12.0 serializer wrote an object's opening brace and the start of a key: three virtual calls.

The writing side changed in five steps, one per subsection below:

flowchart LR
    subgraph before["3.12.0"]
        direction LR
        d1["dump()"] --> a1["make_shared<br>output adapter"] --> v1["virtual call<br>per write"] --> s1[("std::string")]
    end
    subgraph after["3.13.0"]
        direction LR
        d2["dump()"] --> a2["adapter on<br>the stack"] --> b2["1 KiB buffer,<br>bulk append"] --> s2[("std::string")]
    end

The text serializer's output path before and after the first two steps.

A buffer in the serializer¶

All writes go into a 1 KiB buffer that is flushed in bulk, so the virtual call happens once per KiB, and neighboring writes can be merged. Runs longer than the buffer are written straight through, so they are not copied twice.

void put_char(char c)
{
    if (JSON_HEDLEY_UNLIKELY(write_buffer_pos == write_buffer.size())) { flush(); }   // virtual
    write_buffer[write_buffer_pos++] = c;
}

Writing one character: the virtual call happens only when the buffer is full.

No allocation per call¶

The adapter lives on the stack, and there is no indentation string any more: dump(4) writes the spaces straight into the buffer. A compact dump() of a small object went from 4 heap allocations to 1 with libstdc++ (3 to 1 with libc++): the returned string, and none if it fits into the string's small buffer.

A concrete sink for the binary writers¶

The overloads of to_cbor() and friends that return a std::vector use a writer that is a template over the sink type, with no virtual call. Byte order is now swapped with __builtin_bswap* (or _byteswap_* with MSVC) instead of std::reverse on a byte array: compilers can recognize the reversal as a byte swap in a small function, but in the CBOR number path, Clang emitted no bswap at all. Writing into a vector also allows to_bson() to make one pass instead of two, because a vector can be written to at any position: BSON puts the size of every document in front of it, so the writer used to walk the value once for the sizes and once to write. It now writes a placeholder and fills in the size when the document is complete, 12–20% faster on the M1. Streams and other output adapters, which can only append, still take two passes.

A cursor instead of vector::insert()¶

After the previous step, writing an array of integers as CBOR was still no faster than in 3.12.0, and slower with GCC. The profile said why: each number's one to eight bytes were appended with vector::insert(), which both standard libraries implement as an out-of-line insert-anywhere routine; it took 30% of the time, more than encoding the numbers. The sink now keeps a cursor and a limit pointer, reserves space for a whole run of numbers at once, and writes through the cursor:

CharType* const start = oa.reserve(max_run * integer_encoding_space);   // one check per run
CharType* out = start;
for (n = 0; n < max_run && it != end && it->is_number_integer(); ++n, ++it)
{
    out += encode_integer(out, *it);
}
oa.commit(out - start);

Encoding a run of integers through a cursor, with one capacity check for the whole run.

The cursor needs the vector to be as long as the space it writes into, so the sink grew it with resize() to its whole new capacity. That zeroes the new bytes, so every doubling zeroed megabytes of fresh memory at once: it faulted in all the pages and pushed them out of the cache before the writer got there. The writers had gained much less on x86-64 than on the M1 (writing an array of numbers as CBOR got 2.4× as fast on a Skylake laptop and 4.8× on the M1 Max), and this was part of the reason: page faults, zeroing and copies on reallocation of a 9–80 MB output were 30–76% of the 3.13.0 time on x86-64. The sink still reserves the same capacity, but now resizes the vector only to 64 KiB ahead of the cursor, a window that is still in the cache when the writer's stores arrive. On x86-64, writing large arrays of numbers got 9–26% faster; on the M1, 1–2.5%.

Integers without branches¶

The last step also took the branches out of encoding an integer. After the bounds checks, mispredicted branches accounted for about 40% of the remaining time: the type, the sign, and the 1-, 2-, 4-, or 8-byte size class of each integer in a stream are hard to predict. The encoder reads signed and unsigned integers through one masked path, looks the size class up from the number of leading zero bits, and stores the head as one byte plus one 8-byte word, of which the caller keeps only as many bytes as the class needs:

// 1, 2, 3 or 4 if x fits 1, 2, 4 or 8 bytes, without a branch
static unsigned integer_size_class(const std::uint64_t x) noexcept
{
    static constexpr std::array<std::uint8_t, 64> size_class_by_leading_zeros = { /* ... */ };
    return size_class_by_leading_zeros[count_leading_zeros(x | 1u)];
}

std::size_t encode_first_byte_and_tail(CharType* out, std::uint8_t first_byte,
                                       unsigned size_class, std::uint64_t value) const noexcept
{
    out[0] = to_char_type(first_byte);
    const unsigned unused_bits = 64u - ((8u << size_class) >> 1u);
    const std::uint64_t tail = is_little_endian ? byteswap(value) >> unused_bits
                                                : value << unused_bits;
    // always 8 bytes; the caller keeps the first 0, 1, 2, 4 or 8
    std::memcpy(out + 1, &tail, sizeof(tail));
    return ((std::size_t{1} << size_class) >> 1u) + 1;
}

Encoding the head of a CBOR integer without a branch: the size class from a table, the tail as one 8-byte store.

The table is a table because Clang vectorized the equivalent comparisons into slower code, and the unconditional 8-byte store is why reserve() asks for 9 bytes per integer. Writing an array of small signed integers as CBOR, which had not gotten faster before, is now 4.2× as fast as with 3.12.0 with Clang on an M1 Max, and was 2.8× as fast with GCC 13 on x86-64 when the change was made. One more lesson from these changes: the loop over an array is a separate function that is never inlined. Inlined into the function that writes any value, its register needs made every call of that function save more registers, once per nested value, which cost citm_catalog.json 13%.

7 Fix the algorithm first¶

None of the micro-optimizations above matter much next to an algorithm that is quadratic. Seven operations were quadratic or worse in 3.12.0. They are fixes of cases most programs never hit, not tuning, so they are not part of the 3.0× above (except flatten(), which the suite happens to measure); Appendix D has the numbers:

Operations that were quadratic or worse in 3.12.0, on a logarithmic scale; each arrow leads from the time with 3.12.0 to the time with 3.13.0. Less is better.

Comparing in one pass¶

The most instructive one is operator<. Before C++20, json's operator< on two arrays or objects called the operator< of std::vector or std::map, which is std::lexicographical_compare. For every pair of elements it asks a < b and, if that is false, b < a. When the elements are themselves containers, both questions recurse into the whole subtree, and when they are equal, both have to go all the way down. So each level of nesting doubles the work: two equal values nested 20 levels deep took 5.7 ms, 30 levels 3.8 seconds, and 40 levels would take about an hour. That matters to anyone who keeps json values in a std::set, uses them as std::map keys, or sorts them. Since C++20, operator< is derived from operator<=>, which asks once and was never affected.

3.13.0 compares in one pass over an explicit stack of iterator pairs and gets "less", "equal", "greater" or "unordered" for each pair at once, so the ordering question is asked once per element. We found two subtleties by comparing 68,121 results with the old code, not by reading it. std::lexicographical_compare steps over a pair it cannot order (a NaN) and goes on, while the three-way version stops there, so the pass has a flag for which behavior the caller needs. And object keys must be compared with operator== when testing equality, not with the object's comparator: for ordered_json, that comparator is std::equal_to, and using it as "less" reports every equal key as smaller. The single pass has a price: ordering flat arrays of numbers got 27% slower, while flat objects got 34% faster, in the pull request's measurement.

Bounded recursion¶

The other side of this theme is depth. A value nested 100,000 levels deep used to crash dump(), the copy constructor, std::hash, flatten() and the BSON and UBJSON writers with a stack overflow, and in optimized builds operator==, diff() and the other binary writers at 300,000 levels; servers that copy JSON from clients hit that. Other libraries limit the depth of what they parse (Boost.JSON at 32, serde_json at 128, simdjson at 1024) or parse and write iteratively (yyjson), and RapidJSON parses recursively unless asked otherwise. We tried making every walk iterative, but that made dump(), copying and merging up to 20% slower on ordinary documents. So 3.13.0 bounds the recursion instead:

constexpr std::size_t recursion_depth_limit() noexcept { return 128; }

static void destroy_bounded(basic_json& v, std::size_t depth) noexcept
{
    if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
    {
        destroy_walk(v);   // deeper than 128 levels: no recursion at all (below)
        return;
    }
    // for each child that is a container: destroy_bounded(child, depth + 1)
    free_container(v);
}

Bounded recursion: recursive down to 128 levels, iterative below.

The first 128 levels run the old recursive code, and only below that does an explicit stack, or the walk below, take over. A document nested less deeply pays one comparison per container. Testing for equality walks std::vector and std::map itself and passes the depth as an argument. Copying can't do that cheaply: the containers' copy constructors call ours for every element, and a copy that inserts element by element so that it can pass the depth measured 4–16% slower, because std::map's own copy clones its tree without comparing a single key. So copying counts the levels in a thread_local counter instead, about ten nanoseconds per copy of a small object. Three more operations use the same counter: converting a value from another basic_json specialization, the ordering comparisons (<, <=> and the operators derived from them), and equality for custom container types, whose own operator== calls ours.

Destroying without a stack¶

The destructor's walk for deep values uses pointer reversal, the idea behind the Schorr-Waite graph marking algorithm: while it empties a container, the slot of that container's last child holds the link back to the container's parent, so the way back needs no memory (Appendix I shows the loop). The walk allocates nothing and calls nothing that can throw, which fixed two older bugs: the previous destructor flattened children into a std::vector, which ignored the allocator type and could let std::bad_alloc escape from a noexcept destructor. A test checks with a counting allocator that destroying any value allocates nothing. Against 3.12.0, destroying a parsed document is 1.24–1.55× as fast in our suite, and destroying an array of one million small objects 1.39×.

8 Build a different data structure¶

After all of the above, json::parse() is still bound by what it builds: one heap allocation per string, array and object, and a std::map node per member, all of which have to be freed again. Many programs only parse a document, read some fields and throw it away, or change a few values and write it back. For them, 3.13.0 adds json_document and json_view in a new header, <nlohmann/json_view.hpp>. The two figures below show what each parser makes of this 54-byte text:

{"port": 6379, "price": 19.90, "tags": ["eu", "a\"b"]}

The example text of the next two figures.

What json::parse() builds: a tree in which every object, array, and string is a separate heap allocation (dashed). The float is converted to the nearest double while parsing, so the original text 19.90 is gone: the stored value is 19.899999999999998578…, which dump() writes as 19.9, the shortest text that reads back as the same double.

What json_document::parse() builds from the same text: one flat index of nine 16-byte nodes in document order, which point back into the source text, plus a buffer for the one string with an escape. A json_view is two pointers, here to the array.

One flat index¶

The design is not ours either. simdjson's tape and yyjson's immutable document are both flat arrays of fixed-size entries in document order instead of trees of separate allocations, and yyjson's entries are 16 bytes, too. Ours:

struct node
{
    node_kind     kind;    // object = 1, array = 2, string, number, ...
    std::uint8_t  flags;   // escaped, edited, moved, ...
    std::uint16_t extra;   // numbers: digits before and after the point; big objects: hash index
    std::uint32_t off;     // where the value starts in the source text (or in the arena)
    std::uint32_t len;     // strings and float tokens: bytes; containers: number of elements
    std::uint32_t next;    // containers: number of nodes in the subtree, including this one
};
static_assert(sizeof(node) == 16, "node must stay 16 bytes");

A node of json_document's index: 16 bytes for every value.

• Skipping is an addition. A container stores the size of its subtree, so the value after it is n + (is_container(*n) ? n->next : 1), and the members of an object are a key node followed by the value's subtree. There are no pointers in the parsed index (edits add links, below); the parser fills in next when the container closes. is_container() is a single subtraction and comparison, kind - 1u <= 1u, which is why objects and arrays are numbered 1 and 2.
• Strings stay in the input. Only strings with escapes are decoded, into an arena. A document makes only a handful of allocations, no matter how many values it has. The price is that the text must outlive the document, unless the document owns it: parse_copy() copies the text, and parsing an rvalue std::string moves it into the document. Input that can't be read as one contiguous block of bytes, such as a stream or a FILE*, is read into a buffer that the document owns, too. Offsets are 32 bits, which limits a document to 4 GiB.
• Integers are converted while their digits are in the cache, into the last 8 bytes of their node.
• Floats stay text until you read them. The node records the number of digits before and after the point, and reading converts with the same Clinger/Eisel-Lemire code as json::parse. We tried converting them during the parse. Although parsing canada.json and reading every number got 4–7% faster, parsing alone got 36–90% slower, so we dropped it. A default dump() writes the shortest form of a float, as json does; number_format::source copies the source text of every number instead.
• SIMD and tables as in theme 2, and \u escapes are decoded with a table, as in yyjson.
• A hash index for big objects, an idea from Boost.JSON. An object with 128 or more members gets an open-addressing table after parsing; looking up every key of a 10,000-member object went from 59.8 ms to 0.16 ms. The hash is not seeded, so crafted keys could make building the table quadratic. Probing is therefore capped at 64 slots: a key that would need more drops the table, and that one object falls back to a linear search. Parsing can't become quadratic, and a lookup is never slower than the linear search of a small object.
• Validating writes no index. json_document::accept() runs a second instantiation of the same builder with a template parameter that turns every store off, so the checks are the same code, but no node is written and no escaped string is decoded. On x86-64 that made it 17–31% faster on twitter.json, citm_catalog.json and canada.json, and the 56 MB jeopardy.json no longer needs 52 MB of fresh memory just to be validated.
• First duplicate wins. Lookups return the first of duplicate keys, as yyjson and simdjson do, because returning the last one meant scanning every object to its end and made lookups in small objects 1.6–3.4× slower. json::parse() keeps the last one, so for a text with duplicate keys, the two can disagree; the documentation says so.

It can be edited, too¶

A read-only index is the easy case, and a fair objection is that json_document is only fast because it can't change. json_editable_document has set(), insert(), erase() and push_back(). An edit never writes into the source text and never moves the parsed index:

• a new scalar is written over its own node, with new string and number text in an append-only arena;
• new arrays and objects are allocated in chunks of nodes that never move;
• a container whose elements change is marked as moved and gets a separate list of its entries: links to the values, which stay where they are. The list doubles when it grows, so push_back() is amortized constant.

Views stay valid across edits, because values never move. Reading an edited container costs one flag test per container and one test per element for a link. The link case is out of line, so that it compiles to a predicted branch rather than a conditional select that would delay the following loads. Whereas a read-only document carries none of this, an editable one that hasn't been edited reads within 2% of it on most rows and within 8% on the worst. And because untouched values keep their source text, dump() with number_format::source writes back a patched configuration file with its member order and the spelling of every number (19.90) intact, which json (sorted keys) and ordered_json (19.9) can't.

The limits are documented: set() of a new key searches the object's members, so building a large object key by key is quadratic, as with ordered_json, and an object that was edited loses its hash index and is searched linearly. How it compares times the editable document against the mutable trees of other libraries.

How it compares¶

"Which JSON library parses fastest?" has no answer, because "parsing" means different things in different libraries. They hold a parsed JSON text in four different ways:

The kinds of representation the libraries in this comparison offer. Most libraries offer more than one.

So a comparison of parse times across the rows of this table compares different amounts of work. Instead, we compare use cases: tasks a program actually has, each of which every library may solve in whatever way it does best. In nlohmann/json, json is the mutable tree and json_document the flat document, so we show the library twice: its best way with json_document and its best way without it.

Six use cases¶

Each starts from the file's text in a std::string (or a byte slice in Rust) and ends with everything the library built freed again:

• validate: is this text valid JSON? Nothing is kept, so this measures the parser alone, as in a server that checks a request before it passes it on.
• sum: visit every value in document order, convert every number, and add them up. This measures walking a whole document and converting all of its numbers.
• select: read a few fields of every record, such as the id, the user name, and the retweet count of every tweet, as a program does that needs only part of a response. A library that reads lazily can skip the rest.
• edit: change a value, remove a member, and add a member in every record, then write the result as compact JSON: the round trip of a program that patches a message or a configuration.
• minify: read the text and write it as compact JSON, without whitespace. This measures reading and writing together, without changes in between.
• write: write a document that was parsed beforehand as compact JSON; only the writing is timed.

Each use case runs on each of the four test files, and each library may solve it in every way it offers: in place, with a SAX handler, into structs, with a parser that is reused (measured, but not chosen). Before anything is timed, every way's result is checked: the same values, the same string bytes, the same sum of all numbers (to the last bit where they are added in the same order), and an output that reads back as the expected document. The README of the programs lists every way, the fields and edits per file, and the checks.

Where the input comes from¶

Starting from a string in memory is how most JSON benchmarks work: it measures the library, not the disk or the operating system. It is not neutral in every detail, though:

• The string has 64 bytes of spare capacity, so simdjson, which reads up to 64 bytes past the end, can use it without a copy. A program would load the file into such a buffer in the first place.
• yyjson, RapidJSON, and simd-json can parse in place, which changes the buffer. So every call gets a fresh copy of the string, and the copy counts toward their time: 11–14 µs for twitter.json on the Mac, and 30–46 µs on x86-64, where fresh memory costs page faults. yyjson's normal read makes the same copy internally, which is why both ways take about as long. A server that validates a request and then passes it on needs the copy. A program that reads a file into a buffer it owns can skip it, so we also timed the in-place ways with the copy made before the timer starts. On the Mac, yyjson then comes 3–5% closer, and on twitter.json it validates and selects 1–2% faster than json_document. On x86-64, it ties json_document in sum with Clang and takes 1.04× and 1.15× its time with GCC 16 and GCC 13.
• json_document reads the string in place and keeps pointing into it. A document that owns its text (parse_copy()) costs one copy more: 3–5% on the Mac.
• Reading from files and streams is not part of the comparison.

flowchart TB
    f[/"test file"/] -->|"read once, untimed"| s["std::string with 64 spare bytes"]
    s --> timed
    subgraph timed ["timed, every call"]
        direction LR
        c["copy<br>(in-place ways)"] --> p["parse"] --> u["use case"] --> d["free<br>everything"]
    end

What one timed call of a use case contains. For write, the document is parsed before the timer starts, so only the writing is timed. Every way's result is checked before anything is timed.

Checking less, on purpose¶

Some libraries can skip parts of the validation, and that is often why they are fast. A lazy reader such as simdjson's On-Demand API or Glaze's lazy_json only looks at what you ask for; to step over the rest, it matches brackets without looking inside, so reading a few fields of a large record costs little more than finding them. A minifier only needs to find whitespace, not understand the values. Skipping the UTF-8 check is RapidJSON's default and sonic-cpp's too. Each is a good choice for input that is known to be valid, for example text your own program wrote a moment ago. To tell these ways apart, every way is also fed four broken inputs: the file cut short by one byte, a stray comma, an invalid UTF-8 byte in a string, and garbage after the value. A way that rejects all four validates fully, and each cell shows the fastest such way. A way that checks or does less appears in the right half of the cell, with a letter that the caption explains.

A word on who set the rules: we wrote every library's benchmark code, chose the files and the rules, and tuned 3.13.0 on the same machines and files. No maintainer of the other libraries has reviewed their code. If you know a faster way for any of them, please open an issue, and we will rerun the comparison. To check ourselves, we also ran files and a conformance suite we never tuned on (below).

Time per library and use case on an Apple M1 Max: each library's fastest fully validating way, as the geometric mean over twitter.json, citm_catalog.json, canada.json and jeopardy.json, so that every file counts equally, divided by the fastest library's mean, which therefore shows 1.00. Less is better. Where a way that checks or does less is at least 3% faster, it is shown in the right half of the cell: "U" trusts UTF-8, "P" reads lazily, only removes whitespace or, in one Glaze cell on x86-64, doesn't check what follows the value, and "N" copies number text instead of formatting it. "n/a" means the library has no API for the use case. Hover over or tap a cell for the times per file and the way; the tables are in Appendix E. sonic-cpp's README lists only x86-64 Linux; we show it on the Mac anyway.

Results on the Mac¶

On the Apple M1 Max, among the ways that reject all four broken inputs, json_document has the lowest mean on the four files in all six use cases, and yyjson is the next library in every column; in validate, json::accept() comes in between (1.38× against yyjson's 1.41×). That includes editing, where the editable document is faster than yyjson's mutable document: the speed doesn't depend on being read-only.

Some caveats belong next to that:

• The lead is not the same on every file. On twitter.json, yyjson is within 4–7% in validate, sum, and select. Most of the lead comes from citm_catalog.json, the floats of canada.json (next point) and the size of jeopardy.json.
• Floats are converted only when read. That helps json_document in validate and select on canada.json, where most others convert 111,080 floats nobody looks at. Libraries that can skip the conversion use that mode there, but only Boost.JSON gains much: it validates canada.json in 47% of the time. sum reads every number, so it is the fair comparison for floats; there, json_document is still slightly ahead of yyjson.
• Checking less pays in select. If the input needn't be checked fully, simdjson's On-Demand API and Glaze reading into structs select in 0.76× and 0.81× json_document's time (Glaze is faster only on citm_catalog.json and canada.json), and in 0.55–0.61× on x86-64: the right halves of the select column.
• Indentation. Two of the four files are indented, which theme 2 helps. On minified copies of twitter.json and citm_catalog.json, yyjson comes closer in sum (1.11 → 1.01) and a little in validate (1.27 → 1.24), and json_document stays first.
• Number text can be copied. With number_format::source, json_document writes each number by copying its text instead of formatting it, and yyjson and RapidJSON can read numbers as text and write them back unchanged. The output keeps the source's spelling of each number, which is all a minifier needs. It makes writing canada.json with json_document 4.1× as fast and minifying it 1.9× as fast; the "N" halves of edit, minify, and write show it.
• Small documents. On a 2.5 KB message, the fastest libraries take about a microsecond on the Mac and 1.1–2 on x86-64. yyjson validates and sums it as fast as json_document on the Mac, and with GCC on x86-64, simdjson and yyjson sum it as fast.

Reading into structs¶

A view can be converted into your own types with the same from_json() as json, including the ones the NLOHMANN_DEFINE_TYPE_* macros generate. That is as fast as picking the fields by hand from the view; only for the many small records of jeopardy.json does it take 22% longer. On the Mac, Glaze, serde_json, and Boost.JSON's parse_into, reading into the same structs and checking everything, take 1.7×, 2.3×, and 3.1× as long (geometric mean over the four files). On a json value, reading into the structs takes about as long as json::parse() alone.

nlohmann/json without json_document¶

And nlohmann/json without json_document? json builds a tree of separate allocations, and it shows: for select, edit, and minify it takes several times as long as json_document, like Boost.JSON and RapidJSON. Where no tree is needed, the fused parser (One loop instead of two) puts it among the fastest: json::accept() validates the four files in 1.38× json_document's time on the Mac, ahead of every other library, and a SAX handler sums them in 1.34×, behind only yyjson and simdjson. Boost.JSON takes 1.8× and 1.5× as long as json there, and 1.2–1.4× and 1.1–1.3× on x86-64. dump() writes faster than Boost.JSON, RapidJSON and serde_json, and about as fast as Glaze.

Much of Boost.JSON's lead in building a tree comes from its allocator: here it builds the tree in a monotonic_resource, an arena that is freed in one go, which saves it 27–40%. With its default allocator, which allocates every string, array, and object on its own as json does, boost::json::parse() with precise numbers takes 13–17% longer than json::parse() on twitter.json, 19–20% longer on citm_catalog.json, and 2% less time on canada.json (on the Mac, freeing the tree included).

Results on x86-64¶

On x86-64, the same programs ran on a virtual server with Ubuntu 26.04 whose CPU presents itself as Haswell, pinned to one core. We built them with -march=haswell and three compilers: the newest GCC, 16.2 (the figure below), GCC 13.4, and Clang 23 (both in Appendix E). The compiler matters less than one might think. On the Mac, GCC 16 and Clang 23 put the same library first in every column as Apple clang, and on x86-64, the three compilers disagree only about sum.

The same comparison on the x86-64 server with GCC 16.2, with the same cells and halves as the figure above. Less is better; hover over or tap a cell for the times and the way. The figures for GCC 13 and Clang 23 are in Appendix E.

json_document has the lowest mean in five of the six columns with every compiler. The exception is sum, where simdjson is ahead with GCC, and json with a SAX handler with GCC 13 and Clang 23 (with Clang by 5%, all of it on jeopardy.json; below). This machine is noisy, though: between two runs, a library's time relative to json_document's moved by up to 7–8% (95th percentile; 2.4% on the Mac), so a lead of 5% is no result. In validate, the next library takes 1.8–2.0× as long, or 1.6–1.7× if every library may reuse its parser, because validating no longer builds an index (theme 8). The only way in between is json::accept() at 1.3–1.7×, which builds nothing either (One loop instead of two). Behind those means, json_document loses on single files:

• One pass over every value, with GCC. simdjson's On-Demand API sums all numbers of twitter.json, citm_catalog.json, and canada.json in 0.76–0.97× json_document's time with GCC 16 (0.79–0.92× with GCC 13). With Clang, json_document is ahead there, except on canada.json, where yyjson, which converts the floats during the parse, is faster than converting them on access, and on citm_catalog.json, where json's SAX handler and Boost.JSON's are 1–3% faster.
• Fresh large documents. For the 56 MB jeopardy.json, a fresh document needs a fresh 52 MB index, and on Linux the page faults of that much new memory are a large part of the time: copying the file into fresh memory alone takes about 50% as long as json_document's parse. Boost.JSON's SAX parser and sonic-rs, which build nothing, sum it in 0.64–0.71× the time, and json's SAX handler in 0.70–0.75×. Reusing the document cuts json_document's time by 41–43%, to 51 ms, less than sonic-rs's 60 ms. Validating needs no index: json_document::accept() takes 33–35 ms, json::accept() 39–40 ms, and no other library validates jeopardy.json faster, even reused (the next is simdjson's reused parser at 48–52 ms). On the Mac, a fresh document costs hardly more than a reused one.
• Writing floats. yyjson writes canada.json in 0.84–0.93× the time (and Glaze with GCC 13 and json's dump() with GCC 16, which start from doubles, in 0.98× and 0.99×), because json_document first converts most floats from their text. Writing the source text instead (number_format::source, the "N" half) avoids that.
• simdjson's DOM. It selects from citm_catalog.json 19% faster with GCC 16 and 7–13% faster with GCC 13 and Clang 23, and from twitter.json 2–4% faster with GCC. Boost.JSON, Glaze, sonic-rs, sonic-cpp, and yyjson are ahead in a few more cells, by 1–10%: summing citm_catalog.json (Boost.JSON with GCC), and selecting from or writing jeopardy.json. Most of that is within the noise.

The full tables, with each library's variants, are in Appendix E.

Files and checks we didn't choose¶

The four files are the ones 3.13.0 was tuned on, so we also ran validate, sum, minify, and write on the 19 other files of simdjson's jsonexamples. By geometric mean, json_document stays the fastest in validate, minify, and write on every compiler, and in sum with Clang and on the Mac; with GCC, simdjson sums in 0.87–0.88× its time. json::accept() validates them in 1.4–1.6× the time of json_document on x86-64, close to simdjson (1.44–1.46×), the fastest other library. File by file, for validate:

validate on the 19 files: json_document's fastest fully validating way against the fastest other library's, per file; below 1.0× it is the fastest. Less is better. The right column names the next library on x86-64 (GCC 16.2).

json_document validates 18 of the 19 files fastest with GCC 16, Clang 23, and on the Mac: simdjson ties on google_maps_api_response with GCC 16, Boost.JSON on mesh.pretty with Clang 23, and simdjson validates apache_builds 7% faster on the Mac. With GCC 13 it is 17: simdjson validates google_maps_api_response 6% faster, and Boost.JSON mesh.pretty 10% faster (1% in the first of our two runs on November 7). Before validating without an index (theme 8) and skipping fraction digits 16 bytes at a time (theme 2), Boost.JSON validated the number-heavy files up to 1.5× as fast on x86-64.

The four probes are broken inputs that tell full validation from less (Appendix E lists them). As a further check, every validate way also ran the 318 files of JSONTestSuite. Appendix E lists the exceptions, one of them ours. The suite also found that Glaze's generic reader accepts garbage after the value by default, so the figures use its strict option.

We did not measure memory use, a CPU with AVX-512, MSVC, writing from structs, or streaming input.

What didn't work¶

The pull requests record the ideas that measured worse, which is as useful as the ones that made it:

Ideas that were measured and dropped. Appendix H links the pull requests that tried them.

The price¶

None of this is free at compile time. json.hpp grew from 25,526 to 48,025 lines (json_view.hpp is a separate header of 11,909 lines that you only pay for if you include it). To see what that means for a program, we compiled two small ones with both versions:

• count_numbers reads twitter.json and adds up all of its numbers, like the sum use case in How it compares;
• server_config reads a configuration file into a struct with to_json() and from_json(), changes it, and writes it back with dump(2).

Each is one translation unit. We timed it with the 3.12.0 and the 3.13.0 header, and then with 3.13.0 as a precompiled header and as a C++20 module, in a debug build (-O0) and an optimized one (-O2):

How much longer or shorter each program takes to compile than with 3.12.0, less is better: Apple clang 21 and GCC 16.2, CPU time of the compiler, median of 5 interleaved runs, C++20, Apple M1 Max. The one-off cost of building the precompiled header or the module is not included. Hover over a bar for the time in milliseconds; Appendix G has all times, the programs, and the commands.

3.13.0 takes 36–68% longer to compile these programs. A good part of that is reading the header, and a precompiled header or a module does that once instead of in every translation unit:

• In debug builds, which are what you wait for while you edit, a precompiled 3.13.0 compiles faster than 3.12.0 with Apple clang, and within 12–15% of it with GCC.
• In optimized builds, less of the increase goes away, because the optimizer still works through all the code a program instantiates. A precompiled header or a module wins back 41% to all of it.
• The one-off cost is 0.9 s (Apple clang) or 2.3 s (GCC) to build the precompiled header, and 1.4 s or 2.3 s to build the module, once per set of compiler flags. A precompiled header pays for itself after two to four translation units, a module after three to six.
• Modules have caveats. Apple clang needs -fcxx-modules, module support still depends heavily on the compiler version, and macros don't cross a module boundary: with import nlohmann.json;, the NLOHMANN_DEFINE_TYPE_* macros aren't available, which is why server_config writes to_json() and from_json() by hand.

The compile-time page measures these and the other options, such as including <nlohmann/json_fwd.hpp> in headers that only name json, and JSON_NO_AUTOMATIC_UDLS. We did not measure binary size or peak memory.

Summing up¶

Hardly any of the techniques in this post are new. What we think made the difference is their combination, under the conditions of a header-only C++11 library whose API and output had to stay the same.

Thanks¶

Most of what is in this post was invented by other people.

Code we adapted. The library contains code adapted from these four projects. We use it under the MIT License, and the source files that contain it carry the authors' copyright lines and say what was taken; the README lists them too.

• fast_float (Apache-2.0, MIT, or BSL-1.0) by Daniel Lemire and contributors: the Eisel-Lemire algorithm, its table of powers of five, its digit comparison for long numbers, and its parsing of eight digits at a time (N. Mushtak and D. Lemire, "Fast number parsing without fallback", 2023). W. D. Clinger for the fast path, M. Eisel for the algorithm.
• Żmij (MIT) by Victor Zverovich, with the digit conversion of Xiang JunBo, the NEON sequence and compressed powers of ten of Dougall Johnson, and the SSE support of Tobias Schlüter: ported to C++11.
• simdjson (Apache-2.0 or MIT) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors: the "lookup4" UTF-8 check and its tables (J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), in json.hpp's lexer and in json_view.hpp.
• yyjson (MIT) by YaoYuan: table-driven \u decoding and the unrolled checks at fixed offsets in json_view.hpp's parser.

Ideas we took. From these we took the idea, not their code, in the order they appear above:

• serde_json (MIT or Apache-2.0): deriving positions from the offset only on error.
• Sean Eron Anderson's Bit Twiddling Hacks (snippets in the public domain): the zero-byte test behind every word-at-a-time scan.
• Glaze (MIT): skipping indentation in 8-byte steps (skip_matching_ws).
• SQLite (public domain): writing short strings with a table loop, as its jsonAppendString() does.
• orjson (Apache-2.0 or MIT; its escape table MPL-2.0) and sonic-rs (Apache-2.0), from which orjson adapted it: escape sequences as fixed-size table rows (QUOTE_TAB).
• simdjson's tape and yyjson's 16-byte immutable document: a flat index in document order instead of a tree.
• Boost.JSON (BSL-1.0): switching to a hash table for large objects.
• RapidJSON (MIT): recursion as the default, iterative parsing as an option.

I thank the people who sent or shaped pull requests:

• @gregmarr reviewed most of them and suggested the sized-sentinel fast path, the peek design for exact stream positions, one SIMD switch for the whole library, and the vector check of json_view's strings on edit;
• @veselink1 and @michaelsam94 wrote the allocator and exception-safety fixes for the destructor that 3.13.0 took over; the non-recursive destructor itself goes back to a pull request by @nickaein in 2019;
• @commit-msuyog gave the 128-bit multiplication behind Eisel-Lemire MSVC intrinsics;
• @ayush-singh-0601 started the fix for deep values in the binary writers;
• @avionicharshit-byte made diff() linear for shrinking arrays;
• @mvanslobbe sped up the parser's separator check;
• @alexprabhat99 removed the temporary values behind CBOR and MessagePack object keys.

twitter.json, citm_catalog.json and canada.json come from Milo Yip's nativejson-benchmark (MIT).

Appendix¶

A. Test files¶

The test files. The first three are from Milo Yip's nativejson-benchmark. The counts are of the parsed values; the share of whitespace, string, and non-ASCII bytes is of the file as distributed. Several changes target the indentation of citm_catalog.json: in the timeline benchmark, a minified copy of it parses 1.9× as fast as with 3.12.0, against 3.4× for the file as distributed.

B. 3.12.0 against 3.13.0¶

The rows of the first chart:

Time per call with 3.12.0 and 3.13.0, Google Benchmark suite on an Apple M1 Max, median of 12 runs; the speedup is the old time divided by the new one. The Intel column is the speedup on the Intel MacBook Pro of Appendix G. The parse rows do not include destroying the result (the suite pauses the timer for that).

C. The timeline¶

The other four workloads we followed:

• accept() of twitter.json: the lexer alone, since accept() only validates and builds nothing;
• parse() of twitter.json from an std::ifstream: reading from a stream instead of a string;
• dump() of canada.json: writing floating-point numbers;
• to_cbor() of twitter.json: writing a binary format.

Speedup over 3.12.0 after each commit that changed the headers, for the four workloads not shown in the timeline section. More is better; the badges are theme numbers.

The parse panels include destroying the result: the benchmark returns json::parse(text).size(), so the changes to the destructor (#5762, #5835) show up there too, as steps of up to 6% in either direction. When accept() of twitter.json got 2.5% slower over the last 20 commits, we timed the two ends against each other in one binary with six code layouts: they were equal within 0.2%.

The labeled steps of the timeline: the change in time at the first commit of each pull request, against the commit before it; for #6031, which was not merged yet, the step to 3.13.0. #5592 is by @mvanslobbe.

D. Algorithmic fixes¶