Niels Lohmann
71baba09fe
For "last read" in parse errors, the lexer keeps the text of the token it reads. The token start moves only at strings and numbers, so everything after a string or number belongs to it until the next one starts. Input that cannot be read again copies that text while parsing (the istream and FILE* adapters in their carry buffer, the lexer in token_string for iterators), so valid input of 1 followed by 100,000,000 newlines peaked at about 225 MB, and a long string was kept a second time. Replace the carry buffer and token_string by token_text_buffer, which keeps the first 4096 and at least the last 4112 characters of a token and drops the characters in between where it would otherwise grow, so appending stays as cheap as before. get_token_string() shows a token of more than 8192 characters as its first and last 4096 characters around "...", the same for every kind of input; contiguous input copies only these parts. The example input now peaks at 1.8 MB. Instruction counts of parsing from std::string, std::istream and FILE* are unchanged (clang 21 and GCC 15); parsing from std::list<char> needs 1 to 17 % fewer. Fixes #6120. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Documentation
Generate documentation
Note on documentation: The source files contain links to the online documentation at https://json.nlohmann.me.
This URL provides the most recent documentation and also applies to previous versions. Documentation for deprecated
functions is not removed; instead, it is marked as deprecated.
If you want to view the documentation for a specific tag or commit hash, you can generate it locally as follows (example
using tag v3.12.0):
git clone https://github.com/nlohmann/json.git
cd json
git checkout v3.12.0
make install_venv serve -C docs/mkdocs
Open http://127.0.0.1:8000/ in your browser. Replace any URL in the source code that points to
https://json.nlohmann.me with http://127.0.0.1:8000 to view the documentation for the selected tag or commit hash.