Read BJData ND-arrays of float16, restrict ND-array char to 0..127, correct BJData docs (#5931)
* Read ND-arrays of half floats and enforce the char limit in ND-arrays BJData N-dimensional arrays with element type 'h' (float16) failed to read with a misleading parse_error.112, because the reader's dtype name table lacked 'h'. It now maps to "half", the JData name. The writer still does not pack "half"; such an object is written as a plain object and reads back equal. BJData limits 'C' (char) to 0..127 in Draft 2 and Draft 3, and the 1-D reader enforces this, but ND-arrays with _ArrayType_ "char" wrote values up to 255 as 'C' elements and read them back without a check. The writer now only packs values in 0..127 (others fall back to a plain object that round-trips), and the reader rejects 'C' elements above 127 with parse_error.113 like the 1-D reader. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: correct and complete the BJData documentation Verify the BJData chapter against the library and the Draft 2/3/4 specifications and fix what did not match: - bjdata.md: correct the list of unused markers (N, not Z), the ND-array example (3x2, not 2x3), the conditions under which an object is packed as an ND-array (exactly three keys, row vectors, float-only elements for single/double, char 0..127, byte only in Draft 3), the binary table row and notes (which flag/draft combinations write [$B), the "$" type restriction (Draft 3, Z is excluded too), the H marker in the deserialization table, and the claim of a complete deserialization mapping. Document the supported drafts, that Draft 4 (SoA containers, extension type E) and column-major ND-arrays are rejected, ND-arrays of half floats, duplicate object keys, and link the Draft 3 specifications instead of master. - Add an API page for basic_json::bjdata_version_t with an example, and link it from the nav, the API index and to_bjdata. - Update binary_formats/index.md and binary_values.md, which still claimed that BJData has no binary type, and fix a wrong byte comment there. - Document parse_error.115 and the out_of_range.408 cases for from_bjdata, the precise other_error.502 condition for to_bjdata, and reword the note on truncated unsigned integers in exceptions.md (only ND-array elements were truncated before 3.13.0). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Regenerate single_include and apply astyle Result of make amalgamate (including make pretty) for the preceding changes. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review of the BJData documentation The $ type restriction exists since Draft 2 (Draft 3 added B), $ requires a count, ND char above 127 is rejected on read, and the new API page is listed in alphabetical order in the nav. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: format tables, use the Oxford comma Reformat the tables changed by this PR (BJData serialization and deserialization mappings, completeness and binary-value tables) so all rows are aligned, and add the Oxford comma to lists of three or more items in the added text. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
1 parent
dc768a7d5d
commit
60107eda3f
15 files changed
+380
-115
No files matched your search
@@ -3277,6 +3277,8 @@ class binary_reader
|
||||
// get_ubjson_size_value() (the type marker is known before the dimension vector
|
||||
// that determines size_and_type.first is read, so it is emitted first there to
|
||||
// match the documented _ArrayType_, _ArraySize_, _ArrayData_ key order)
|
||||
// 'C' elements are read as unsigned bytes, but BJData limits a char to 0..127
|
||||
const bool nd_is_char = size_and_type.second == 'C';
|
||||
if (size_and_type.second == 'C' || size_and_type.second == 'B')
|
||||
{
|
||||
size_and_type.second = 'U';
|
||||
@@ -3290,6 +3292,24 @@ class binary_reader
|
||||
|
||||
for (std::size_t i = 0; i < size_and_type.first; ++i)
|
||||
{
|
||||
if (nd_is_char)
|
||||
{
|
||||
get();
|
||||
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("char")))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if (JSON_HEDLEY_UNLIKELY(current > 127))
|
||||
{
|
||||
return last_byte_error(exception_id::invalid_string_or_size, "byte after 'C' must be in range 0x00..0x7F", "char", "");
|
||||
}
|
||||
if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(current))))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_value(size_and_type.second)))
|
||||
{
|
||||
return false;
|
||||
@@ -4655,6 +4675,8 @@ class binary_reader
|
||||
return "uint8";
|
||||
case 'd':
|
||||
return "single";
|
||||
case 'h':
|
||||
return "half";
|
||||
case 'i':
|
||||
return "int8";
|
||||
case 'l':
|
||||
|
||||
Reference in new issue
Block a user