diff --git a/docs/mkdocs/docs/api/basic_json/from_bjdata.md b/docs/mkdocs/docs/api/basic_json/from_bjdata.md new file mode 100644 index 000000000..2f8f1ca04 --- /dev/null +++ b/docs/mkdocs/docs/api/basic_json/from_bjdata.md @@ -0,0 +1,3 @@ +# nlohmann::basic_json::from_bjdata + +TODO diff --git a/docs/mkdocs/docs/api/basic_json/index.md b/docs/mkdocs/docs/api/basic_json/index.md index 68ac063ff..bc4dba153 100644 --- a/docs/mkdocs/docs/api/basic_json/index.md +++ b/docs/mkdocs/docs/api/basic_json/index.md @@ -268,10 +268,12 @@ Access to the JSON value ### Binary formats +- [**from_bjdata**](from_bjdata.md) (_static_) - create a JSON value from an input in BJData format - [**from_bson**](from_bson.md) (_static_) - create a JSON value from an input in BSON format - [**from_cbor**](from_cbor.md) (_static_) - create a JSON value from an input in CBOR format - [**from_msgpack**](from_msgpack.md) (_static_) - create a JSON value from an input in MessagePack format - [**from_ubjson**](from_ubjson.md) (_static_) - create a JSON value from an input in UBJSON format +- [**to_bjdata**](to_bjdata.md) (_static_) - create a BJData serialization of a given JSON value - [**to_bson**](to_bson.md) (_static_) - create a BSON serialization of a given JSON value - [**to_cbor**](to_cbor.md) (_static_) - create a CBOR serialization of a given JSON value - [**to_msgpack**](to_msgpack.md) (_static_) - create a MessagePack serialization of a given JSON value diff --git a/docs/mkdocs/docs/api/basic_json/to_bjdata.md b/docs/mkdocs/docs/api/basic_json/to_bjdata.md new file mode 100644 index 000000000..2cace544d --- /dev/null +++ b/docs/mkdocs/docs/api/basic_json/to_bjdata.md @@ -0,0 +1,3 @@ +# nlohmann::basic_json::to_bjdata + +TODO diff --git a/docs/mkdocs/docs/features/binary_formats/bjdata.md b/docs/mkdocs/docs/features/binary_formats/bjdata.md new file mode 100644 index 000000000..34d3d4dd8 --- /dev/null +++ b/docs/mkdocs/docs/features/binary_formats/bjdata.md @@ -0,0 +1,20 @@ +# BJData + +The Binary JData (BJData) Specification defines an efficient serialization protocol for unambiguously storing complex +and strongly-typed binary data found in diverse applications. The BJData specification is the binary counterpart to the +JSON format, both of which are used to serialize complex data structures supported by the +[JData specification](https://openjdata.org). The BJData spec is derived and extended from the +[Universal Binary JSON(UBJSON)](https://ubjson.org) specification (Draft 12). It adds supports for N-dimensional packed +arrays and extended binary data types. + +!!! abstract "References" + + - [BJData Specification](https://github.com/NeuroJSON/bjdata/blob/Draft_2/Binary_JData_Specification.md) + +## Serialization + +TODO + +## Deserialization + +TODO diff --git a/docs/mkdocs/docs/features/binary_formats/index.md b/docs/mkdocs/docs/features/binary_formats/index.md index 3a969a5dc..736130467 100644 --- a/docs/mkdocs/docs/features/binary_formats/index.md +++ b/docs/mkdocs/docs/features/binary_formats/index.md @@ -1,7 +1,9 @@ # Binary Formats -Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports +Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over +a network. Hence, the library supports +- [BJData](bjdata.md) (Binary JData), - [BSON](bson.md) (Binary JSON), - [CBOR](cbor.md) (Concise Binary Object Representation), - [MessagePack](messagepack.md), and @@ -15,6 +17,7 @@ to efficiently encode JSON values to byte vectors and to decode such vectors. | Format | Serialization | Deserialization | |-------------|-----------------------------------------------|----------------------------------------------| +| BJData | complete | complete | | BSON | incomplete: top-level value must be an object | incomplete, but all JSON types are supported | | CBOR | complete | incomplete, but all JSON types are supported | | MessagePack | complete | complete | @@ -24,6 +27,7 @@ to efficiently encode JSON values to byte vectors and to decode such vectors. | Format | Binary values | Binary subtypes | |-------------|---------------|-----------------| +| BJData | not supported | not supported | | BSON | supported | supported | | CBOR | supported | supported | | MessagePack | supported | supported | @@ -35,11 +39,12 @@ See [binary values](../binary_values.md) for more information. | Format | canada.json | twitter.json | citm_catalog.json | jeopardy.json | |--------------------|-------------|--------------|-------------------|---------------| -| BSON | 85,8 % | 95,2 % | 95,8 % | 106,7 % | -| CBOR | 50,5 % | 86,3 % | 68,4 % | 88,0 % | -| MessagePack | 50,6 % | 86,0 % | 68,5 % | 87,9 % | -| UBJSON | 53,2 % | 91,3 % | 78,2 % | 96,6 % | -| UBJSON (size) | 58,6 % | 92,3 % | 86,8 % | 97,4 % | -| UBJSON (size+type) | 55,9 % | 92,3 % | 85,0 % | 95,0 % | +| BJData | 42.8 % | 77.3 % | 76.3 % | 98.8 % | +| BSON | 85.8 % | 95.2 % | 95,8 % | 106,7 % | +| CBOR | 50.5 % | 86.3 % | 68,4 % | 88,0 % | +| MessagePack | 50.6 % | 86.0 % | 68,5 % | 87,9 % | +| UBJSON | 53.2 % | 91.3 % | 78,2 % | 96,6 % | +| UBJSON (size) | 58.6 % | 92.3 % | 86,8 % | 97,4 % | +| UBJSON (size+type) | 55.9 % | 92.3 % | 85,0 % | 95,0 % | Sizes compared to minified JSON value. diff --git a/docs/mkdocs/docs/features/binary_values.md b/docs/mkdocs/docs/features/binary_values.md index c58834c05..5ad6433cf 100644 --- a/docs/mkdocs/docs/features/binary_values.md +++ b/docs/mkdocs/docs/features/binary_values.md @@ -1,8 +1,12 @@ # Binary Values -The library implements several [binary formats](binary_formats/index.md) that encode JSON in an efficient way. Most of these formats support binary values; that is, values that have semantics define outside the library and only define a sequence of bytes to be stored. +The library implements several [binary formats](binary_formats/index.md) that encode JSON in an efficient way. Most of +these formats support binary values; that is, values that have semantics define outside the library and only define a +sequence of bytes to be stored. -JSON itself does not have a binary value. As such, binary values are an extension that this library implements to store values received by a binary format. Binary values are never created by the JSON parser, and are only part of a serialized JSON text if they have been created manually or via a binary format. +JSON itself does not have a binary value. As such, binary values are an extension that this library implements to store +values received by a binary format. Binary values are never created by the JSON parser, and are only part of a +serialized JSON text if they have been created manually or via a binary format. ## API for binary values @@ -19,7 +23,9 @@ class json::binary_t { "std::vector" <|-- json::binary_t ``` -By default, binary values are stored as `std::vector`. This type can be changed by providing a template parameter to the `basic_json` type. To store binary subtypes, the storage type is extended and exposed as `json::binary_t`: +By default, binary values are stored as `std::vector`. This type can be changed by providing a template +parameter to the `basic_json` type. To store binary subtypes, the storage type is extended and exposed as +`json::binary_t`: ```cpp auto binary = json::binary_t({0xCA, 0xFE, 0xBA, 0xBE}); @@ -87,7 +93,9 @@ Binary values are serialized differently according to the formats. ### JSON -JSON does not have a binary type, and this library does not introduce a new type as this would break conformance. Instead, binary values are serialized as an object with two keys: `bytes` holds an array of integers, and `subtype` is an integer or `null`. +JSON does not have a binary type, and this library does not introduce a new type as this would break conformance. +Instead, binary values are serialized as an object with two keys: `bytes` holds an array of integers, and `subtype` +is an integer or `null`. ??? example @@ -115,11 +123,72 @@ JSON does not have a binary type, and this library does not introduce a new type !!! warning "No roundtrip for binary values" - The JSON parser will not parse the objects generated by binary values back to binary values. This is by design to remain standards compliant. Serializing binary values to JSON is only implemented for debugging purposes. + The JSON parser will not parse the objects generated by binary values back to binary values. This is by design to + remain standards compliant. Serializing binary values to JSON is only implemented for debugging purposes. + +### BJData + +[BJData](binary_formats/bjdata.md) neither supports binary values nor subtypes, and proposes to serialize binary values +as array of uint8 values. This translation is implemented by the library. + +??? example + + Code: + + ```cpp + // create a binary value of subtype 42 (will be ignored in BJData) + json j; + j["binary"] = json::binary({0xCA, 0xFE, 0xBA, 0xBE}, 42); + + // convert to BJData + auto v = json::to_bjdata(j); + ``` + + `v` is a `std::vector` with the following 20 elements: + + ```c + 0x7B // '{' + 0x69 0x06 // i 6 (length of the key) + 0x62 0x69 0x6E 0x61 0x72 0x79 // "binary" + 0x5B // '[' + 0x55 0xCA 0x55 0xFE 0x55 0xBA 0x55 0xBE // content (each byte prefixed with 'U') + 0x5D // ']' + 0x7D // '}' + ``` + + The following code uses the type and size optimization for UBJSON: + + ```cpp + // convert to UBJSON using the size and type optimization + auto v = json::to_bjdata(j, true, true); + ``` + + The resulting vector has 22 elements; the optimization is not effective for examples with few values: + + ```c + 0x7B // '{' + 0x23 0x69 0x01 // '#' 'i' type of the array elements: unsigned integers + 0x69 0x06 // i 6 (length of the key) + 0x62 0x69 0x6E 0x61 0x72 0x79 // "binary" + 0x5B // '[' array + 0x24 0x55 // '$' 'U' type of the array elements: unsigned integers + 0x23 0x69 0x04 // '#' i 4 number of array elements + 0xCA 0xFE 0xBA 0xBE // content + ``` + + Note that subtype (42) is **not** serialized and that UBJSON has **no binary type**, and deserializing `v` would + yield the following value: + + ```json + { + "binary": [202, 254, 186, 190] + } + ``` ### BSON -[BSON](binary_formats/bson.md) supports binary values and subtypes. If a subtype is given, it is used and added as unsigned 8-bit integer. If no subtype is given, the generic binary subtype 0x00 is used. +[BSON](binary_formats/bson.md) supports binary values and subtypes. If a subtype is given, it is used and added as +unsigned 8-bit integer. If no subtype is given, the generic binary subtype 0x00 is used. ??? example @@ -159,7 +228,9 @@ JSON does not have a binary type, and this library does not introduce a new type ### CBOR -[CBOR](binary_formats/cbor.md) supports binary values, but no subtypes. Subtypes will be serialized as tags. Any binary value will be serialized as byte strings. The library will choose the smallest representation using the length of the byte array. +[CBOR](binary_formats/cbor.md) supports binary values, but no subtypes. Subtypes will be serialized as tags. Any binary +value will be serialized as byte strings. The library will choose the smallest representation using the length of the +byte array. ??? example @@ -185,7 +256,8 @@ JSON does not have a binary type, and this library does not introduce a new type 0xCA 0xFE 0xBA 0xBE // content ``` - Note that the subtype is serialized as tag. However, parsing tagged values yield a parse error unless `json::cbor_tag_handler_t::ignore` or `json::cbor_tag_handler_t::store` is passed to `json::from_cbor`. + Note that the subtype is serialized as tag. However, parsing tagged values yield a parse error unless + `json::cbor_tag_handler_t::ignore` or `json::cbor_tag_handler_t::store` is passed to `json::from_cbor`. ```json { @@ -198,7 +270,9 @@ JSON does not have a binary type, and this library does not introduce a new type ### MessagePack -[MessagePack](binary_formats/messagepack.md) supports binary values and subtypes. If a subtype is given, the ext family is used. The library will choose the smallest representation among fixext1, fixext2, fixext4, fixext8, ext8, ext16, and ext32. The subtype is then added as signed 8-bit integer. +[MessagePack](binary_formats/messagepack.md) supports binary values and subtypes. If a subtype is given, the ext family +is used. The library will choose the smallest representation among fixext1, fixext2, fixext4, fixext8, ext8, ext16, and +ext32. The subtype is then added as signed 8-bit integer. If no subtype is given, the bin family (bin8, bin16, bin32) is used. @@ -239,7 +313,8 @@ If no subtype is given, the bin family (bin8, bin16, bin32) is used. ### UBJSON -[UBJSON](binary_formats/ubjson.md) neither supports binary values nor subtypes, and proposes to serialize binary values as array of uint8 values. This translation is implemented by the library. +[UBJSON](binary_formats/ubjson.md) neither supports binary values nor subtypes, and proposes to serialize binary values +as array of uint8 values. This translation is implemented by the library. ??? example @@ -251,7 +326,7 @@ If no subtype is given, the bin family (bin8, bin16, bin32) is used. j["binary"] = json::binary({0xCA, 0xFE, 0xBA, 0xBE}, 42); // convert to UBJSON - auto v = json::to_msgpack(j); + auto v = json::to_ubjson(j); ``` `v` is a `std::vector` with the following 20 elements: @@ -287,7 +362,8 @@ If no subtype is given, the bin family (bin8, bin16, bin32) is used. 0xCA 0xFE 0xBA 0xBE // content ``` - Note that subtype (42) is **not** serialized and that UBJSON has **no binary type**, and deserializing `v` would yield the following value: + Note that subtype (42) is **not** serialized and that UBJSON has **no binary type**, and deserializing `v` would + yield the following value: ```json { diff --git a/docs/mkdocs/mkdocs.yml b/docs/mkdocs/mkdocs.yml index 38a8cadfa..dd4da8a0c 100644 --- a/docs/mkdocs/mkdocs.yml +++ b/docs/mkdocs/mkdocs.yml @@ -42,6 +42,7 @@ nav: - features/arbitrary_types.md - Binary Formats: - features/binary_formats/index.md + - features/binary_formats/bjdata.md - features/binary_formats/bson.md - features/binary_formats/cbor.md - features/binary_formats/messagepack.md @@ -109,6 +110,7 @@ nav: - 'exception': api/basic_json/exception.md - 'find': api/basic_json/find.md - 'flatten': api/basic_json/flatten.md + - 'from_bjdata': api/basic_json/from_bjdata.md - 'from_bson': api/basic_json/from_bson.md - 'from_cbor': api/basic_json/from_cbor.md - 'from_msgpack': api/basic_json/from_msgpack.md @@ -178,6 +180,7 @@ nav: - 'string_t': api/basic_json/string_t.md - 'swap': api/basic_json/swap.md - 'std::swap<basic_json>': api/basic_json/std_swap.md + - 'to_bjdata': api/basic_json/to_bjdata.md - 'to_bson': api/basic_json/to_bson.md - 'to_cbor': api/basic_json/to_cbor.md - 'to_msgpack': api/basic_json/to_msgpack.md