What is bao?
The basic format Assign Onward uses to communicate
*Bao is also well suited for compact data storage, but the specifics of data storage and retrieval implementations are outside the Ⓐ protocol scope, at the discretion of the application architects. messages is called bao. It is a strongly typed binary encoding which can be easily translated to and from json,
*Although bao can be translated to json, and back again, not all json can be represented in bao - due first to the strong type definition, and also to dictionary references required for keys and key codes. and to dot representation.
What is in bao?
The actual data stored in bao is in one of the defined types:
*While some types are redundant, they are specified separately to ease implementation (for instance: int64 is much easier to access than arbitrary length integer) as well as preventing common problems such as confusion between arbitrary binary data and strings which are intended to be human readable. Types like floating point are specifically excluded to discourage their use in protocols which are meant to be easily 100% reproducable on varying hardware and software.
Like json, bao wraps data in objects or arrays which may be nested. In bao data items, objects and arrays are always paired with keys.
- signed int64
- arbitrary length integer
- ratio of two arbitrary length integers
- key code
- UTF-8 string
- octet stream
- bao object
- an array of any of the above types
Bao keys are defined in a dictionary. Each defined key has a unique human readable name which is used in json translations of bao objects and code references to the key. The key also has a unique integer representation, and that integer is encoded to a compact binary representation in bao byte streams. The lower bits of bao keys specify the type of the data which follows, as well as whether the data is a single item, or an array of items all of the same type as specified in the key.
The meaning of keys is often context specific, depending on what wrappers they are contained in. The first key, on the outermost wrapper of a bao message, defines the type of the message and what protocol rules apply to everything within the message's wrappers.
A bao object contains zero or more key value pair elements. Each element in a bao object has a key which is unique within the object, followed by either an "atomic" data item (listed above), an array of data items or objects, or a single object - the key defines both the type of data in the element and how that data is to be handled by the operating protocol.
Bao arrays are a single key followed by an integer size zero or larger which specifies how many data elements (of the type specified by the key) follow. Bao arrays may contain objects,
*Any object may contain zero or more key value pairs where the value is an array, but... if the desire is to implement a multi-dimensional structure like a table or volume description, I would rather suggest that the data be laid into a one-dimensional array type and include the dimensions of the table / volume / whatever in a wrapper object which includes the array. but may not directly contain arrays.
*In a sense, bao key codes specify an array of dimension zero or one. With the addition of a single bit to the array dimension encoding, two and three dimensional arrays could be specified in the array, enabling more json like arrays of arrays where each element in the outer array could contain an inner array of a different length. While academically interesting, the complexity of implementation seems a poor tradeoff for the value of having such encoding options available.
Bao encoding is a fairly straightforward system where the outermost key is followed by the binary encode of the value belonging to that key. Object values start with a coded integer telling how many key-value elements are in the following object, then each key + value of the key in the object follows. Arrays similarly start with a coded integer telling how many value elements follow and then each value follows. Remember that values may themselves be objects which then nest further... The atomic types are encoded as follows:
When bao objects are stored, the elements are sorted by key value. Order of bao arrays is preserved, not sorted. This means that a the bao binary representation of an object is specific, and will always be the same for a valid representation of the object. Json representations of bao are often in a very different order, sorted by the key name instead of its code value, but when they are translated back to bao, the original bao order is restored due to sorting by key value instead of name.
- signed int64 - 8 bytes lsb first.
- arbitrary length integer - bcd digits, terminated by hex E for a positive and F for a negative number. If the terminator is the first nybble of a byte, it is followed by hex D.
- ratio of two arbitrary length integers - two arbitrary length integers as above, numerator first.
- key code - a string of one to nine bytes, most significant bits first, where the seven lsb of each byte are part of the value, and the first byte with a 0 in the most significant bit is the final byte of the string.
- octet stream - an encoded length followed by that number of bytes.
- UTF-8 string - exactly as the octet stream, but the contents are required to be valid UTF-8.
12 October 2021
Copyright (c) 2021 Assign Onward
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE