ASAsense Binary Table (ABT) format: verschil tussen versies
Uit ASAsense Documentation
(Nieuwe pagina aangemaakt met 'floatThe ASAsense binary table format is an alternative for the CSV format, but for binary data. The structure is as follows: {| class="wikitable" |+ !name !length (bytes) !type !allowed values !explanation |- |file_type |1 |u8 |1 |The internal number of the file type and version |- |n_cols |1 |u8 |1-255 |the amount of headers |- |col_widths |1 x n_cols |u8[n_cols] |1-255 per column |n_cols u8 integers describing the byte-width of the columns |- |metadata |unt...') |
|||
| (16 tussenliggende versies door dezelfde gebruiker niet weergegeven) | |||
| Regel 1: | Regel 1: | ||
The ASAsense binary table format is an alternative for the CSV format, but for binary data with fixed row lengths. The structure is as follows: | |||
{| class="wikitable" | {| class="wikitable" | ||
|+ | |+ | ||
| Regel 15: | Regel 15: | ||
|- | |- | ||
|n_cols | |n_cols | ||
| | |4 | ||
| | |u32 | ||
|1- | |1-u32::max | ||
|the amount of | |the amount of columns | ||
|- | |- | ||
|col_widths | |col_widths | ||
| | |4 x n_cols | ||
| | |u32[n_cols] | ||
|1- | |1-u32::max per column | ||
|n_cols | |n_cols u32 integers describing the byte-width of the columns | ||
|- | |||
|metadata_length | |||
|4 | |||
|u32 | |||
|0-u32::max | |||
|the length of the metadata field (can be 0 if it needs to be skipped) | |||
|- | |- | ||
|metadata | |metadata | ||
| | |metadata_length | ||
|char[] | |char[] | ||
| | |string of fixed length (so no '\0' character to mark the end of the string) | ||
|JSON-encoded metadata | |JSON-encoded metadata | ||
|- | |- | ||
| Regel 60: | Regel 66: | ||
|a list of column definition objects | |a list of column definition objects | ||
|an list of objects describing the columns in more details (should have n_cols entries) | |an list of objects describing the columns in more details (should have n_cols entries) | ||
|- | |||
|extra | |||
|false | |||
|JSON object | |||
|a JSON object that can freely be specified | |||
|} | |} | ||
A "column definition object" should have the following format | A "column definition object" should have the following format | ||
| Regel 100: | Regel 111: | ||
|float | |float | ||
|interpret as floating point data (only possible for 4 or 8 byte fields) | |interpret as floating point data (only possible for 4 or 8 byte fields) | ||
|- | |||
|bool | |||
|boolean value (0 = false, anything else = true) | |||
|- | |- | ||
|numtype/div | |numtype/div | ||
|"div" should be replaced by any floating point number. The data is then first interpreted as "numtype" (int, uint or float) and then divided by "div". This is mainly useful for storing decimal numbers in integer fields. | |"div" should be replaced by any floating point number. The data is then first interpreted as "numtype" (int, uint or float) and then divided by "div". This is mainly useful for storing decimal numbers in integer fields. | ||
|- | |- | ||
|unix_timestamp | |unix_timestamp(numtype/div) | ||
|Interprets the data as | |Interprets the data as specified within brackets (so any numtype, with optional divider), but takes into account that it is a unix_timestamp (offset seconds since the Unix Epoch on January 1st, 1970 at UTC) | ||
|} | |||
Other, unknown datatype identifiers, should be treated as opaque binary data (bytes) by the reader. | |||
== Common Extra Metadata fields == | |||
In many ABT files ASAsense uses, the following extra matadata fields will be found | |||
{| class="wikitable" | |||
|+ | |||
!Field name | |||
!Field type | |||
!Default value | |||
!Description | |||
|- | |- | ||
|sorted_by_timestamp | |||
|bool | |bool | ||
| | |true | ||
|Indicates whether the data in the ABT file is sorted by timestamp (column 1) | |||
|- | |||
|frame_width_s | |||
|float | |||
|NaN | |||
|Used to indicate the frame width of one sample (for example the sampling period or spectrogram frame width) | |||
|- | |||
| | |||
| | |||
| | |||
| | |||
|} | |} | ||
== Cli tool for ABT handling == | |||
To ease the use of the ABT format, a few precompiled binaries are available. | |||
* Linux | |||
** [http://asasense.com/abt-binaries/abt-cli-ubuntu-22.04-x86_64.zip Ubuntu 22.04 x86_64] | |||
** [http://asasense.com/abt-binaries/abt-cli-ubuntu-20.04-x86_64.zip Ubuntu 20.04 x86_64] | |||
* [http://asasense.com/abt-binaries/abt-cli-win11-x64.zip Windows 11 x64] | |||
The MD5sums of the zipfiles is as follows. | |||
adcd6a227861bb6b188ef8c5748faae8 abt-cli-ubuntu-20.04-x86_64.zip | |||
6f4c63cbd7416ead026f170c97b3ab89 abt-cli-ubuntu-22.04-x86_64.zip | |||
39db54827d2634262d9d666b4262b212 abt-cli-win11-x64.zip | |||
Huidige versie van 7 aug 2025 om 09:26
The ASAsense binary table format is an alternative for the CSV format, but for binary data with fixed row lengths. The structure is as follows:
| name | length (bytes) | type | allowed values | explanation |
|---|---|---|---|---|
| file_type | 1 | u8 | 1 | The internal number of the file type and version |
| n_cols | 4 | u32 | 1-u32::max | the amount of columns |
| col_widths | 4 x n_cols | u32[n_cols] | 1-u32::max per column | n_cols u32 integers describing the byte-width of the columns |
| metadata_length | 4 | u32 | 0-u32::max | the length of the metadata field (can be 0 if it needs to be skipped) |
| metadata | metadata_length | char[] | string of fixed length (so no '\0' character to mark the end of the string) | JSON-encoded metadata |
| data | variable, until EOF | byte[] | any | The actual data |
The JSON-metadata should be an object with the following structure
| field | required | type | explanation |
|---|---|---|---|
| n_rows | false | number | The amount of rows in the file, if known upfront (can be omitted for streaming data) |
| comment | false | string | a free string |
| columns | true | a list of column definition objects | an list of objects describing the columns in more details (should have n_cols entries) |
| extra | false | JSON object | a JSON object that can freely be specified |
A "column definition object" should have the following format
| field | required | type | explanation |
|---|---|---|---|
| name | false | string | an optional name of the column |
| comment | false | string | a free string |
| datatype | true | string | An identifier for the datatype of the column |
The datatype identifiers can be anything, but the following should at least be supported by the reader:
| identifier | description |
|---|---|
| utf8 | interpret the data as an UTF8-string (with fixed length, does not need to be terminated with '\0') |
| int | interpret the data as signed integer |
| uint | interpret the data as unsigned integer |
| float | interpret as floating point data (only possible for 4 or 8 byte fields) |
| bool | boolean value (0 = false, anything else = true) |
| numtype/div | "div" should be replaced by any floating point number. The data is then first interpreted as "numtype" (int, uint or float) and then divided by "div". This is mainly useful for storing decimal numbers in integer fields. |
| unix_timestamp(numtype/div) | Interprets the data as specified within brackets (so any numtype, with optional divider), but takes into account that it is a unix_timestamp (offset seconds since the Unix Epoch on January 1st, 1970 at UTC) |
Other, unknown datatype identifiers, should be treated as opaque binary data (bytes) by the reader.
Common Extra Metadata fields
In many ABT files ASAsense uses, the following extra matadata fields will be found
| Field name | Field type | Default value | Description |
|---|---|---|---|
| sorted_by_timestamp | bool | true | Indicates whether the data in the ABT file is sorted by timestamp (column 1) |
| frame_width_s | float | NaN | Used to indicate the frame width of one sample (for example the sampling period or spectrogram frame width) |
Cli tool for ABT handling
To ease the use of the ABT format, a few precompiled binaries are available.
The MD5sums of the zipfiles is as follows.
adcd6a227861bb6b188ef8c5748faae8 abt-cli-ubuntu-20.04-x86_64.zip 6f4c63cbd7416ead026f170c97b3ab89 abt-cli-ubuntu-22.04-x86_64.zip 39db54827d2634262d9d666b4262b212 abt-cli-win11-x64.zip
