SPEKTRA Edge Wide-Column Store Usage

Wide-column store provides alternative data storage compared to the document storage from Goten. It is focused on wide-column data stores, and currently, we support Google Bigtable and ScyllaDB as the backend engine. However, from the user’s perspective, it is worth mentioning the properties and features of this particular database interface:

• this is a no-relational db with no joins or transactions. It has a simple interface: Query and Save (can overwrite). There is no “Get” equivalent and does not support streaming reading.
• it is semi-schemaless, it mostly stores data in binary and non-queryable format. However, it allows to definition of column families and a set of filterable field keys.
• data is stored using timestamp order. When saving, it is required to provide a timestamp per each column value. Query always requires a time range.
• data is automatically GC-ed after the defined TTL per column family.
• optimized for large data sets and throughput.
• possible query operators: EQUAL, IN, NOT EQUAL, NOT IN
• a single filter can contain many conditions connected with the AND operator.
• query itself may have many filters - effectively supporting OR operator.

This store is fully implemented in the SPEKTRA Edge repository. The store interface and implementation can be found in common/widecolumn/store/store.go. The package for this file also contains primary user-visible objects.

We have a couple of use cases (resources!) within the SPEKTRA Edge repository where we use this storage:

• in the Monitoring service, we use it for TimeSeries
• in the Audit service, we use it for ActivityLogs and ResourceChangeLogs
• in the Logging service, we use it for Logs

For each of these cases, we are wrapping wide-column store into the proper interface.

In the service specification for these resources, we are disabling metadata and standard CRUD.