OVSDB Persistent Storage Manager

Design

Feature purpose: implementing persistence of selected OVSDB data items. The data is stored in persistent storage of a node, and restored early during the OpenSync startup process. Any changes to the data while OpenSync is running is synchronized with the node’s persistent storage. Data synchronization with persistent storage is performed by Persistent Storage Manager (PSM).

OVSDB

Creation of new tables is not necessary to support this feature. Instead, a new column is created. This column flags a row for persistency. This method allows persistency of current and future tables with minimal modification of the OVSDB schema and source code.

Dedicated Persistence Column: os_persist

The os_persist boolean column is a generic column that can be defined in any table. The value of this flag determines whether the data of the row is saved or not. This enables data persistence granularity at a row level.

PSM downloads the current schema from the device and monitors all tables that include the os_persist column. When a change is detected and the os_persist column is set to true, the data is serialized and written to persistent storage. When the os_persist column is set to false or the row is deleted, the data is erased from persistent storage.

Upgrade

Restoring data to an upgraded table should not be an issue as long as columns are not deleted. See section Limitations below.

References

Restoring references is difficult when full database persistence is not used. For this reason, restoring of references is not supported, at least not initially. Tables with references can still be persistent, but the columns that reference other tables are ignored.

Persistent Storage Manager (PSM)

A new manager for row persistency has been introduced - PSM. At node startup, PSM performs these actions:

1. Downloading the schema from OVSDB (RFC 7074 -- get_schema method).

2. Building an internal list of tables that support persistence (include the os_persist boolean column)

3. Monitoring the persistent tables for updates.

When a persistent table is modified, PSM first checks the content of the os_presist column. If its value is true, the row is saved to persistent storage using the OpenSync Persistent Storage API (OSPS). When the os_persist column is set to false or if the os_presist column is deleted, the row is deleted from persistent storage.

Early Start

Some of the persistent data might be required during an early startup of the node. For this reason, a command line switch is implemented. The switch only restores the persistent data and exits. This can be used by the Diagnostics Manager (DM) to trigger an early restore before initializing the manager startup sequence.

Writing

At startup, PSM subscribes to all persistent tables (these include the os_persist column). When an update is detected, the entire row is stored to persistent storage.

The OSPS API works with key:value pairs, so a key must be generated. The key will be generated using a hash (SHA256) of the table name and row value.

Key Format (hashed)

The proposed format for the data-to-be-hashed is:

• The format of the data is as follows: 

  • TABLE_NAME\0COLUMN_DATA\0

• COLUMN_DATA contains the JSON row data in string representation in a sorted, escaped and compact format:

  • Libsson can produce the output described above using the json_dumps() function with the use of the JSON_COMPACT, JSON_SORT_KEYS and JSON_ENSURE_ASCII flags

  • jq can produce this output using the -acS command line switches

Once the full string is constructed, a hash is calculated from the data and used as the key. The hash algorithm is SHA256 as openssl is already a dependency of OpenSync.

In case a key with the same value exists, the new value is not written to persistent storage. PSM writes the data anyway as this kind of optimization is already guaranteed by the OSPS API.

Data Format

The data is a JSON object with two keys, table and row. Table contains the table name (as string) and row contains a JSON object that is suitable for using as the row parameter of a OVSDB insert operation (see https://tools.ietf.org/html/rfc7047#section-5.2.1 ) - this is a JSON dictionary consisting of key:value pairs where the key is the column name, and value is the JSON value of the row.

For space efficiency and to ensure consistency, the JSON_COMPACT | JSON_SORT_KEYS | JSON_ENSURE_ASCII flags are used to generate this data.

Restoring

Restoring of the data occurs once per manager restart (or more specifically, DM restart). This is triggered via a command line switch.

Since some data may be required rather early in the OpenSync start-up process, DM runs PSM in restore mode (--restore command line switch) before it starts any other manager. PSM in restore mode loads all data from the persistent storage and writes as much as it can to OVDSB.

PSM enumerates all keys in the persistent storage and reads the keys one by one. Once the key and data is read, the table and row values in the data object are used to craft an OVSDB insert operation. In case the insert operation fails, PSM continues to restore as much data as it can.

Deleting

Data deleting occurs when either os_persist will be set to false or when the row is deleted. When this happens, the row key will be simply removed from the persistent storage.

OpenSync API Extensions

Persistent storage requires enumeration of keys. This was currently not supported by the OSPS API, so the API has been extended.

The current proposed API is as follows:

typedef struct osps_keys osps_list_t;
 
/* Initialize the list object and return the first key in the storage */
const char *osps_list_begin(osps_t *ps, osps_list_t *list);
 
/* End the list object */
void osps_list_end(osps_t *ps, osps_list *list);
 
/* Return the next key in the list */
const char *osps_list_next(osps_t *ps, osps_list_t *list);
 
/* Return the data associated with the current object in the list */
ssize_t osps_list_get(osps_t *ps, osps_list_t *list, void *value, size_t value_sz);

Example Usage (pseudo code)

osps_list_t plist;
const char *key;
 
for (key = osps_list_begin(ps, &list); key != NULL; key = osps_list_next(ps, &list)
{
    char data[8192];
    ssize_t datasz;
 
    datasz = osps_list_get(ps, &list, data, sizeof(data) - 1);
 
    if (datasz <= 0)
    {
        printf(“Error retrieving data for key: %s\n”, key);
        continue;
    }
 
    data[datasz] = ‘\0’;
    printf(“%s -> %s\n”, key, data);
}
 
osps_list_end(ps, &list);

Limitations

• Due to how the algorithm for data saving works, the rows in the same table with the exact same data are saved once and are therefore restored only as a single row.

• Restoring table references is not supported. Restoring references is complex to implement with persistence granularity at the row level. To do this properly, persistence must be ensured across all tables and dependencies.

• Deleting columns from a table results in an error while restoring the data. This is certainly fixable, but it would require PSM to cross check the current schema and do an intersection of the columns.

WAN OVSDB Settings

With the introduction of persistent rows, WAN settings were moved from persistent storage to OVSDB. The new WAN settings are set in the WAN_Config table:

other_config

The other_config field is dependent on the type parameter. This is a list of supported parameters:

Examples

ovsh i WAN_Config enable:=true priority:=10 type:=pppoe other_config::'["map", [["username", "testuser"], ["password", "testpass"]]]'

Published Content

https://www.opensync.io/s/ERE-021-110-402_OpenSync_Persistent_Storage_Requirements.pdf