To periodically monitor the memory usage of each manager process started by DM, as listed in the Node_Services table was added. Other Linux processes are not within the scope of this feature. If a monitored process exceeds its specified maximum memory usage to many times, a SIGABRT signal will be sent to terminate the process, prompting it to release memory resources. Subsequently, the DM will perform the predefined action for the terminated child process, as already defined in Node_Service (other_config), with a special crash reason reported via MQTT.

When the DM process starts, it retrieves the memory limits from OVSDB, which is pre-populated with the Kconfig defaults for each manager.

In case the limit is changed in database, a database callback function will refresh the effective limit for each manager.

If max_memory is not defined, the default value specified in Kconfig (DM_DEFAULT_MAX_MEMORY) will be used. If it is set to -1 for the given process, this check will be disabled, and no action will be taken by DM for that specific process.

The physical memory usage (PSS) will be examined for each process to ensure consistency with the current statistics reported to the cloud.

The number of consecutive times the memory limit can be exceeded before action is taken can be configured per process via the max_memory_cnt field. If not specified, the default value from Kconfig (DM_DEFAULT_MAX_MEMORY_CNT) is used.
With a hardcoded polling period of 30 seconds, this results in a default 5-minute grace period before the process is aborted—unless the limit is modified or the memory usage falls below the threshold, resetting the counter to 0.

• The Crash/Reports MQTT topic will be reused to send JSON-formatted reports for managers killed by DM due to exceeding memory limitations.

• The existing JSON crash report mechanism will be reused here with a new reason Maximum memory limit exceeded (xxxxxx kB).

• Example JSON Report:

  { "nodeId": "ABCDEFG10E", "locationId": "67a333ad2b0163894b5956f3", "firmwareVersion": "7.0.0-32599-g38d0bb-opensync", "model": "PP123X", "pid": "<NA>", "name": "OpenSync", "reason": "Maximum memory limit exceeded (32000 kB)", "timestamp": 1738838543963, "backtrace": "0 > 0xb6bcca00: os_backtrace 0x34 /usr/opensync/lib/libopensync.so\n 1 > 0xb6bccab8: os_backtrace_dump_generic 0xa4 /usr/opensync/lib/libopensync.so\n 2 > 0xb6bccedc: os_backtrace_dump_manager_restart 0xbc /usr/opensync/lib/libopensync.so\n 3 > 0xb6c6fc34: target_device_restart_managers_helper 0x50 /usr/opensync/lib/libopensync.so\n 4 > 0x1f874: cm2_trigger_restart_managers 0x1c0 /usr/opensync/bin/cm\n 5 > 0x2bdd4: (null) 0 /usr/opensync/bin/cm\n 6 > 0x2bfe4: cm2_connection_req_stability_check 0x8c /usr/opensync/bin/cm\n 7 > 0x20058: cm2_update_state 0x768 /usr/opensync/bin/cm\n 8 > 0xb6996fb4: ev_invoke_pending 0x7c /lib/libev.so.4\n 9 > 0xb6999530: ev_run 0x730 /lib/libev.so.4\n 10 > 0x1828c: main 0x1a8 /usr/opensync/bin/cm\n 11 > 0xb61c38cc: __libc_start_main 0x110 /lib/libc.so.6\n" }

Even though the data (max_memory) is exposed in the ovsdb database, it does not need to be managed by the cloud. The default limits will be provided at build time in Kconfig for each manager. This does not exclude the possibility that these limits could be controlled by the cloud in the future.

Support for new key/value pairs is introduced in Node_Services.other_config schema.

Example

N/A

Proposed memory limits are defined based on the expectation of a safe limit, derived from measurements of memory usage on field devices (OpenSync 6.6) and from measurements on an idle device immediately after establishing a connection.