Watchdog service

The Watchdog service can monitor the operation of your device, test the system for problems, and automatically restart that device if it detects a fault or failure. You can also see metrics for the Watchdog service and performance results of the tests performed.

When the Watchdog service has been enabled, the service name and green check mark displays in the dashboard.

Configure the Watchdog service

To configure the Watchdog service on your Connect IT 16/48:

  1. Log into Digi Remote Manager, or log into the local Web UI as a user with full Admin access rights.
  2. Access the device configuration:

    1. Locate your device as described in Use Digi Remote Manager to view and manage your device.
    2. Click the Device ID.
    3. Click Settings.
    4. Click to expand Config.

    1. On the menu, click System. Under Configuration, click Device Configuration.

      The Configuration window is displayed.

  3. Click System > Advanced Watchdog.
  4. The watchdog is disabled by default. To enable, click to toggle off Disable.
  5. For Watchdog test interval, type the amount of time between running system tests.

    Allowed values are any number of days, hours, minutes, or seconds, and take the format number{d|h|m|s}.

    For example, to set Watchdog test interval to ten minutes, enter 10m or 600s.

    The maximum is two days (2d), and the default is five minutes (5m).

  6. Type or select the Number of test failures before a reboot.
  7. Configure the tests that the watchdog will perform:
    1. Click to expand Fault detection tests.
    2. Click to expand Memory usage.
      1. The memory check is enabled by default. To disable, click the Enable memory check toggle.
      2. For RAM usage threshold to trigger a warning, type or select the percentage of RAM usage that will trigger a warning. The minimum value is 60 percent, the maximum is 100 percent. The default is 90 percent.
      3. Type or select the Percentage of system memory used before triggering a reboot. The minimum value is 60 percent, the maximum is 100 percent. The default is 95 percent.
      4. To log memory usage with every watchdog memory usage test, click to enable Log memory usage every interval.
    3. Click to expand Interface tests.
      1. Click the Enable interface(s) down check toggle to enable. The system periodically checks the interfaces you configure here and, after the specified amount of time, reboots them.
      2. Click to expand Check interface(s).
      3. Click to add a new interface.
      4. For Interface, choose the interface you want to test.
    4. Click to expand Modem down. This configuration is enabled by default.
      1. Click the Enable modem check toggle to disable.
      2. Click the Enable modem power cycle toggle if you want the modem to be power cycled after an initial timeout instead of this timeout being reported as a failure.
      3. For Downtime, type the amount of time the modem is down before it is reported.
  8. Click Apply to save the configuration and apply the change.
  1. Select the device in Remote Manager and click Actions > Open Console, or log into the Connect IT 16/48 local command line as a user with full Admin access rights.

    Depending on your device configuration, you may be presented with an Access selection menu. Type admin to access the Admin CLI.

  2. At the command line, type config to enter configuration mode:

    > config
(config)>

  3. The watchdog is enabled by default. To disable:
    (config)> system watchdog enable false
(config)>
  4. Set the amount of time between running system tests:
    (config)> system watchdog interval value
(config)>

    where value is any number of days, hours, minutes, or seconds, and takes the format number{d|h|m|s}.

    For example, to set interval to ten minutes, enter either 10m or 600s:

    (config)> system watchdog interval 600s
(config)>

    The maximum is two days (2d), and the default is five minutes (5m).

  5. Set the number of test failures before the system reboots:
    (config)> system watchdog num_failures int
(config)>

  6. Configure the tests that the watchdog will perform:

    1. The memory check is enabled by default. To disable:
      (config)> system watchdog tests memory enable false
(config)>
    2. Set the percentage of RAM usage that will trigger a warning:
      (config)> system watchdog tests memory max_memory_warning int
(config)>

      The minimum value is 60 percent, the maximum is 100 percent. The default is 90 percent.

    3. Set the percentage of RAM usage that will trigger a reboot of the device:
      (config)> system watchdog tests memory max_memory_critical int
(config)>

      The minimum value is 60 percent, the maximum is 100 percent. The default is 95 percent.

    4. To log memory usage with every watchdog memory usage test, enable log_memory:
      (config)> system watchdog tests memory log_memory true
(config)>
    5. To have the interface(s) checked and rebooted after the specified amount of time: 
      (config)> system watchdog tests interfaces interfaces add [value]
(config)>

      6. with value being the name of the interface.

    6. To have the modem power cycled after an initial timeout instead of this timeout being reported as a failure:
      7. (config)> system watchdog tests modem 
(config)>
  7. Save the configuration and apply the change.
    (config)> save
Configuration saved.
>
  8. Type exit to exit the Admin CLI.

    Depending on your device configuration, you may be presented with an Access selection menu. Type quit to disconnect from the device.

View Watchdog metrics

To view metrics for the Watchdog service and the tests performed:

In the local Web UI of your Connect IT 16/48:

  1. Log in to the local Web UI of your device as a user with full Admin access rights.

  2. To access the Watchdog Service page:

    1. In the Services card, you can see the operational status of the Watchdog service.

    2. Click Watchdog to view metrics.

    Click Status > Services > Watchdog to see the page.

  1. Click Devices, and select a device from the list.

  2. Click Metrics.

  3. Click to expand Sys Details.

  4. Click Sys Watchog Failures.

A new window opens and displays a chart showing the test failures and when they occurred.

 

To view the results of the Watchdog tests:

  1. Access the Command Line Interface for your Connect IT, from either the local web UI as an administrator with full access rights or from Digi Remote Manager.
  2. At the prompt, type
    show watchdog

    All tests that were performed, as well as their status are listed.

  3. Type exit to exit the CLI.