Fortanix DSM for Monitoring and Visualizing the Metrics

1.0 Introduction

The purpose of this article is to describe how to configure, collect, and visualize Fortanix Data Security Manager (DSM) metrics.

In Fortanix DSM, metrics are available in Prometheus format, which can be scraped by a Prometheus server and visualized in tools such as, Grafana.

The following two categories of time-series metrics on each node are provided:

• Node Metrics: System-level performance statistics

  • CPU usage

  • Load average

  • Memory usage

  • Disk I/O statistics

  • Filesystem usage

  • Network usage

• Fortanix DSM Metrics: Service-level performance statistics

  • Number of active connections

    • Public (port 443)

    • KMIP (port 5969)

    • Internal admin (port 4444)

  • Logging backlog queue length

    • Elasticsearch

    • Splunk

    • Other log integrations

2.0 Terminology References

• KMIP – Key Management Interoperability Protocol

• CPU – Central Processing Unit

• TLS – Transport Layer Security

• IP – Internet Protocol

3.0 Installing Fortanix DSM Monitoring Package

If the Fortanix DSM monitoring package is not already installed, run the following command on each Fortanix DSM node:

sudo apt-get install sdkms-monitoring

NOTE

If the package was installed before upgrading to version 3.21, this step is not required.

4.0 Setting Up Metrics Collection

This section describes the steps to enable and start the required services for publishing metrics information.

Perform the following steps on each Fortanix DSM node:

1. Run the following command to configure the default settings for node_exporter:

   sudo cp /opt/fortanix/sdkms/monitoring/node_exporter.default /etc/default/node_exporter

2. Run the following command to enable the node-exporter service:

   sudo systemctl enable node-exporter

3. Run the following command to start the node-exporter service:

   sudo systemctl start node-exporter

4. Run the following command to verify the status of the node-exporter service:

   sudo systemctl status node-exporter

5. Run the following command to configure the default settings for sdkms_exporter:

   sudo cp /opt/fortanix/sdkms/monitoring/sdkms_exporter.default /etc/default/sdkms_exporter

6. Run the following command to enable the sdkms-metrics service:

   sudo systemctl enable sdkms-metrics

7. Run the following command to start the sdkms-metrics service:

   sudo systemctl start sdkms-metrics

8. Run the following command to verify the status of the sdkms-metrics service:

   sudo systemctl status sdkms-metrics

5.0 Configuring TLS for Metrics Collection

This section describes the steps to configure the sdkms-metrics service to use TLS for secure metrics collection.

Perform the following steps on each Fortanix DSM node:

1. Obtain a TLS private key and certificate.

   1. Both files must be in PEM format.

   2. Save them in the /opt/fortanix/sdkms/monitoring/ directory.

2. Run the following command to edit the sdkms-metrics service file:

   sudo vi /etc/systemd/system/sdkms-metrics.service

3. In the sdkms-metrics service file, replace the existing ExecStart line with the following line:

   ExecStart=/opt/fortanix/sdkms/monitoring/exporter_exporter \
     -config.file /opt/fortanix/sdkms/monitoring/sdkms_exporter.yml \
     -web.tls.cert /opt/fortanix/sdkms/monitoring/CERT_FILENAME \
     -web.tls.key /opt/fortanix/sdkms/monitoring/KEY_FILENAME \
     -web.tls.listen-address :9998

   NOTE

   • Replace CERT_FILENAME and KEY_FILENAME with the actual certificate and key file names respectively.

   • This example uses port 9998. You can configure a different port, if required.

4. Run the following command to reload the systemd configuration:

   sudo systemctl daemon-reload

5. Run the following command to restart the sdkms-metrics service:

   sudo systemctl restart sdkms-metrics.service

6.0 Metrics Endpoints

Metrics are published on the following endpoints by default.

NOTE

If TLS is enabled, update the endpoint URL to use https and the configured port number.

6.1 Node Metrics

You can access node metrics using the following endpoint:

http://NODE_IP_ADDRESS:9999/proxy?module=node

The sample data is available from the Prometheus/ node_exporter.

6.2 Fortanix DSM Metrics

You can access Fortanix DSM metrics using the following endpoint:

http://NODE_IP_ADDRESS:9999/proxy?module=sdkms

The following is the sample output from the endpoint:

# HELP es_backlog Number of pending ES documents
# TYPE es_backlog gauge
es_backlog 0
# HELP other_log_integrations Number of pending audit logs
# TYPE other_log_integrations gauge
other_log_integrations 0
# HELP kmip_connections Number of active kmip connections
# TYPE kmip_connections gauge
kmip_connections 0
# HELP splunk_queue_len Number of pending Splunk log events
# TYPE splunk_queue_len gauge
splunk_queue_len 0
# HELP splunk_pending_logs Number of pending Splunk logs
# TYPE splunk_pending_logs gauge
splunk_pending_logs 0
# HELP admin_connections Number of active admin connections
# TYPE admin_connections gauge
admin_connections 1
# HELP public_connections Number of active public connections
# TYPE public_connections gauge
public_connections 1

7.0 Configuring Prometheus

You can extend your existing Prometheus configuration to scrape Fortanix DSM metrics.

Add the following jobs to your Prometheus configuration file:

NOTE

Update the targets with the IP addresses of your deployment nodes.

- job_name: 'node_metrics'
    scrape_interval: 300s
    metrics_path: /proxy
    params:
      module:
        - node
    static_configs:
      - targets: ['NODE1_IP:9999']
      - targets: ['NODE2_IP:9999']
      - targets: ['NODE3_IP:9999']

  - job_name: 'sdkms_metrics'
    scrape_interval: 60s
    metrics_path: /proxy
    params:
      module:
        - sdkms
    static_configs:
      - targets: ['NODE1_IP:9999']
      - targets: ['NODE2_IP:9999']
      - targets: ['NODE3_IP:9999']

8.0 Visualization

If Prometheus server is collecting metrics, you can use Grafana to visualize them.

• Node Metrics: Use the prebuilt Node Exporter dashboard in Grafana to visualize system-level metrics. You can also customize the dashboard as needed.

  

• Fortanix DSM Metrics: Create a custom Grafana dashboard using the collected Fortanix DSM metrics.