Fortanix DSM with Sensu Monitoring and Alerting

Prev Next

1.0 Introduction

This article provides a comprehensive overview of the Sensu monitoring and alerting solution for Fortanix-Data-Security-Manager (DSM). It outlines how to set up Sensu for real-time alerting and system and service monitoring to ensure proactive management and Fortanix DSM operational dependability.

2.0 Terminology References

  • SNMP - Simple Network Management Protocol

  • TLS - Transport Layer Security

  • CPU - Central Processing Unit

  • RHEL - Red Hat Enterprise Linux

  • TCP - Transmission Control Protocol

  • CA - Certificate Authority

  • SSL - Secure Sockets Layer

  • IP - Internet Protocol

3.0 Prerequisites

Before deploying Sensu, ensure that the server hosting the Sensu virtual machine (VM) meets the following minimum specifications:

  • CPU: 2 CPUs with 2 cores each

  • Memory: 8 GB RAM

  • Storage: 256 GB Disk/ROM

  • Operating System (OS): RHEL, Ubuntu, and CentOS

3.1 List of Required Ports

The following ports must be accessible on the monitoring server for the Sensu Web user interface (UI) and for receiving notifications from monitoring agents. The values listed are default ports and can be modified if required.

Protocol

Port Number

Purpose

TCP

8081

Receive notification from agents

TCP

8080

Sensu Backend

TCP

22

Access the Sensu backend VM

TCP

3000

Sensu Dashboard Web UI

TCP

80

Asset download from Sensu Web Server

4.0 Architecture Diagram

This solution is delivered as a self-contained virtual appliance that customers can install and set up in their deployment environment. The solution follows a client-server architecture.

MonitoringAlerting.png

Figure 1: Fortanix DSM Monitoring and Alerting Solution - Deployment Architecture

Fortanix DSM nodes are pre-installed with a monitoring agent that:

  • Performs various health checks.

  • Publishes the check results to the defined transport mechanism. The results are then received by the Sensu server running in the solution VM.

To enable monitoring, the Fortanix DSM node must be configured to point to the customer’s deployed instance of the Fortanix DSM monitoring and alerting solution.

4.1 Alerting Mechanisms

When the system detects an alert, it can deliver notification about the alert through a configured mechanism.  

By default, the solution supports the following alerting mechanisms:

  • Email: Requires an SMTP email configuration to deliver emails.

  • Slack: Requires a Slack API key to push alerts into Slack.

  • SNMP Trap: Requires SNMP trap receiver information to send traps to.

  • Custom: This is based on a shell script that the alerting server will invoke to send out alert notifications. This mechanism can be used to invoke any third-party client or executable.

5.0 Configure Sensu Monitoring for Fortanix DSM

The Fortanix DSM monitoring and alerting solution is delivered as a software bundle that users can deploy on their server or VM.

5.1 Setting Up Sensu Server

Download the Fortanix Monitoring Server artifacts onto the Sensu backend VM from here.

Perform the following steps:

  1. Download or copy the server artifacts tarball (Monitoring-Server-Artifacts.tgz) to the designated VM.

  2. Run the following command to extract the tarball:

    tar zxvf Monitoring-Server-Artifacts.tgz
  3. Run the following command to navigate to the Monitoring-Server-Artifacts folder:

    cd Monitoring-Server-Artifacts
  4. Run the following commands to install the sensu-backend and sensu-cli packages based on your OS (RHEL/Ubuntu):

    sudo rpm -i sensu-go-backend-6.12.0-7321.x86_64.rpm/deb
    sudo rpm -i sensu-go-cli-6.12.0-7321.x86_64.rpm/deb
  5. Run the following commands to copy the backend configuration file:

    cp backend.yml.example backend.yml
    cp backend.yml /etc/sensu
  6. Run the following command to enable the sensu-backend service:

    systemctl enable sensu-backend
  7. Run the following command to start the sensu-backend service:

    systemctl start sensu-backend
  8. Run the following command to verify the sensu-backend service status:

    systemctl status sensu-backend
  9. Run the following commands to initialize the sensu-backend service:

    export SENSU_BACKEND_CLUSTER_ADMIN_USERNAME=admin
    export SENSU_BACKEND_CLUSTER_ADMIN_PASSWORD=password
    sensu-backend init
  10. Run the following command to configure the command-line tool Sensuctl:

    sensuctl configure -n  --username 'admin' --password 'password' --namespace default --url 'http://127.0.0.1:8080'
  11. It is recommended to change the default admin password. Run the following command:

    sensuctl user change-password --interactive
  12. Sensu creates a default agent user with the password P@ssw0rd!, which corresponds to the defaults used by the Sensu agent. It is recommended to update this password by running the following command:

    sensuctl user change-password agent --current-password 'P@ssw0rd!' --new-password fortanix
  13. Run the following commands to start the web server to host Sensu assets:

    cd Monitoring-Server-Artifacts
    cd sensu-assets
    chmod +x web_server
    sudo ./web_server&
    cd ..
  14. Run the following command to create assets:

    ./add_assets.sh

    Ensure to provide the Sensu backend VM IP when prompted.

  15. Run the following command to create checks:

    ./add_checks.sh
  16. Open the Sensu dashboard in a browser and verify that all checks are present:

    http://<Sensu Server IP Address>:3000
  17. If you want to configure TLS for secure communication, perform the following:

    1. Copy the TLS certificate, key, and CA certificate file to the /etc/sensu folder.

    2. Update the following in the backend.yml file:

      • API URL: Change the prefix from http to https:

        api-url: "https://localhost:8080"
      • SSL configuration: Set the following lines and update change the file name based on your files:

        cert-file: "/etc/sensu/cert.pem"
        key-file: "/etc/sensu/key.pem"
        trusted-ca-file: "/etc/sensu/ca.pem"
        insecure-skip-tls-verify: true
  18. Run the following commands to initialize daemon-reload and restart the sensu-backend service:

    systemctl daemon-reload
    systemctl restart sensu-backend
  19. Access the Sensu dashboard securely using the URL: https://<Sensu Server IP Address>:3000.

5.2 Setting Up Sensu Agent on Fortanix DSM Nodes

Run the following steps on each Fortanix DSM node:

  1. Run the following command to install the Fortanix DSM Monitoring package:

    sudo apt-get install sdkms-monitoring
  2. Run the following commands to copy the agent.yml configuration file:

    cp /opt/fortanix/sdkms/monitoring/agent.yml /etc/sensu/
  3. Run the following command to navigate to the /etc/sensu folder:

    cd /etc/sensu
  4. Edit the agent.yml file with the following content to point to the Sensu server VM:

    backend-url:
      - "ws://<SENSU BAKCEND SERVER IP ADDRESS>:8081"

    Ensure to uncomment the lines after modifying the agent.yml file.

  5. To configure TLS for secure communication, perform the following:

    1. Copy the CA file used by the Sensu server to the /etc/sensu folder.

    2. Update the following in the agent.yml file:

      1. Trusted CA File: Set the following lines and update the file name based on your files.

        trusted-ca-file: "/etc/sensu/ca.pem"
      2. Backend URL: Use the protocol prefix “wss” instead of “ws”.

        backend-url:
          - "ws://<sensu-backend-IP>:8081"
    3. If the certificate is self-signed and the root CA is not present on the Fortanix DSM servers, then add the following line:

      insecure-skip-tls-verify: true
  6. Run the following command to copy the edited agent.yml file to the /etc/sensu folder:

    sudo cp agent.yml /etc/sensu/
  7. Run the following commands to start and enable the sensu-agent service:

    sudo systemctl daemon-reload
    sudo systemctl start sensu-agent
    sudo systemctl enable sensu-agent
    sudo systemctl status sensu-agent
  8. Run the following command to check the status of the sensu-agent service:

    sudo systemctl status sensu-agent