Splunk Integration with Sensu Server


This guide describes the steps to integrate Splunk with an existing Sensu server.

Integration Steps

  1. List the existing handlers.
    sensuctl handler list

  2. Create a TCP handler to send the events to external systems such as Splunk.
    sensuctl handler create event-storage --type set --handlers splunk_tcp

  3. Now edit the event-storage to save the TCP handler configuration.
    sensuctl edit handler event-storage
    This will open the file in a file editor.
  4. Update the host to localhost and port to 7758. Press the “ESC” command and then “SHIFT+:WQ!” to save the file. SplunkSensu3.png

  5. To list the newly created handler, run the following command.
    sensuctl handler list

  6. To list all the checks on the Sensu servers, run the following command.
    sensuctl check list

    You will see all the checks (cpu-usage, disk-usage and so on.) are added to the email handlers. Now you need to add splunk_tcp handler created in Step 2 and 3 so that the events can be sent to the TCP handler.
  7. Now edit the check to add splunk_tcp handler.
    sensuctl edit check cpu-usage
    Add “splunk_tcp” under handlers and save the file. SplunkSensu7.png

    Similarly, edit all the checks and add the splunk_tcp handler.
  8. You can check locally using Ncat if the events are being sent to port 7758 (if the nc command is not available, the package can be installed with “#yum install nc”.
    nc -v -l -p 7758

    You can see the JSON output of the event.
  9. Download and install the Splunk forwarder on the Sensu server to forward the events to Splunk.
    wget -O splunkforwarder-8.2.0-e053ef3c985f-linux-2.6-x86_64.rpm 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=8.2.0&product=universalforwarder&filename=splunkforwarder-8.2.0-e053ef3c985f-linux-2.6-x86_64.rpm&wget=true'

  10. Create a file called “inputs.conf” under “/opt/splunkforwarder/etc/system/local” directory and add the configuration below.
    index=sensu [Make sure the index name “sensu” present in Splunk]

  11. Start Splunk forwarder.
    /opt/splunkforwarder/bin/splunk start --accept-license
    The command above will prompt for username and password, enter the username and password.
  12. Configure forwarder connection to the index server.
    /opt/splunkforwarder/bin/splunk add forward-server hostname.domain:7758
    Replace the hostname.domain name with Splunk index server name or IP, in this example we are using Splunk server with IP SplunkSensu11.png

    Make sure port 7758 is opened and added to receive data.
    1. Log in to Splunk and go to Settings.
    2. Under the Data section, select Forwarding and receiving.
    3. Click Receiving data (add new) and enter port 7758 and click Save. SplunkSensu12.png
      Figure 1: Splunk server
  13. Test the forwarder connection.
    /opt/splunkforwarder/bin/splunk list forward-server

  14. Log in to the Splunk console and check if the events are being sent to the index. SplunkSensu14.png
    Figure 2: Check events in Splunk


Please sign in to leave a comment.

Was this article helpful?
0 out of 0 found this helpful