Splunk Integration with Sensu Server

Introduction

This guide describes the steps to integrate Splunk with an existing Sensu server.

Integration Steps

  1. List the existing handlers.
    sensuctl handler list
    SplunkSensu1.png

  2. Create a TCP handler to send the events to external systems such as Splunk.
    sensuctl handler create event-storage --type set --handlers splunk_tcp
    SplunkSensu2.png

  3. Now edit the event-storage to save the TCP handler configuration.
    sensuctl edit handler event-storage
    This will open the file in a file editor.
  4. Update the host to localhost and port to 7758. Press the “ESC” command and then “SHIFT+:WQ!” to save the file. SplunkSensu3.png

    SplunkSensu4.png
  5. To list the newly created handler, run the following command.
    sensuctl handler list
    SplunkSensu5.png

  6. To list all the checks on the Sensu servers, run the following command.
    sensuctl check list
    SplunkSensu6.png

    You will see all the checks (cpu-usage, disk-usage and so on.) are added to the email handlers. Now you need to add splunk_tcp handler created in Step 2 and 3 so that the events can be sent to the TCP handler.
  7. Now edit the check to add splunk_tcp handler.
    sensuctl edit check cpu-usage
    Add “splunk_tcp” under handlers and save the file. SplunkSensu7.png

    Similarly, edit all the checks and add the splunk_tcp handler.
  8. You can check locally using Ncat if the events are being sent to port 7758 (if the nc command is not available, the package can be installed with “#yum install nc”.
    nc -v -l -p 7758
    SplunkSensu8.png

    You can see the JSON output of the event.
  9. Download and install the Splunk forwarder on the Sensu server to forward the events to Splunk.
    wget -O splunkforwarder-8.2.0-e053ef3c985f-linux-2.6-x86_64.rpm 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=8.2.0&product=universalforwarder&filename=splunkforwarder-8.2.0-e053ef3c985f-linux-2.6-x86_64.rpm&wget=true'
    SplunkSensu9.png

  10. Create a file called “inputs.conf” under “/opt/splunkforwarder/etc/system/local” directory and add the configuration below.
    [tcp://:7758]
    index=sensu [Make sure the index name “sensu” present in Splunk]
    SplunkSensu10.png

  11. Start Splunk forwarder.
    /opt/splunkforwarder/bin/splunk start --accept-license
    The command above will prompt for username and password, enter the username and password.
  12. Configure forwarder connection to the index server.
    /opt/splunkforwarder/bin/splunk add forward-server hostname.domain:7758
    Replace the hostname.domain name with Splunk index server name or IP, in this example we are using Splunk server with IP 52.188.83.113. SplunkSensu11.png

    NOTE
    Make sure port 7758 is opened and added to receive data.
    1. Log in to Splunk and go to Settings.
    2. Under the Data section, select Forwarding and receiving.
    3. Click Receiving data (add new) and enter port 7758 and click Save. SplunkSensu12.png
      Figure 1: Splunk server
  13. Test the forwarder connection.
    /opt/splunkforwarder/bin/splunk list forward-server
    SplunkSensu13.png

  14. Log in to the Splunk console and check if the events are being sent to the index. SplunkSensu14.png
    Figure 2: Check events in Splunk

Comments

Please sign in to leave a comment.

Was this article helpful?
0 out of 0 found this helpful