Using Fortanix Confidential Computing Manager for Easy Deployment of Elasticsearch on AWS Nitro

1.0 Introduction

The guide describes easy deployment of Elasticsearch cluster on Amazon Web Service (AWS) Nitro using Fortanix Confidential Computing Manager and AWS CloudFormation. It also contains the information related to:

• Deploying Solution in Amazone Web Service (AWS) Cloud

• Type of deployments

• Providing necessary permissions to AWS user

• Ingesting sample data into Elasticsearch

• Cloning sample data into Elasticsearch

• Installing and configuring Elasticsearch client - Kibana

• Sample searches using Kibana

• Deleting solution stack from AWS

1.1 Overview

Securing Elasticsearch with Confidential Computing enables the operation of the Elasticsearch database within a secure trusted execution environment, safeguarding the database's information during its active state when Elasticsearch data is processed within the CPU.

The solution orchestrates the following resources:

• Amazon Web Service

  • Security Policies

  • Resource Stack – VPC, Subnet, EC2, Security Group, IAM Role or Policy, and so on.

  • Nitro Enclave Software

• Fortanix

  • Node Agent Software

  • Elasticsearch Nitro Enclave(s) – single-node and multi-node

For detailed information, refer to Securing Elasticsearch with Confidential Computing.

1.2 Prerequisites

Ensure the following:

• AWS Account Subscription and sign-in.

• Download the Fortanix CBD-Elasticsearch CloudFormation template for AWS here.

• AWS Permissions to deploy or remove solution stack. Request the administrator to add the following IAM Policy to allow required permissions for creating or deleting AWS resources within the specified subscription.

2.0 Fortanix Confidential Database - Elasticsearch

To begin with the easy deployment of the Elasticsearch, it is mandatory to initiate a session within the AWS Management Console. This console serves as the primary interface for managing AWS resources and services, granting you access to the requisite components for configuration and deployment.

2.1 Log into AWS

Perform the following steps:

Visit https://aws.amazon.com/ and sign in.

2.2 Create Solution Stack

Perform the following steps:

1. On successful login, search for the CloudFormation option under the Services tab  

   

2. Select the CloudFormation menu and then click the Create stack button.  

   

3. On the Create stack page, select the required radio button:

   • Template is ready under Prerequisite – Prepare template section.

   • Amazon S3 URL under Specify template section.

4. Enter the Amazon S3 URL provided for the Secure Elasticsearch CloudFormation template. Alternatively, if the template is readily available, you can drag and drop it. Then click the Next button to proceed further.  

   

2.3 Input Parameters

Fill in the input parameters on the next page. These input parameters are categorized into two sections: General Parameters (AWS-specific parameters) and Confidential Database (CBD) parameters. The required input parameters are deliberately kept minimal to simplify the deployment process.

1. General parameters

   1. Stack name – Enter a required name for the solution stack.

   2. KeyPairName – Specify the AWS Keypair to connect to EC2 instances.

   3. AvailabilityZone – Select the availability zone where you want to deploy this solution.

   4. InstanceType – Select an instance type from supported options: c5a.2xlarge, c5a.4xlarge, or c5a.8xlarge.

   5. Storage – Select the storage size as per requirements.

2. Confidential DB parameters

   1. ElastcsearchVersion – Select the Elasticsearch version from the drop-down menu. The supported versions are 8.9.0, 8.8.0, and 8.7.0.

   2. DeploymentType – Select the deployment type from the supported deployment types - single node or three node cluster.

   3. ElasticsearchPassword – Set a valid password for Elasticsearch.

   

   Verify the entered values for each input parameter on the following screen and then click the Submit button.  

   

   

2.4 Solution Resources Parameters

The Resources tab furnishes an in-depth perspective on the components associated with a specific CloudFormation stack. This section delivers crucial insights about each resource, encompassing details like its type, physical ID, status, and additional metadata.

2.5 Solution Output Parameters

The Outputs tab showcases the results outlined within your stack. It reveals details regarding the stack, which might include resource identifiers, web addresses, or other pertinent data, once the stack has been successfully created or updated.

3.0 Ingest and Data Into Elasticsearch

A range of data input methods are at the disposal of Elasticsearch users.

3.1 Using Elasticsearch Python Client

Run the following command to install the Elasticsearch Python Client:

python -m pip install elasticsearch 

To illustrate, the following code snippet demonstrates the process of ingesting sample data through multiple approaches.

from elasticsearch import Elasticsearch

# Connect to Elasticsearch
es = Elasticsearch('http://localhost:9200',,basic_auth=('username', 'password'))) # Replace with your Elasticsearch server URL

# Index name and document data
index_name = 'your_index_name' # Replace with the desired index name
document = {
  'title': 'Sample Document',
  'content': 'This is some example content for the document.'
}

# Ingest the document
response = es.index(index=index_name, document=document)

# Print the response
print('Document ingested successfully:', response)
                                    

3.2 Reindex From Remote (Non-Nitro) Cluster

Execute the following REST API request to copy the documents from a source location to a destination:

POST _reindex
{
  "source": {
      "remote": {
      "host": "http://otherhost:9200",
      "username": "user",
      "password": "pass"
    },
    "index": "my-index-000001",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}

                                    

4.0 Install and Configure Kibana

Perform the following steps:

1. Run the following command to install Kibana using docker:

   docker pull docker.elastic.co/kibana/kibana:8.9.0 

2. Run the following command to execute the Kibana docker image:

   docker run -p 5601:5601 docker.elastic.co/kibana/kibana:8.9.0

3. Run the following command to configure the password for the built-in user “kibana_system” to authenticate Kibana to the Elasticsearch Cluster running in a secure enclave:

   curl -k -X POST "http://:@ec2-54-153-27-69.us-west-1.compute.amazonaws.com:9200/_security/user/kibana_system/_password?pretty" -H 'Content-Type: application/json' -d' {"password" : "changeme"}'

4. Access the Kibana through port 5601. The following dialog box might appear on the screen:  

   

5. Click the Configure manually button and enter the Elasticsearch endpoint.  

   

6. Click the Configure Elastic button.

7. Enter the password for the previously configured kibana_system user.  

   

8. Click the Configure Elastic button.  

   

9. Enter the Password for Elastic user to access the Kibana UI