1.0 Introduction
This article describes how to set up an Elastic Kubernetes Service (EKS) cluster as worker nodes in Fortanix Confidential Computing Manager (CCM) and and run the Nitro application on the EKS cluster.
1.1 Prerequisites
- The worker nodes of the EKS cluster should be of type i3en.xlarge or larger.
- You must have EKS cluster created.
2.0 Running Pre-User Data Script
- Run the following command to log in to each worker node using the
ssh
client connect:ssh -i "psa-710-nitro.pem" ec2-user@ec2-3-101-65-194.us-west-1.compute.amazonaws.com
- Run the
pre_userdata.sh
script on each worker nodes:sudo amazon-linux-extras install aws-nitro-enclaves-cli -y
sudo yum install aws-nitro-enclaves-cli-devel -y
sudo usermod -aG ne ec2-user
sudo usermod -aG docker ec2-user
#sudo sed -i "s/cpu_count: 2/cpu_count: 4/g" /etc/nitro_enclaves/allocator.yaml
sudo sed -i "s/memory_mib: 512/memory_mib: 2560/g" /etc/nitro_enclaves/allocator.yaml
sudo systemctl start nitro-enclaves-allocator.service
sudo systemctl enable nitro-enclaves-allocator.service
sudo systemctl start docker
sudo systemctl enable docker
sudo chgrp ne /dev/nitro_enclaves
echo "Done with AWS Nitro enclave Setup"sudo reboot
- You must increase the memory of HughPages. Run the following commands to update the value of
memory_mib
parameter to2560
in/etc/nitro_enclaves/allocator.yaml
file:--- # Enclave configuration file. # # How much memory to allocate for enclaves (in MiB). memory_mib: 2560 # # How many CPUs to reserve for enclaves. cpu_count: 2
- You must increase the memory of HughPages. Run the following commands to update the value of
- Restart the worker node to reflect the changes:
reboot
3.0 Get Kubernetes Credentials
- Run the following command to get
kubeconfig
:eksctl utils write-kubeconfig --cluster=<cluster_name>
- Run the following command to label all the nodes with Nitro Enclave OS capabilities:
kubectl label node <node-name> smarter-device-manager=enabled
- Run the following command to label all the nodes with
enclave.example.com/type=nitro
:kubectl label node <node-name> enclave.example.com/type=nitro
- Run the following command to install the Smarter Device Manager on the Kubernetes cluster:
kubectl label node <node-name> enclave.example.com/type=nitro
- Run the following command to install Smart Device Manager:
apply -f smarter-device-manager-ds-with-cm.yaml
4.0 Creating Secrets
- Run the following command to create a secret to access the ECR registry:
kubectl create secret docker-registry regcred --docker-server=513076507034.dkr.ecr.us-west-1.amazonaws.com --docker-username=AWS --docker-password=$(aws ecr get-login-password)
- Run the following command to create a secret for join token:
kubectl create secret generic em-token --from-literal=token=<join-token-from-account>
5.0 Installing Node Agent
Run the following command to install the node agent as daemonset
:
kubectl apply -f agent-daemonset.yaml
The following is the content for agent-daemonset.yaml
file:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: em-agent
namespace: default
labels:
component: em-agent
spec:
selector:
matchLabels:
component: em-agent
template:
metadata:
labels:
component: em-agent
spec:
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
volumes:
- name: hugepage
emptyDir:
medium: HugePages
- name: log
hostPath:
path: /var/log/nitro_enclaves
- name: socket-path
emptyDir: {}
- name: node-data
hostPath:
path: /tmp/em-agent-nitro
imagePullSecrets:
- name: regcred
containers:
- name: em-agent
image: "513076507034.dkr.ecr.us-west-1.amazonaws.com/em-agent-nitro:3.13.52"
resources:
limits:
smarter-devices/nitro_enclaves: "1"
hugepages-2Mi: 512Mi
memory: 2Gi
cpu: 250m
requests:
smarter-devices/nitro_enclaves: "1"
volumeMounts:
- mountPath: /dev/hugepages
name: hugepage
readOnly: false
- name: log
mountPath: /var/log/nitro_enclaves
- name: socket-path
mountPath: /run/nitro_enclaves
- name: node-data
mountPath: /tmp/em-agent-nitro
ports:
- containerPort: 9092
name: http
protocol: TCP
hostPort: 9092
env:
- name: AGENT_MANAGER_AUTH_BASIC_TOKEN
valueFrom:
secretKeyRef:
name: em-token
key: token
- name: MANAGER_ENDPOINT
value: "ccm.test.fortanix.com"
- name: MALBORK_LOG_DEBUG
value: "true"
- name: NODE_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
6.0 Reinstalling Node Agent in a Different Account
This section lists the steps for installing the node agent in an account and switching it to different account.
- Run the following command to delete
/tmp/em-agent-nitro
parameter from each worker nodes:
# log into the node through debug container.
$ kubectl debug node/<node-name> -it --image=busybox
# inside container now run chroot to access the node.
$ chroot /host bash
# Delete the file
$ rm -rf /tmp/em-agent-nitro
# Exit twice to get out of the container
$ exit - Run the following command to create a new
em-token
secret:$ kubectl delete secret em-token
$ kubectl create secret generic em-token --from-literal=token=<join-token-from-account> - Run the following command to restart the
em-agent
secret:kubectl rollout restart ds em-agent
7.0 Converting Nitro Enclave OS Application
Refer to the User's Guide: Create an Image to know the steps for creating application in Fortanix Confidential Computing Manager user interface.
7.1 Creating NGINX Deployment
The following is the content of app-development.yaml
file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-pod
labels:
app: my-pod
spec:
replicas: 1
selector:
matchLabels:
app: my-pod
template:
metadata:
labels:
app: my-pod
spec:
imagePullSecrets:
- name: regcred
containers:
- name: nginx
image: 513076507034.dkr.ecr.us-west-1.amazonaws.com/development-images/ccm-automation-input-images:hellowworld
ports:
- name: http
containerPort: 80
imagePullPolicy: Always
securityContext:
privileged: true
resources:
limits:
smarter-devices/nitro_enclaves: "1"
hugepages-1Gi: 2Gi
memory: 2Gi
cpu: 250m
requests:
smarter-devices/nitro_enclaves: "1"
hugepages-1Gi: 2Gi
volumeMounts:
- mountPath: /dev/hugepages
name: hugepage
readOnly: false
- name: log
mountPath: /var/log/nitro_enclaves
- name: socket-path
mountPath: /run/nitro_enclaves
env:
- name: NODE_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
- name: NODE_AGENT
value: http://$(NODE_IP):9092/v1/
- name: RUST_LOG
value: debug
volumes:
- name: hugepage
emptyDir:
medium: HugePages
- name: log
hostPath:
path: /var/log/nitro_enclaves
- name: socket-path
emptyDir: {}
Update the container's image details to run any other image. Run the following command to deploy the application:
kubectl apply -f app-deployment.yaml
Comments
Please sign in to leave a comment.