Azure Kubernetes Service with Fortanix Confidential Computing Manager

Introduction

This article describes how to set up an Azure Kubernetes Service (AKS) cluster as worker nodes in Fortanix Confidential Computing Manager (CCM).

Prerequisites

• You need to have an active Azure subscription.
• You must be logged in to the Azure CLI.
• You must have the latest Azure CLI.

Setup AKS Cluster as Worker Nodes in Fortanix CCM

1. Set up an SGX-capable cluster using the following commands:
   
   1. Create a resource group.

      az group create --name myResourceGroup --location westus2

   2. Create an SGX capable cluster with a Confidential Computing addon.

      az aks create -l westus2 -g myResourceGroup -n myAKSCluster --vm-set-type VirtualMachineScaleSets --network-plugin azure --node-count 1 --node-vm-size Standard_DC4s_v2 --aks-custom-headers usegen2vm=true --enable-sgxquotehelper --enable-addon confcom --generate-ssh-keys

   3. Get the Kubernetes credentials. This will store the credentials in your .kube/config file.

      az aks get-credentials --admin --name myAKSCluster --resource-group myResourceGroup --overwrite-existing

2. Use the following commands to verify that the nodes are created properly and the SGX-related DaemonSets are running on DCsv2 node pools.

   kubectl get nodes -o wide
   kubectl get pods --all-namespaces

3. Retrieve the join token for your Fortanix CCM account from the CCM UI and store it as a Kubernetes secret in your cluster.
   1. Navigate to the COMPUTE NODES tab in the CCM UI and click “+ ENROLL NODE to bring up the token dialog. Copy the token. Figure 1: Enroll node
   2. Use the following command to store the token as a Kubernetes secret for the cluster.
      • Replace the <token> value below with your token.

        kubectl create secret generic em-token --from-literal=token=<token>

4. Deploy the node agent DaemonSet using the CCM node agent YAML file below.
   1. Fortanix CCM node agent YAML file:

      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        name: em-agent
        namespace: default
        labels:
          component: em-agent
      spec:
        selector:
          matchLabels:
            component: em-agent
        template:
          metadata:
            labels:
              component: em-agent
          spec:
            hostNetwork: true
            dnsPolicy: ClusterFirstWithHostNet
            volumes:
              - name: em-agent-data
                emptyDir: {}
              - name: dev
                hostPath:
                  path: /dev
              - name: var-run-aesmd
                hostPath:
                  path: /var/run/aesmd
              - name: agent-manager-auth
                secret:
                  secretName: agent-manager-auth
            containers:
              - name: em-agent
                image: "fortanix/em-agent"
                resources:
                  limits:
                    sgx.intel.com/epc: "12Mi"
                  requests:
                    sgx.intel.com/epc: "12Mi"
                volumeMounts:
                  - name: em-agent-data
                    mountPath: /var/opt/fortanix/em-agent/node
                  - name: dev
                    mountPath: /dev/host
                  - name: var-run-aesmd
                    mountPath: /var/run/aesmd
                ports:
                  - containerPort: 9092
                    name: http
                    protocol: TCP
                    hostPort: 9092
                env:
                  - name: AGENT_MANAGER_AUTH_BASIC_TOKEN
                    valueFrom:
                      secretKeyRef:
                        name: em-token
                        key: token
                  - name: ATTESTATION_TYPE
                    value: "DCAP"
                  - name: MANAGER_ENDPOINT
                    value: "ccm.fortanix.com:443"
                  - name: NODE_IP
                    valueFrom:
                      fieldRef:
                        fieldPath: status.hostIP
                  - name: NODE_NAME
                    valueFrom:
                      fieldRef:
                        fieldPath: spec.nodeName
      

   2. Deploy the node agent DaemonSet.

      kubectl create -f agent-daemonset.yaml

5. The CCM node agent DaemonSets are now deployed. Validate that the node agent pod is up and running using the command:

   kubectl get pods --all-namespaces

References

• Microsoft documentation to Deploy an AKS cluster: https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-nodes-aks-overview