Azure Kubernetes Service with Fortanix Confidential Computing Manager

Introduction

This article describes how to set up an Azure Kubernetes Service (AKS) cluster as worker nodes in Fortanix Confidential Computing Manager (CCM).

Prerequisites

• You need to have an active Azure subscription.
• You must be logged in to the Azure CLI.
• You must have the latest Azure CLI.
• Install aks-preview extension using: az extension add --name aks-preview.

Setup AKS Cluster as Worker Nodes in Fortanix CCM

1. Set up an SGX-capable cluster using the following commands:
   
   1. Create a resource group.

      az group create --name myResourceGroup --location westus2

   2. Create an SGX capable cluster with Confidential Computing addon.

      az aks create -l westus2 -g myResourceGroup -n myAKSCluster --vm-set-type VirtualMachineScaleSets --network-plugin azure --node-count 1 --node-vm-size Standard_DC4s_v2 --aks-custom-headers usegen2vm=true --enable-sgxquotehelper --enable-addon confcom

   3. Get the Kubernetes credentials. This will store the credentials in your .kube/config file.

      az aks get-credentials --admin --name myAKSCluster --resource-group myResourceGroup --overwrite-existing

2. Use the following commands to verify that the nodes are created properly and the SGX-related DaemonSets are running on DCsv2 node pools.

   kubectl get nodes -o wide
   kubectl get pods --all-namespaces

3. Retrieve the join token for your Fortanix CCM account from the CCM UI and store it as a Kubernetes secret in your cluster.
   1. Navigate to the COMPUTE NODES tab in the CCM UI and click “+ ENROLL NODE to bring up the token dialog. Copy the token. Figure 1: Enroll node
   2. Use the following command to store the token as Kubernetes secret for the cluster.
      • Replace the <token> value below with your token.

        kubectl create secret generic em-token --from-literal=token=<token>

4. Deploy the node agent DaemonSet using the CCM node agent YAML file below.
   1. Fortanix CCM node agent YAML file:

      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        name: em-agent
        namespace: default
        labels:
          component: em-agent
      spec:
        selector:
          matchLabels:
            component: em-agent
        template:
          metadata:
            labels:
              component: em-agent
          spec:
            hostNetwork: true
            dnsPolicy: ClusterFirstWithHostNet
            volumes:
            - name: em-agent-data
      	emptyDir: {}
            - name: dev
              hostPath:
                path: /dev
            - name: var-run-aesmd
              hostPath:
                path: /var/run/aesmd
            - name: agent-manager-auth
              secret:
                secretName: agent-manager-auth
            containers:
            - name: em-agent
              image: "fortanix/em-agent"
              resources:
                limits:
                  sgx.intel.com/epc: "12Mi"
                requests:
                  sgx.intel.com/epc: "12Mi"
              volumeMounts:
              - name: em-agent-data
                mountPath: /var/opt/fortanix/em-agent/node
              - name: dev
                mountPath: /dev/host
              - name: var-run-aesmd
                mountPath: /var/run/aesmd
              ports:
              - containerPort: 9092
                name: http
                protocol: TCP
                hostPort: 9092
              env:
              - name: AGENT_MANAGER_AUTH_BASIC_TOKEN
                valueFrom:
                  secretKeyRef:
                    name: em-token
                    key: token
              - name: ATTESTATION_TYPE
                value: "DCAP"
              - name: MANAGER_ENDPOINT
                value: "ccm.fortanix.com:443"
              - name: NODE_IP
                valueFrom:
                  fieldRef:
                    fieldPath: status.hostIP
              - name: NODE_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
      

   2. Deploy the node agent DaemonSet.

      kubectl create -f agent-daemonset.yaml

5. The CCM node agent DaemonSets are now deployed. Validate that the node agent pod is up and running using the command:

   kubectl get pods --all-namespaces

References

• Microsoft documentation to Deploy an AKS cluster: https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-nodes-aks-get-started