Azure Kubernetes Service with Fortanix Confidential Computing Manager

1.0 Introduction

This article describes how to set up an Azure Kubernetes Service (AKS) cluster as worker nodes in Fortanix Confidential Computing Manager (CCM).

2.0 Prerequisites

Ensure the following:

• You need to have an active Azure subscription.

• You must be logged in to the Azure CLI.

3.0 Set up AKS Cluster as Worker Nodes in Fortanix CCM

Perform the following steps to configure an Azure Kubernetes Service (AKS) cluster as worker nodes managed by Fortanix CCM:

1. Set up an SGX-capable cluster using the following commands.

   1. Create a resource group.

      az group create --name myResourceGroup --location westus2

   2. Create an SGX-capable cluster with the Confidential Computing add-on.

      NOTE

      The following command is an example for setting up a single-node cluster. Modify the parameters based on your deployment requirements.

      az aks create -l westus2 -g myResourceGroup -n myAKSCluster --vm-set-type VirtualMachineScaleSets --network-plugin azure --node-count 1 --node-vm-size Standard_DC4s_v2 --aks-custom-headers usegen2vm=true --enable-sgxquotehelper --enable-addon confcom

   3. Get the Kubernetes credentials. This will store the credentials in your .kube/config file.

      az aks get-credentials --admin --name myAKSCluster --resource-group myResourceGroup --overwrite-existing

2. Run the following commands to verify that the nodes are created successfully and that the SGX-related DaemonSets are running on the DCsv2 node pools:

   kubectl get nodes -o wide
   kubectl get pods --all-namespaces

3. Retrieve the join token for your Fortanix CCM account from the CCM user interface (UI) and store it as a Kubernetes secret in your cluster. To generate your join token, log in to https://ccm.fortanix.com/ the URL https://armor.fortanix.com/ if you are using the Armor SaaS deployment. For on-premises deployments, navigate to the configured Armor domain (https://<armor-domain>).

   1. In the CCM UI left navigation panel, navigate to Infrastructure → COMPUTE NODES → Intel SGX, and then click ADD NODE.

      

   2. In the Enroll Compute Node window, click COPY to copy the Join Token. This Join Token is used by the compute node to authenticate itself.

   3. Use the following command to store the token as a Kubernetes secret for the cluster.

      • Replace the <token> value below with your token.

        kubectl create secret generic em-token --from-literal=token=<token>

4. Deploy the Fortanix CCM node agent DaemonSet using the following YAML configuration.

   1. Save the following node agent YAML file agent-daemonset.yaml:

      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        name: em-agent
        namespace: default
        labels:
          component: em-agent
      spec:
        selector:
          matchLabels:
            component: em-agent
        template:
          metadata:
            labels:
              component: em-agent
          spec:
            hostNetwork: true
            dnsPolicy: ClusterFirstWithHostNet
            volumes:
              - name: em-agent-data
                emptyDir: {}
              - name: dev
                hostPath:
                  path: /dev
              - name: var-run-aesmd
                hostPath:
                  path: /var/run/aesmd
              - name: agent-manager-auth
                secret:
                  secretName: agent-manager-auth
            containers:
              - name: em-agent
                image: "fortanix/em-agent"
                resources:
                  limits:
                    sgx.intel.com/epc: "12Mi"
                  requests:
                    sgx.intel.com/epc: "12Mi"
                volumeMounts:
                  - name: em-agent-data
                    mountPath: /var/opt/fortanix/em-agent/node
                  - name: dev
                    mountPath: /dev/host
                  - name: var-run-aesmd
                    mountPath: /var/run/aesmd
                ports:
                  - containerPort: 9092
                    name: http
                    protocol: TCP
                    hostPort: 9092
                env:
                  - name: AGENT_MANAGER_AUTH_BASIC_TOKEN
                    valueFrom:
                      secretKeyRef:
                        name: em-token
                        key: token
                  - name: ATTESTATION_TYPE
                    value: "DCAP"
                  - name: MANAGER_ENDPOINT
                    value: "ccm.fortanix.com:443"
                  - name: NODE_IP
                    valueFrom:
                      fieldRef:
                        fieldPath: status.hostIP
                  - name: NODE_NAME
                    valueFrom:
                      fieldRef:
                        fieldPath: spec.nodeName
      

   2. Deploy the node agent DaemonSet. By default, the node agent DaemonSet supports DCAP attestation. Currently, EPID attestation is not supported and will be added in a future release of Fortanix CCM.

      kubectl create -f agent-daemonset.yaml

5. The CCM node agent DaemonSet is now deployed. Validate that the node agent pod is up and running. Look for the following in the Fortanix CCM node agent logs.

   kubectl get pods --all-namespaces

4.0 References

For more information on how to deploy an AKS cluster, refer to the Microsoft Azure Kubernetes Service (AKS) official documentation.