Azure Kubernetes Service with Fortanix Confidential Computing Manager

Introduction

This article describes how to set up an Azure Kubernetes Service (AKS) cluster as worker nodes in Fortanix Confidential Computing Manager (CCM).

Setup AKS Cluster as Worker Nodes in Fortanix CCM

  1. Set up an SGX-capable cluster. For instructions refer to the URL:
    https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-nodes-aks-get-started
    NOTE
    Enable SGX quote helper while creating AKS.
    --enable-sgxquotehelper
    For example:
    az aks create -g myResourceGroup --name myAKSCluster --generate-ssh-keys --enable-sgxquotehelper --enable-addon confcom
  2. Run kubectl get ds -n kube-system to see the configured DaemonSets in the kube-system namespace.
  3. When you create the AKS cluster with the addon -confcom, the sgx-plugin and sgx-webhook DaemonSet is set up by default. This is the recommended method.

    If the sgx-plugin DaemonSet is not present, follow the instructions in Deploying the SGX device plugin to set up Microsoft’s SGX device plugin in your cluster.
    kubectl create -f https://github.com/Azure/aks-engine/raw/master/docs/topics/sgx/device-plugin.yaml
  4. The sgx-quote-helper DaemonSet will be up and running when you enable sgxquotehelper in step 1. This is the recommended method to create the sgx-quote-helper DaemonSet.
    If the sgx-quote-helper DaemonSet is not present, use the YAML file below and install the SGX quote helper.
    kubectl create -f sgx-quote-helper-1.17.yaml
    File contents of sgx-quote-helper-1.17.yaml:
    # Modified from the pre-1.17 version supplied by MS over email.
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
    name: sgx-quote-helper
    namespace: kube-system
    labels:
    app: sgx-quote-helper
    spec:
    selector:
    matchLabels:
    app: sgx-quote-helper
    template:
    metadata:
    labels:
    app: sgx-quote-helper
    spec:
    affinity:
    nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    nodeSelectorTerms:
    - matchExpressions:
    - key: node.kubernetes.io/instance-type
    operator: In
    values:
    - Standard_DC2s
    - Standard_DC4s
    - Standard_DC1s_v2
    - Standard_DC2s_v2
    - Standard_DC4s_v2
    - Standard_DC8_v2
    - key: kubernetes.io/os
    operator: In
    values:
    - linux
    containers:
    - name: sgx-quote-helper
    image: mcr.microsoft.com/aks/acc/sgx-attestation:0.1
    imagePullPolicy: IfNotPresent
    resources:
    limits:
    sgx.intel.com/epc: "1Mi"
    volumeMounts:
    - name: var-log
    mountPath: /var/log
    - name: var-run-aesmd
    mountPath: /var/run/aesmd
    securityContext:
    allowPrivilegeEscalation: false
    capabilities:
    drop:
    - ALL
    volumes:
    - name: var-log
    hostPath:
    path: /var/log
    - name: var-run-aesmd
    hostPath:
    path: /var/run/aesmd
  5. Retrieve the join token for your CCM zone from the CCM UI and store a copy as a Kubernetes secret in your cluster.
    1. Navigate to the COMPUTE NODES tab in the CCM UI and click “+ ENROLL NODE to bring up the token dialog. Copy the token. Quickstart18.png                                                          Figure 1: Enroll node
       
    2. Create a Kubernetes secret for the token. Replace the <token> value below with your token.
      kubectl create secret generic em-token --from-literal="token=(token-from-ccm.fortanix.com)" 
  6. Save and download the following YAML file.
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
    name: em-agent
    namespace: default
    labels:
    component: em-agent
    spec:
    selector:
    matchLabels:
    component: em-agent
    template:
    metadata:
    labels:
    component: em-agent
    spec:
    hostNetwork: true
    dnsPolicy: ClusterFirstWithHostNet
    volumes:
    - name: em-agent-data
    hostPath:
    path: /mnt/em-agent-data
    - name: dev
    hostPath:
    path: /dev
    - name: var-run-aesmd
    hostPath:
    path: /var/run/aesmd
    - name: agent-manager-auth
    secret:
    secretName: agent-manager-auth
    containers:
    - name: em-agent
    image: "fortanix/em-agent"
    resources:
    limits:
    sgx.intel.com/epc: "10Mi"
    requests:
    sgx.intel.com/epc: "10Mi"
    volumeMounts:
    - name: em-agent-data
    mountPath: /var/opt/fortanix/em-agent/node
    - name: dev
    mountPath: /dev/host
    - name: var-run-aesmd
    mountPath: /var/run/aesmd
    ports:
    - containerPort: 9092
    name: http
    protocol: TCP
    hostPort: 9092
    env:
    - name: AGENT_MANAGER_AUTH_BASIC_TOKEN
    valueFrom:
    secretKeyRef:
    name: em-token
    key: token
    - name: NODE_IP
    valueFrom:
    fieldRef:
    fieldPath: status.hostIP
    - name: NODE_NAME
    valueFrom:
    fieldRef:
    fieldPath: spec.nodeName
  7. Deploy the node agent DaemonSet.
    kubectl create -f agent-daemonset.yaml
  8. Include the following definitions in the k8s specifications of containers that use SGX. Adjust the requested amount of Electronic Product Code (EPC) memory to suit your node type and workload.
    resources:
    limits:
    sgx.intel.com/epc: "10Mi"
    env:
    - name: NODE_IP
    valueFrom:
    fieldRef:
    fieldPath: status.hostIP
    - name: NODE_AGENT_BASE_URL
    value: http://$(NODE_IP):9092/v1

    For more information about scheduling pods with EPC reservations, see Scheduling Pods to TEE enabled Hardware.

Was this article helpful?
0 out of 0 found this helpful