--- title: "(Release 5.6) Kubernetes Version Upgrade to 1.34 K8s" slug: "fortanix-data-security-manager-release-56-kubernetes-version-upgrade-to-134-k8s" updated: 2026-04-01T07:36:34Z published: 2026-02-26T19:19:15Z canonical: "support.fortanix.com/fortanix-data-security-manager-release-56-kubernetes-version-upgrade-to-134-k8s" --- > ## Documentation Index > Fetch the complete documentation index at: https://support.fortanix.com/llms.txt > Use this file to discover all available pages before exploring further. # (Release 5.6) Kubernetes Version Upgrade to 1.34 K8s ## 1.0 Introduction The article describes the procedure to upgrade Kubernetes from version 1.32.5 to 1.34.3 for Fortanix-Data-Security-Manager (DSM) release 5.6. ## 2.0 Overview The Fortanix DSM 5.6 release will upgrade the system from Kubernetes version 1.32 to 1.34. Subsequent Kubernetes upgrades will be released as part of regular upgrades or could continue to be independent upgrades. After upgrading Fortanix DSM to the 5.6 version, you will not be able to downgrade to previous releases. The Fortanix DSM UI will not allow a downgrade after 5.6 is installed. Kindly work with Fortanix Support to ensure you have a valid backup that can be used to perform a manual recovery. Also, you will need to upgrade Fortanix DSM to 5.6 before moving to any future release. ## 3.0 Pre-Upgrade Checks Before upgrading the Kubernetes, ensure the following: ### 3.1 Check and Manage Disk Space Run the following command to check if the disk space of more than 15 GB is available in `/var` and root (`/`) directories: If the disk space is less than 15 GB, delete the oldest version of Fortanix DSM from the user interface (UI). ```bash sudo df -h /var/ / ``` The following is the sample output: ```bash Filesystem Size Used Avail Use% Mounted on /dev/nvme0n1p1 993G 29G 964G 3% / /dev/nvme0n1p1 993G 29G 964G 3% / ``` ### 3.2 Check Software Versions in Endpoints Run the following command to check if all software versions are available in all the endpoints: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get ep -n swdist ``` The following is the sample output: ```bash NAME      ENDPOINTS                                         AGE swdist    10.244.0.212:22,10.244.1.191:22,10.244.2.152:22   242d v2649     10.244.0.212:22,10.244.1.191:22,10.244.2.152:22   4d v2657     10.244.0.212:22,10.244.1.191:22,10.244.2.152:22   2d ``` ### 3.3 Check Cluster and Node Health 1. Run the following command to ensure that the overlay mount matches with this on each node: ```bash sudo cat /etc/systemd/system/var-opt-fortanix-swdist_overlay.mount.d/options.conf ``` ```bash [Mount] Options=lowerdir=/var/opt/fortanix/swdist/data/vXXXX/registry:/var/opt/fortanix/swdist/data/vYYYY/registry ``` Here, ‘`vXXXX`’ is the previous version and ‘`vYYYY`’ is the upgraded version. 2. Ensure that the latest backup is triggered and verify that it is a successful backup (size and other metrics). 3. All nodes must report as healthy and be running `Kubernetes` version `1.32.5` and kernel `6.8.0-83-1.0-fortanix-appliance`. Run the following command to get the nodes and list the IP: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get nodes -o wide ``` Look for the version number under the column `VERSION` and it must be `v1.32.5` for each of the nodes. ```bash NAME            STATUS ROLES        AGE   VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME ip-172-31-0-189 Ready control-plane 3h44m v1.32.5 172.31.0.189 Ubuntu 20.04.6 LTS 6.8.0-83-1.0-fortanix-appliance containerd://1.7.12 ip-172-31-1-110 Ready control-plane 3h37m v1.32.5 172.31.1.110 Ubuntu 20.04.6 LTS 6.8.0-83-1.0-fortanix-appliance containerd://1.7.12 ip-172-31-2-217 Ready control-plane 3h33m v1.32.5 172.31.2.217 Ubuntu 20.04.6 LTS 6.8.0-83-1.0-fortanix-appliance containerd://1.7.12 ``` 4. All pods are healthy in `default`, `swdist`and `kube-system` namespaces. 5. Run the following command to check `kubeadm` configuration on the cluster: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get configmap kubeadm-config -oyaml -nkube-system ``` This should return the following values for parameters in the master configuration: - kubernetesVersion: v1.32.5 - imageRepository: *containers.fortanix.com:5000* ### 3.4 Check Etcd Cluster and Component 1. Run the following command to check the status of `etcd` and if `isLeader=true` is assigned to one of the `etcd` node. 1. `etcd` should be TLS migrated. 2. Run the following command to generate the list of `etcd` members: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl exec -nkube-system -- etcdctl --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/healthcheck-client.crt --key /etc/kubernetes/pki/etcd/healthcheck-client.key --endpoints https://127.0.0.1:2379 endpoint status ``` The following is the sample output of the above command: ```bash Defaulted container "etcd" out of: etcd, etcd-wait (init) bf2dc0512cac45c3, started, dev-test-3, https://10.197.192.251:2380, https://10.197.192.251:2379, false ``` 3. Run the following command to ensure that the version of `etcd` on each of the `etcd` pods is `3.5.16`: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl exec -nkube-system -- etcd --version ``` The following is the sample output of the above command: ```bash Defaulted container "etcd" out of: etcd, etcd-wait (init) etcd Version: 3.5.16 Git SHA: e72f3c2 Go Version: go1.24.9 Go OS/Arch: linux/amd64 ``` 4. Run the following command to check the health of `etcd` cluster and ensure that the cluster is healthy: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl exec -nkube-system -- etcdctl --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/healthcheck-client.crt --key /etc/kubernetes/pki/etcd/healthcheck-client.key --endpoints https://127.0.0.1:2379 endpoint health ``` The following is the sample output of the above command: ```bash Defaulted container "etcd" out of: etcd, etcd-wait (init) https://127.0.0.1:2379 is healthy: successfully committed proposal: took = 5.953722ms ``` 5. On each node, navigate to `/etc/kubernetes/manifests` directory and run the following command to check the image versions for all Kubernetes control-plane components: ```bash sudo grep -i "image:" /etc/kubernetes/manifests/*.yaml ``` 6. Perform the following steps to check the expiry of the Kubernetes certificates. 1. Run the following commands to check the expiry of the certificates under `/etc/kubernetes/pki` and `/etc/kubernetes/pki/etcd` directories: ```bash sudo find /etc/kubernetes/pki/ -name '*.crt' -exec openssl x509 -noout -dates -in {} \; | grep notAfter sudo find /etc/kubernetes/pki/etcd -name '*.crt' -exec openssl x509 -noout -dates -in {} \; | grep notAfter ``` 2. Run the following command to renew the expired certificates: ```bash sudo /opt/fortanix/sdkms/bin/renew-k8s-certs.sh ``` 7. Run the following command on each node to check the status of `kubelet`, `container`, and `docker-registry` service: ```bash sudo systemctl status containerd sudo systemctl status kubelet sudo systemctl status docker-registry ``` > [!NOTE] > NOTE > > Ensure that the status of the services is `Running`. ## 4.0 Post-Upgrade Checks Ensure to refer to [*Section 3.0: Pre-Upgrade Checks*](/v1/docs/fortanix-data-security-manager-release-434-kubernetes-version-upgrade-to-130-k8s#30-preupgrade-checks) before upgrading the Kubernetes: ### 4.1 Check Node and Deployment Status 1. Run the following command to check the status of the deploy pod: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get pods | grep deploy ``` The following is the sample output of the above command: ```bash deploy-vqq7r     0/1     Completed   0    32m ``` > [!NOTE] > NOTE > > Ensure that the status of the pod is `Completed`. 2. Run the following command to get the list of the deploy job: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get job deploy ``` The following is the sample output of the above command: ```bash NAME     STATUS   COMPLETIONS   DURATION   AGE deploy   Complete    1/1           20m      41m ``` > [!NOTE] > NOTE > > Verify the completion and duration of the job. 3. If you are using [DC Labeling](/v1/docs/fortanix-data-security-manager-data-center-labeling), run the following command to verify if the zone label is added by the YAML of the node: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get node node_name -o yaml | grep -i 'zone' ``` 4. Run the following command to check the status of the nodes and the k8s version and the role must be control-plane: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get nodes -o wide ``` The following is the sample output of the above command: ```bash NAME         STATUS     ROLES         AGE   VERSION    INTERNAL-IP    EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME dsm-test-1   Ready    control-plane   9m   v1.34.3   10.197.192.252   Ubuntu 24.04.3 LTS   6.8.0-100-1.0-fortanix-appliance containerd://1.7.12 ``` > [!NOTE] > NOTE > > Ensure the following: > > - `STATUS` of the nodes is `Ready` > - `VERSION` column reflects `v1.34.3` > - `ROLES` column reflects `control-plane` > - `KERNEL-VERSION` reflects `6.8.0-100-1.0-fortanix-appliance` ### 4.2 Check Kubernetes and Component Version 1. Run the following command to generate the list of `etcd` members: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl exec etcd-dsm-test-1 -nkube-system -- etcdctl --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/healthcheck-client.crt --key /etc/kubernetes/pki/etcd/healthcheck-client.key --endpoints https://127.0.0.1:2379 member list ``` The following is the sample output of the above command: ```bash Defaulted container "etcd" out of: etcd, etcd-wait (init) ff5eaaee755acae0, started, dsm-test-1, https://10.197.192.252:2380, https://10.197.192.252:2379, false ``` 2. Run the following command to check if `kube-proxy` is upgraded to image `1.34.3`: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl describe ds kube-proxy -nkube-system | grep Image ``` The following is the sample output of the above command: ```plaintext Image:      containers.fortanix.com:5000/kube-proxy:v1.34.3 ``` 3. Run the following command to check if `kured` pod is running with image version `1.21.0`: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl describe ds kured -nkube-system | grep Image ``` The following is the sample output of the above command: ```bash Image: containers.fortanix.com:5000/kured:1.21.0 ``` 4. Run the following command on each of the nodes in the cluster to check if `kube-apiserver`, `kube-controller-manager`, `kube-scheduler` are upgraded to `1.34.3`: ```bash sudo grep -i "image:" /etc/kubernetes/manifests/*.yaml ``` 5. Run the following command to check the version of `etcd`: ```bash sudo /etc/kubernetes/manifests# kubectl get pod etcd-dsm-lab-11 -n kube-system -o  yaml | grep image: ``` The following is the sample output of the above command: ```bash image: containers.fortanix.com:5000/etcd:3.6.5-0 image: containers.fortanix.com:5000/etcd-init:0.1.0 image: containers.fortanix.com:5000/etcd:3.6.5-0 image: containers.fortanix.com:5000/etcd-init:0.1.0 ``` 6. Run the following command to check the version of `cert-manager` helm chart: ```bash sudo helm list -A ``` The following is the sample output of the above command: ```bash NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                          APP VERSION certmanager     cert-manager    2               2024-09-21 14:43:18.753311051 +0000 UTC deployed        cert-manager-v1.15.3            v1.15.3 csiplugin       cert-manager    2               2024-09-21 14:43:21.133524751 +0000 UTC deployed        cert-manager-csi-driver-v0.10.1 v0.10.1 ``` > [!NOTE] > NOTE > > Ensure that the `helm chart` version is `1.15.3` and `csiplugin` version is `0.10.1`. 7. Run the following command to check if the Kubernetes version is upgraded to `v1.34.3` (including `kubeadm`, `kubectl`, `kubelet packages`): ```bash sudo dpkg -l | grep kube ``` The following is the sample output of the above command: ```bash ii  kubeadm                               1.34.3-1.1                amd64        Kubernetes Cluster Bootstrapping Tool ii  kubectl                               1.34.3-1.1                        amd64        Command-line utility for interacting with a Kubernetes cluster ii  kubelet                               1.34.3-1.1                        amd64        Node agent for Kubernetes clusters ii  kubernetes-cni                        1.6.0-1.1                          amd64        Kubernetes CNI ``` 8. Run the following command to check if image tag `1.0.10` for `swdist` container is updated: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl describe ds swdist -nswdist | grep Image ``` The following is the sample output of the above command: ```bash    Image:      containers.fortanix.com:5000/swdist:1.0.10    Image:      containers.fortanix.com:5000/swdist:1.0.10    Image:      containers.fortanix.com:5000/swdist:1.0.10    Image:      containers.fortanix.com:5000/swdist:1.0.10    Image:      containers.fortanix.com:5000/swdist:1.0.10    Image:      containers.fortanix.com:5000/swdist:1.0.10    Image:      containers.fortanix.com:5000/swdist:1.0.10    Image:      containers.fortanix.com:5000/swdist:1.0.10 ``` 9. Run the following command to check the replicas of `coredns` deployment: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get pods -nkube-system -owide | grep coredns ``` The following is the sample output of the above command: ```bash coredns-5bdcd56d4b-6t7g2               1/1     Running   0             32m   10.244.0.61      dsm-test-1             ``` > [!NOTE] > NOTE > > Ensure that number of duplicate `coredns` must be equal to the number of nodes in the cluster. 10. Run the following command to check the version of `flannel` and `flannel-plugin`: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get ds kube-flannel-ds -n kube-system -o yaml | grep image: ``` The following is the sample output of the above command: ```bash image: containers.fortanix.com:5000/flannel:v0.28.1 image: containers.fortanix.com:5000/flannel-cni-plugin:v1.9.0-flannel1 image: containers.fortanix.com:5000/flannel:v0.28.1 ``` > [!NOTE] > NOTE > > Ensure that the `flannel` version is `0.28.1` and `flannel plugin` version is `1.9.0-flannel1`. ### 4.3 Check cert-manager Configuration Run the following command to check all the resources of `cert-manager`: ```bash sudo KUBECONFIG=/etc/kubernetes/admin.conf kubectl get all -n cert-manager ``` The following is the sample output of the above command: ```bash NAME READY STATUS RESTARTS AGE pod/cert-manager-csi-driver-9lvw2 3/3 Running 4 (14h ago) 15h pod/certmanager-cert-manager-5fd9f859bb-7slz2 1/1 Running 0 14h pod/certmanager-cert-manager-cainjector-5998546469-pk9kb 1/1 Running 0 14h pod/certmanager-cert-manager-webhook-878f95fb5-699lp 1/1 Running 0 14h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/certmanager-cert-manager ClusterIP 10.245.213.126   9402/TCP 15h service/certmanager-cert-manager-webhook ClusterIP 10.245.20.237   443/TCP 15h NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/cert-manager-csi-driver 1 1 1 1 1   15h NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/certmanager-cert-manager 1/1 1 1 15h deployment.apps/certmanager-cert-manager-cainjector 1/1 1 1 15h deployment.apps/certmanager-cert-manager-webhook 1/1 1 1 15h NAME DESIRED CURRENT READY AGE replicaset.apps/certmanager-cert-manager-5fd9f859bb 1 1 1 14h replicaset.apps/certmanager-cert-manager-6c6bdd85d9 0 0 0 15h replicaset.apps/certmanager-cert-manager-cainjector-5998546469 1 1 1 14h replicaset.apps/certmanager-cert-manager-cainjector-7b7cbc6988 0 0 0 15h replicaset.apps/certmanager-cert-manager-webhook-555cbb78cd 0 0 0 15h replicaset.apps/certmanager-cert-manager-webhook-878f95fb5 1 1 1 14h ``` ## 5.0 Troubleshooting 1. In case kubelet client certificates expire (`/var/lib/kubelet/pki/kubelet-client.crt`) and there is no `/var/lib/kubelet/pki/kubelet-client-current.pem` file present, then you can create the certificates using the following commands: ```bash TEMP_DIR=/etc/kubernetes/tmp mkdir -p $TEMP_DIR BACKUP_PEM="/var/lib/kubelet/pki/kubelet-client-current.pem" KEY="/var/lib/kubelet/pki/kubelet-client.key" CERT="/var/lib/kubelet/pki/kubelet-client.crt" echo "Stopping kubelet service" systemctl stop kubelet echo "Creating a new key and cert file for kubelet auth" nodename=$(echo "$HOSTNAME" | awk '{print tolower($0)}') openssl req -out $TEMP_DIR/tmp.csr -new -newkey rsa:2048 -nodes -keyout $TEMP_DIR/tmp.key -subj "/O=system:nodes/CN=system:node:$nodename" cat > $TEMP_DIR/kubelet-client.ext << HERE keyUsage = critical,digitalSignature,keyEncipherment extendedKeyUsage = clientAuth HERE echo "Signing the generated csr with kubernetes CA" openssl x509 -req -days 365 -in $TEMP_DIR/tmp.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out $TEMP_DIR/tmp.crt -sha256 -extfile $TEMP_DIR/kubelet-client.ext cp $TEMP_DIR/tmp.crt $CERT cp $TEMP_DIR/tmp.key $KEY chmod 644 $CERT chmod 600 $KEY if grep -q "client-certificate-data" $KUBELET_CONF; then    echo "Updating file $KUBELET_CONF to add reference to restored certificates"    sed -i "s|\(client-certificate-data:\s*\).*\$|client-certificate: $CERT|" $KUBELET_CONF    sed -i "s|\(client-key-data:\s*\).*\$|client-key: $KEY|" $KUBELET_CONF fi echo "Starting kubelet service" systemctl start kubelet ``` 2. Upgrade on two node cluster can fail due to `etcd` quorum failure. In such a scenario, if pods are healthy, you can re-run the deploy job manually using the following command. This will eventually upgrade the cluster to `1.34.3`. ```bash sudo sdkms-cluster deploy --stage DEPLOY --version ``` > [!WARNING] > WARNING > > Two node upgrades are not recommended. Fortanix Data Security Manager (DSM) is the world’s first cloud service secured with Intel® SGX. With Fortanix DSM, you can securely generate, store, and use cryptographic keys and certificates, as well as other secrets such as passwords, API keys, tokens, or any blob of data. Your business-critical applications and containers can integrate with Fortanix DSM using legacy cryptographic interfaces (PKCS#11, CNG, and JCE) or using the native Fortanix DSM RESTful interface.