Using Fortanix Data Security Manager with EDB Postgres for TDE

1.0 Introduction

This article describes the integration procedure for Fortanix-Data-Security-Manager (DSM) and Postgres Enterprise Database (EDB) to enable Transparent Data Encryption (TDE).

This article provides detailed information on the following:

• Enabling secure communication and authentication between Fortanix DSM and Postgres EDB through the utilization of Key Management Interoperability Protocol (KMIP) and certificates.

• Configuring and initializing Fortanix DSM.

• Generating client certificates to enhance security as part of the integration process.

• Generating Postgres master key and encrypting the database.

2.0 KMIP and Certificate Requirements

This section outlines the essential requirements for the Key Management Interoperability Protocol (KMIP) and certificates in the integration process. KMIP plays a crucial role in enabling secure communication between EDB Postgres and Fortanix DSM. It leverages Transport Layer Security (TLS) to establish a secure connection, and Fortanix DSM uses this protocol to authenticate KMIP clients to effectively manage and utilize keys stored within Fortanix DSM.

Additionally, X.509 certificates are used to facilitate secure communication and authentication between both Fortanix DSM and Postgres. Fortanix DSM is configured with a server certificate signed by an internal Certificate Authority (CA). Furthermore, it is necessary to generate a client certificate for EDB Postgres using tools such as OpenSSL. This certificate can either be signed externally or self-signed.

3.0 Prerequisites

Ensure the following:

• A functioning EDB Postgres distribution.

• Fortanix DSM version 4.4 or later must be installed and operational.

• Fortanix DSM must be accessible by EDB Postgres on port 5696 (default) or on a custom KMIP port, if configured.

• Users must have access to OpenSSL or another suitable tool for generating a client certificate and private key in the Privacy Enhanced Mail (PEM) format.

• Python and the PyKMIP utility must be installed on your server.

  NOTE

  These requirements are only applicable to versions 15.2 and later of EDB Postgres Advanced Server and versions 15.2 and later of EDB Postgres Extended Server, as only these versions support TDE.

4.0 Configure Fortanix DSM

A Fortanix DSM service must be configured, and the URL must be accessible. To create a Fortanix DSM account and group, refer to the following sections:

4.1 Signing Up

To get started with the Fortanix DSM cloud service, you must register an account at <Your_DSM_Service_URL>. For example, https://eu.smartkey.io.

For detailed steps on how to set up the Fortanix DSM, refer to the User's Guide: Sign Up for Fortanix Data Security Manager SaaS documentation.

4.2 Creating an Account

Access <Your_DSM_Service_URL> in a web browser and enter your credentials to log in to Fortanix DSM.

For more information on how to set up an account in Fortanix DSM, refer to the User's Guide: Getting Started with Fortanix Data Security Manager - UI.

4.3 Creating a Group

Perform the following steps to create a group in the Fortanix DSM:

1. In the DSM left navigation panel, click the Groups menu item, and then click the + button to create a new group.

   

2. On the Adding new group page, do the following:

   1. Title: Enter a name for your group.

   2. Description (optional): Enter a short description of the group.

3. Click SAVE to create the new group.

The new group is added to the Fortanix DSM successfully.

4.4 Creating an Application

Perform the following steps to create an application (app) in the Fortanix DSM:

1. In the DSM left navigation panel, click the Apps menu item, and then click the + button to create a new app.

   

2. On the Adding new app page, do the following:

   1. App name: Enter the name for your application.

   2. ADD DESCRIPTION (optional): Enter a short description of the application.

   3. Authentication method: Select the default API Key as the authentication method from the drop down menu. For more information on these authentication methods, refer to the User's Guide: Authentication.

   4. Assigning the new app to groups: Select the group created in Section 4.3: Creating a Group from the list.

3. Click SAVE to add the new application.

The new application is added to the Fortanix DSM successfully.

4.5 Copying the App UUID

Perform the following steps to copy the app UUID from the Fortanix DSM:

1. In the DSM left navigation panel, click the Apps menu item, and then click the app created in Section 4.4: Creating an Application to go to the detailed view of the app.

2. From the top of the app’s page, click the copy icon next to the app UUID to copy it to use in Section 4.8: Generating the Certificate as the value of Common Name (CN) to generate a self-signed certificate and a private key.

4.6 Creating a Security Object

Perform the following steps to generate an AES key in the Fortanix DSM:

1. In the DSM left navigation panel, click the Security Objects menu item, and then click the + button to create a new security object.

   

2. On the Add new Security Object page, do the following:

   1. Security Object name: Enter the name for your security object.

   2. Group: Select the group as created in Section 4.3: Creating a Group.

   3. Select the GENERATE radio button.

   4. In the Choose a type section, select the AES key type.

   5. In the Key Size section, select the size of the key in bits.

   6. In the Key operations permitted section, select the required operations to define the actions that can be performed with the cryptographic keys, such as encryption, decryption, signing, and verifying.

      NOTE

      Ensure that the Encrypt and Decrypt permissions are selected.

3. Click GENERATE to create the new security object.

4.7 Copying the Security Object UUID

Perform the following steps to copy the security object UUID from the Fortanix DSM:

1. In the DSM left navigation panel, click the Security Objects menu item, and then click the security object created in Section 4.6: Creating a Security Object to go to the detailed view of the security object.

2. From the top of the security object’s page, click COPY ID drop down and select the COPY UUID option to copy the UUID to use for your PGDATAKEYWRAPCMD and PGDATAKEYUNWRAPCMD commands.

4.8 Generating the Certificate

If an application or client requires certificate-based authentication to Fortanix DSM, it is essential to embed the app UUID within the certificate as the value of the Common Name (CN), as described above. For example, CN: ae044928-f670-48d1-a4aa-111baf5640a6.

Perform the following steps to create a self-signed certificate, ensuring you have the app UUID to include in the self-signed certificate:

1. Run the following command to log in to your EDB Postgres distribution system as the database superuse:

   sudo su - enterprisedb

2. Run the following command to create a dsm folder in Linux machine and enter to the folder:

   mkdir dsm
   cd dsm

3. Run the following command to generate an RSA key for authentication with Fortanix DSM:

   edbopenssl req -newkey rsa:2048 -nodes -keyout client-key.pem -x509 -days 365 -out client-cert.pem

   When prompted for the Common Name, enter the Fortanix DSM app UUID as copied in Section 5.5: Copying the App UUID.
   This process will generate two essential files:

   • client-cert.pem

   • client-key.pem

   

   NOTE

   The given procedure is with Self Signed Certificate, but Fortanix DSM can also support CA Signed Certificate, if preferred by the user.

4.9 Updating the Authentication Method

Perform the following steps to change the authentication method:

1. Go to the detailed view of the app created in Section 4.4: Creating an Application and click Change authentication method and select the Certificate option to change the authentication method to Certificate.

2. Click SAVE.

3. The certificate content can be read using the command cat client-cert.pem:

   

4. On the Add certificate dialog box, click UPLOAD NEW CERTIFICATE to upload the certificate file or paste the content of the certificate generated in previous section.

5. Select both check boxes to confirm your understanding of the action.

6. Click UPDATE to save the changes.

5.0 Installing TDE-KMIP Client on the Server

If you do not have the KMIP client installed, perform the following steps to install the edb-tde-kmip-client on your system as a root user:

1. Open a terminal or command prompt.

2. Execute the installation command based on your system.

   For example,

   1. If you are using Linux RHEL server, run the following command:

      dnf install edb-tde-kmip-client

   2. If you are using Ubuntu server, run the following command:

      apt-get install edb-tde-kmip-client

6.0 Creating the pykmip.conf File

Perform the following steps to configure the pykmip.conf file:

1. On the system where your EDB Postgres distribution is located, navigate to /usr/lib/edb/kmip/ to access the edb_tde_kmip_client.py client.

2. In the /usr/lib/edb/kmip directory, create a file named pykmip.conf and provide the following configuration details:

   • Host: The Fully Qualified Domain Names (FQDN) of the KMIP hosts.

   • Port: The KMIP port, typically set to 5696.

   • Keyfile: The path to the PEM-encoded client certificate key file.

   • Certfile: The path to the PEM-encoded client certificate file.

   • ca_certs: The path to the Fortanix DSM certificate chain. The Fortanix DSM certificate chain shall be retrieved from the browser when navigating to the respective DSM URL (example: eu.smartkey.io).
     For example:

     [client]
     host=<DSM_URL>
     port=5696
     keyfile=/var/lib/edb/dsm/client-key.pem 
     certfile=/var/lib/edb/dsm/client-cert.pem
     ca_certs=/var/lib/edb/dsm/ca.pem 

     For more information on the pykmip.conf file and its contents, refer to the PyKMIP documentation.

7.0 Verifying Encryption and Decryption

To ensure that the key you have created can successfully encrypt and decrypt data, execute the following two commands as the superuser on your system with the EDB Postgres distribution:

7.1 Encrypt Data

Run the following command: If encryption is successful without any error, the file test.bin will be created. The corresponding encryption logs are available in the Fortanix DSM.

python3 /usr/edb/kmip/client/edb_tde_kmip_client.py encrypt --out-file=test.bin --pykmip-config-file=/var/lib/edb/dsm/pykmip.conf --key-uid='173bb3c8-ef16-4e4e-b066-54fe72cdd296' --variant=pykmip

Where,

• KMIP client Location: /usr/edb/kmip/client/edb_tde_kmip_client.py

• Output file: test.bin

• Location of PyKMIP configuration file: /var/lib/edb/dsm/pykmip.conf

• Encrypted key output: TDE key output.

• Variant: Allows for KMIP compatibility with Fortanix.

7.2 Decrypt Data

Run the following command:

python3 /usr/edb/kmip/client/edb_tde_kmip_client.py decrypt --in-file=test.bin --pykmip-config-file=/var/lib/edb/dsm/pykmip.conf --key-uid='173bb3c8-ef16-4e4e-b066-54fe72cdd296' --variant=pykmip

If successful, the decryption command will produce the output of the original "secret." The corresponding encryption and decryption logs are available in the Fortanix DSM.

8.0 Key Rotation

This section provides the steps to securely rotate the KEK in a TDE setup.

8.1 Rotating the Key Encryption Key

This section describes the steps to change the Master Encryption Key (MEK). You need to manually execute the unwrap command, specifying the old key, and then provide the output to the wrap command, specifying the new key. These operations can be performed while the database server is running. The wrapped data key is used only on startup and is not utilized during the server's runtime.

The following is the unwrap command:

python3 /usr/lib/edb/kmip/edb_tde_kmip_client.py decrypt --in-file=key.bin --pykmip-config-file=/var/lib/edb-as/dsm/pykmip.conf  --key-uid='<Current Key ID>' --variant=pykmip

The following is the wrap command:

python3 /usr/lib/edb/kmip/edb_tde_kmip_client.py encrypt --out-file=key.bin --pykmip-config-file=/var/lib/edb-as/dsm/pykmip.conf --key-uid='<New Key ID>' --variant=pykmip

Perform the following steps to rotate the encryption key:

1. Run the following commands to create a new AES 256 key using pykmip:

   cd ~/pykmip/
   python3.9 ./kmip/demos/pie/create.py -a AES -l 256

   The following is the output:

   2024-09-11 10:05:39,278 - demo - INFO - Successfully created symmetric key with ID: 34816dab-24e5-4635-
   9990-00684b84a8c4

   The key is created in the KMIP vault.

2. Run the following command to change directory to the pg_encryption folder in the database:

   cd $PGDATA/pg_encryption

   For example,

   cd /var/lib/edb-as/16/data/pg_encryption

3. Run the following command to save the original key.bin file:

   cp key.bin key.bin.original

4. Run the following commands to create the new key.bin file based on the newly created key:

   python3.9 /usr/edb/kmip/client/edb_tde_kmip_client.py decrypt --pykmip-config-file=/etc/pykmip/pykmip.conf --key-uid="<Current Key ID>" --in-file=key.bin --variant=pykmip | python3.9 /usr/edb/kmip/client/edb_tde_kmip_client.py encrypt --out-file=key.bin --pykmip-config-file=/etc/pykmip/pykmip.conf --key-uid="<New Key ID>" --variant=pykmip

   Where,

   • <Current Key ID> refers to the ID of the key currently in use to decrypt the data encryption key.

   • <New Key ID> refers to the ID of the newly created key that will be used for future decryption operations after key rotation.

   For example,

   python3 /usr/lib/edb/kmip/edb_tde_kmip_client.py decrypt --in-file=key.bin --pykmip-config-file=/var/lib/edb-as/dsm/pykmip.conf --key-uid='<Current Key ID>' --variant=pykmip | python3 /usr/lib/edb/kmip/edb_tde_kmip_client.py encrypt --out-file=key.bin --pykmip-config-file=/var/lib/edb-as/dsm/pykmip.conf --key-uid='<New Key ID>' --variant=pykmipThis command creates a new key.bin file.

5. Navigate to the data directory PGDATA and edit the postgresql.conf file.

6. Change the key in the data_encryption_key_unwrap_command under the Authentication section in the postgresql.conf file:

   data_encryption_key_unwrap_command = 'python3.9 /usr/edb/kmip/client/edb_tde_kmip_client.py decrypt
   --pykmip-config-file=/etc/pykmip/pykmip.conf --key-uid="<New Key ID>" --in-file=%p
   --variant=pykmip'

7. Run the following command to back to the HOME directory:

   cd

8. Run the following command to stop the database:

   /usr/edb/as16/bin/pg_ctl -D /var/lib/edb/as16/data -l $HOME/logfile stop

   The following is the output of the command:

   waiting for server to shut down.... done
   server stopped

9. Run the following command to start the database:

   /usr/edb/as16/bin/pg_ctl -D /var/lib/edb/as16/data -l $HOME/logfile start

   The following is the output of the command:

   waiting for server to start.... done
   server started

10. Run the following command to connect to the database:

    /usr/edb/as16/bin/psql hr

11. Run the following command to attempt to read the data in the dept table:

    hr=# SELECT * FROM dept;

    The following is the output of the command:

     deptno |   dname    |   loc
    --------+------------+----------
         10 | ACCOUNTING | NEW YORK
         20 | RESEARCH   | DALLAS
    (2 rows)

The key rotation is successful when the system can access and decrypt the data.

9.0 Performing initdb for the Database

After creating the key and verifying encryption and decryption, you can export the PGDATAKEYWRAPCMD and PGDATAKEYUNWRAPCMD to wrap and unwrap your encryption key and initialize your database.

Perform the following steps:

1. Navigate to the /bin directory where your executables are located. For example:

   cd /usr/edb/as15/bin

2. Perform your initdb as required for your database, for example:

   ./initdb --data-encryption -D /var/lib/edb/as15/data

3. If the initialization is successful, your output should confirm this.

   

4. Start your database using the following command:

   ./pg_ctl -D /var/lib/edb/as15/data -l logfile start

   The following is the output of the command:

   waiting for server to start.... done
   server started

5. Navigate to your /data directory to view the postgresql.conf file. Ensure that your data_encryption_key_unwrap_command, which you have set with export PGDATAUNWRAPCMD, is present under the Authentication section in the postgresql.conf file.

For more information on how Transparent Data Encryption (TDE) is integrated with EDB Postgres Advanced Server and EDB Postgres Extended Server, refer to the EDB Transparent Data Encryption documentation.