Using Fortanix Data Security Manager with BigID

1.0 Introduction

This article describes how to integrate Fortanix Data Security Manager (DSM) with BigID. This integration provides data protection at scale using BigID’s data discovery with Fortanix DSM’s data protection capabilities.

Users of the application can trigger data-protection actions on their Relational Database Management System (RDBMS) objects (tables/columns) through Fortanix using the BigID platform. The BigID platform is used to identify any security and privacy-related findings at scale across your data ecosystem and dynamically trigger data-protection actions on the applicable assets (relational databases as of now) using the tokenization capabilities of the Fortanix platform.

2.0 Actions To Be Performed On BigID

The Fortanix- BigID application package, release.zip includes the following:

• Docker images (.tar file)
• Docker-Compose file (bigid-fortanix.yml)
• The following shell script files:
  • start-bigid-fortanix.sh
  • stop-bigid-fortanix.sh

1. Copy the application package to your server (the following example is for sftp):

   /usr/bin/sftp -i ~/.ssh/<your-certificate> bigid@<server-ip>
   sftp> put release.zip
   Sftp> bye

2. On the server, extract the release.zipfile using the following command:

   $ unzip -d /release release.zip

3. Navigate to the extracted files and start up the App container using the following command:

   $ cd release
   $ ./start-bigid-fortanix.sh

   NOTE

   If you should need to take the container down for any reason, run the following command from the same directory:

   $ ./stop-bigid-fortanix.sh

4. Verify the dockers are up and running using the following command:

   $ docker ps
   bigexchange/bigid-fortanix:1.0   ...    Up     ....
   

3.0 Configuring Fortanix DSM

1. Sign up at https://smartkey.io/.
2. Log in to the Fortanix DSM UI.
3. Create a group in your Fortanix DSM account. For more information on creating a group, refer to Getting Started Guide.
   Figure 1: Create a group
4. Create an application (App) within the group created in the previous step. For more information on creating an App, refer to Getting Started Guide.
   Figure 2: Create an App
   NOTE

   Select API Key as the Authentication method for the application.
5. Click COPY API KEY to copy the application’s API key. This will be used to authenticate Fortanix DSM with BigID.
   Figure 3: Copy API key
6. Create a tokenization security object . For more information on creating a tokenization security object, refer to User's Guide: Tokenization.
   1. To create a tokenization security object, click the Security Objects tab and click the   button to add a new security object.
   2. In the Add New Security Object form:
      1. Create a tokenization security object with the name database.table.column.
      2. Select GENERATE to generate a tokenization secret.
      3. In the Choose a type section, select the key type as Tokenization.
      4. Select the Data type to be tokenized.
      5. Click GENERATE to generate the key.
         Figure 4: Create a tokenization security object
         Figure 5: Tokenization security object
7. Import the Fortanix DSM’s ‘DB Gateway Plugin’ into the group created in Step 3. For more information on importing plugins, refer to User's Guide: Plugins.
   
   Alternatively, you can create a new plugin by copying the following source code:

   
   --
   -- IMPORT: Name "DB Secrets Plugin"
   --
   --
   -- EXAMPLES:
   --
   --{
   --     "operation":"create",
   --     "server":"something"
   --     "timeout":300
   --}
   --
   --{
   --     "operation":"drop",
   --     "name":"some_client"
   --}
   --
   --{
   --    "operation":"tokenize",
   --    "server":"18.144.47.239",
   --    "dbname":"ccs",
   --    "table":"employee",
   --    "col":"cc"
   --}
   
   function get_date_from_unix(unix_time)
       local day_count, year, days, month = function(yr) return (yr % 4 == 0 and (yr % 100 ~= 0 or yr % 400 == 0)) and 366 or 365 end, 1970, math.ceil(unix_time/86400)
     
       while days >= day_count(year) do
         days = days - day_count(year) year = year + 1
       end
       local tab_overflow = function(seed, table) for i = 1, #table do if seed - table[i] <= 0 then return i, seed end seed = seed - table[i] end end
       month, days = tab_overflow(days, {31,(day_count(year) == 366 and 29 or 28),31,30,31,30,31,31,30,31,30,31})
       local hours, minutes, seconds = math.floor(unix_time / 3600 % 24), math.floor(unix_time / 60 % 60), math.floor(unix_time % 60)
       local period = hours > 12 and "pm" or "am"
       hours = hours > 12 and hours - 12 or hours == 0 and 12 or hours
       return string.format("%04d%02d%dT%02d%02d%02dZ", year, month, days, hours, minutes, seconds)
     end
     
     function generate_secret()
       local chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*()_+-={}|[]`~"
       local length = 32
       local randomString = ""
     
       charTable = {}
       for c in chars:gmatch"." do
         table.insert(charTable, c)
       end
     
       for i = 1, length do
         randomString = randomString .. charTable[math.random(1, #charTable)]
       end
       return randomString
     end
     
     function http_request(endpoint, authorization_header, port, path, request_body, method)
       local content_type = "application/json"
       if authorization_header ~= nil then
           local headers = { ["Content-Type"] = content_type, ["Authorization"] = authorization_header }
       else
           local headers = { ["Content-Type"] = content_type }
       end
       local request_url = ""
       if port ~= nil then
        	request_url = "https://" .. endpoint .. ":" .. port .. "/" .. path
       else
        	request_url = "https://" .. endpoint .. "/" .. path
       end
       
       if request_body ~= nil then
         response, err = request { method = method, url = request_url, headers = headers, body = json.encode(request_body) }
       else
         response, err = request { method = method, url = request_url, headers = headers }
       end
     
       return response, err
     end
     
   function run(input)
   	if input.operation == "create" then
           local sobject, err = Sobject { name = input.server }
           if sobject == nil or err ~= nil then
               err = "[DSM SDK]: Database configuration unknown."
               return nil, err
           end
           if sobject.custom_metadata["gateway"] == nil then
               err = "[DSM SDK]: Database configuration incorrect."
               return nil, err
           end
           if sobject.custom_metadata["service_account"] == nil then
               err = "[DSM SDK]: Database configuration incorrect."
               return nil, err
           end
           local gateway = sobject.custom_metadata["gateway"]
           local port = sobject.custom_metadata["port"]
           local username = sobject.custom_metadata["service_account"]
           local dbtype = sobject.custom_metadata["dbtype"]
           local password = sobject:export().value:bytes()
       	local time = Time.now_insecure()
           local new_client = "dbclient_" .. time:unix_epoch_seconds()
           if input.timeout ~= nil then
               if input.timeout == 0 then
                   -- when 0 is specified, the deactive time is unlimited
                   deactive_time = 0
               else
                   deactive_time = time:unix_epoch_seconds() + input.timeout
               end
           else
               -- default deactive time is 5 minutes
               deactive_time = time:unix_epoch_seconds() + 300
           end
       	AuditLog.log { message = "[I]: Creating new security object: " .. input.server .. " - Name: " .. new_client, severity = 'INFO' }
           local new_secret = generate_secret()
   	    local payload = {
               name    = new_client,
               dbhost  = input.server,
               dbuser  = username,
               dbpass  = password,
               secret  = new_secret,
               timeout = deactive_time
   		}
           local response, err = http_request(gateway, nil, port, dbtype .. "/create", payload, "POST")
           if err == nil then
               if deactive_time == 0 then
                   local new_sobject = assert(Sobject.import { name = new_client, obj_type = "SECRET", custom_metadata = {server = input.server}, value = Blob.from_bytes(new_secret)})
                   local resp_payload = {
                       status   = response["status"],
                       kid      = new_sobject.kid,
                       name     = new_sobject.name,
                       group_id = new_sobject.group_id
                   }
                   return resp_payload
               else
                   local new_sobject = assert(Sobject.import { name = new_client, obj_type = "SECRET", deactivation_date = get_date_from_unix(deactive_time), custom_metadata = {server = input.server}, value = Blob.from_bytes(new_secret)})
                   local resp_payload = {
                       status   = response["status"],
                       kid      = new_sobject.kid,
                       name     = new_sobject.name,
                       group_id = new_sobject.group_id
                   }
                   return resp_payload
               end
           else
               err = "[DBG SDK]: Secrets not created successfully on database."
               return nil, err
           end
       	--return response
       elseif input.operation == "drop" then
           local sobject, err = Sobject { name = input.name }
           if sobject == nil or err ~= nil then
               err = "[DSM SDK]: No known security object named " .. input.name .. "."
               return nil, err
           end
           if sobject.custom_metadata["server"] == nil then
               err = "[DSM DSK]: Security object " .. input.name .. " is not a database secret."
               return nil, err
           end
           local server_object, err = Sobject { name = sobject.custom_metadata["server"] }
           if server_object == nil or err ~= nil then
               err = "[DSM SDK]: Database configuration unknown."
               return nil, err
           end
           if server_object.custom_metadata["gateway"] == nil then
               err = "[DSM SDK]: Database configuration incorrect."
               return nil, err
           end
           if server_object.custom_metadata["service_account"] == nil then
               err = "[DSM SDK]: Database configuration incorrect."
               return nil, err
           end
           local gateway = server_object.custom_metadata["gateway"]
           local port = server_object.custom_metadata["port"]
           local username = server_object.custom_metadata["service_account"]
           local password = server_object:export().value:bytes()
           local dbtype = server_object.custom_metadata["dbtype"]
           local dbhost = sobject.custom_metadata["server"]
           local group_id = sobject.group_id
   	    local payload = {
               name    = input.name,
               dbhost  = dbhost,
               dbuser  = username,
               dbpass  = password,
   		}
           local response, err = http_request(gateway, nil, port, dbtype .. "/drop", payload, "POST")
           if err ~= nil then
               err = "[DBG SDK]: Secrets not dropped successfully on database."
               return nil, err
           else
               assert(sobject:delete())
               AuditLog.log { message = "[I]: Dropped security object: " .. dbhost .. " - Name: " .. input.name, severity = 'INFO' }
               local resp_payload = {
                   status   = response["status"],
                   name     = input.name,
                   group_id = group_id
               }
               return resp_payload
           end
       elseif input.operation == "tokenize" then
           local server_object, err = Sobject { name = input.server }
           if server_object == nil or err ~= nil then
               err = "[DSM SDK]: Database configuration unknown."
               return nil, err
           end
           if server_object.custom_metadata["gateway"] == nil then
               err = "[DSM SDK]: Database configuration incorrect."
               return nil, err
           end
           if server_object.custom_metadata["service_account"] == nil then
               err = "[DSM SDK]: Database configuration incorrect."
               return nil, err
           end
           local gateway = server_object.custom_metadata["gateway"]
           local port = server_object.custom_metadata["port"]
           local username = server_object.custom_metadata["service_account"]
           local password = server_object:export().value:bytes()
           local dbtype = server_object.custom_metadata["dbtype"]
       	local payload = {
               dbhost  = input.server,
               dbname  = input.dbname,
               dbuser  = username,
               dbpass  = password,
               table   = input.table,
               col     = input.col
   		}
           local tok_sobject, err = Sobject { name = input.server .. "-" .. input.dbname .. "." .. input.table .. "." .. input.col }
           if tok_sobject == nil or err ~= nil then
               err = "[DSM SDK]: Tokenization Security Object unknown."
               return nil, err
           end
           local response, err = http_request(gateway, nil, port, dbtype .. "/select", payload, "POST")
   
           if response["status"] == 200 then
               rows = json.decode(response["body"])["dbres"]["rows"]
               repldbres = {}
               if (#rows) > 0 then
                   for _, row in ipairs(rows) do
                       tok_response = tok_sobject:encrypt { plain = Blob.from_bytes(row[input.col]), mode = "FF1", alg = "AES" }
                       local payload = {
                           dbhost  = input.server,
                           dbname  = input.dbname,
                           dbuser  = username,
                           dbpass  = password,
                           table   = input.table,
                           col     = input.col,
                           orig    = row[input.col],
                           repl    = tok_response["cipher"]:bytes()
                       }
                       local response, err = http_request(gateway, nil, port, dbtype .. "/update", payload, "POST")
                       if response["status"] == 201 then
                           repl_table = {}
                           repl_table[input.col] = row[input.col]
                           repl_table[input.col .. "_new"] = tok_response["cipher"]:bytes()
                           table.insert(repldbres, repl_table)
                       else
                           local resp_payload = {
                               status   = response["status"],
                               server   = input.server,
                               table    = response["table"],
                               col      = response["col"]
                           }
                           return resp_payload
                       end
                   end
                   local resp_payload = {
                       status   = response["status"],
                       server   = input.server,
                       table    = response["table"],
                       col      = response["col"],
                       dbres    = json.decode(response["body"])["dbres"]["rows"],
                       repldbres = repldbres
                   }
                   return resp_payload
               else
                   local resp_payload = {
                       status   = response["status"],
                       server   = input.server,
                       table    = response["table"],
                       col      = response["col"],
                       dbres    = json.decode(response["body"])["dbres"]["rows"],
                   }
                   return resp_payload
               end
           else
               local resp_payload = {
                   status   = response["status"],
                   server   = input.server,
                   table    = response["table"],
                   col      = response["col"],
               }
               return resp_payload
           end
       else
           err = "[DSM SDK]: No operation was specified"
         return nil, err
       end
   end
   
   

8. Copy the plugin URL for configuring on BigID.
   Figure 6: Copy plugin URL
9. Import a new Secret security object that contains the name as the IP/hostname of the database and set custom attributes as shown in the screen shots below.
   For more information on importing a Secret security object, refer to User's Guide: Key Lifecycle Management.
   Figure 7: Secret security object
   Figure 8: Custom attributes

4.0 App Configuration–BigID

1. Load the project into BigID.
   1. Make sure the SHOW_CUSTOM_APPS feature flag is enabled in your BigID environment.
      1. Select Administration > Advanced Tools and click the Services Configuration button.
      2. Search for the environment variable SHOW_CUSTOM_APPS and set its value to ‘true’.
   2. Add the App to your BigID environment.
      1. Select Applications Management and click the Add App button.
2. Enter the Application Base URL, http://bigid-fortanix:8083, and click GO.
   Figure 9: Application base URL
   NOTE

   Do not add a trailing slash ('/') to the URL.

3. Configure the following General Parameters:
   Figure 10: General parameters
   • Fortanix Plugin URL: Point it to the BigID plugin URL obtained in Step 8 of Section 3.0.
   • Fortanix API Key (auth): Enter the API Auth key obtained in Step 5 of Section 3.0 that is used to call the Fortanix plugin.
4. Click SAVE.

5.0 App Dependencies

The following are the App dependencies:

• Both BigID and Fortanix DSM should be installed.
• BigID-Fortanix integration App is installed as a separate component (using docker or Kubernetes).
  • The App is installed within the BigID Application Framework.
  • Configure the App to point it to the applicable Fortanix instance.
• The BigID plugin is installed and configured within the Fortanix instance.

6.0 Actions

6.1 Protect Using Fortanix

This action pulls objects of type 'RDB' from the BigID catalog using the provided ‘Catalog Filter‘. If an 'Attribute Filter' is provided, then it extracts any columns that are mapped to the provided BigID attributes or all the columns if no attributes are provided. It then converts this relational database metadata into a universal meta-model and makes it available for download if needed (can be generated using the 'Export JSON' 'on' flag). This universal meta-model is then converted to a specific model understood by the Fortanix plugin and triggers the “Protect” action on Fortanix DSM for each matching column.

Figure 11: Protect using Fortanix

6.1.1 Parameters

7.0 Usage

The App is designed to be used in concert with Fortanix DSM. The one App action may be executed on-demand or scheduled to run on a repeating basis. Dependencies are required to be met within both BigID and Fortanix DSM for the App to function properly and deliver value.

8.0 Network Requirements

The application docker container assumes bi-directional access with the BigID API endpoint. This allows the following:

• BigID can access the App docker container on port 8083 (HTTP).
• The App docker container can access BigID ("BigID Base Address") with an HTTP(S) request. This application also assumes bi-directional access with a Fortanix DSM API endpoint.
• The App docker container can POST requests to and receive responses from the Fortanix API endpoint using HTTPS.

9.0 Data Used By Application

Using the action Protect Using Fortanix, this application:

• Leverages the BigID Data Catalog API to read data for Relational Databases (RDB) type data sources retrieved using the Catalog Filter input parameter:

  https://{BigID Host}/api/v1/data-catalog/tables/?filter={filter input param} AND type=rdb 

  
  The following Data Source elements are returned:
  • Fully qualified data source name
  • Source
  • Container name
  • Object name
• It then filters that data to only include those columns that contain the attribute(s) included in the Attribute Filter input parameter.
  Below are samples of the JSON data models used within the application:

  
  # Sample universal meta-model (contains list of all applicable columns). Used for data in the downloadable file.
        
  [
    {
      "host": "10.2.0.91:5432",
      "database": "identities",
      "databaseType": "rdb-postgresql",
      "schema": "public",
      "table": "identities",
      "column": "email",
      "attribute": "email",
      "categories": [
        "Personal sensitive",
        "Confidential"
      ]
    }
  ]
  
  # Converted Fortanix specific model (payload used to call plugin endpoint for each column)
  
  {
      "operation": "tokenize",
      "server": "10.2.0.91:5432",
      "dbname": "identities",
      "table": "identities",
      "col": "email",
      "schema": "public"
  }
  
  

10.0 Limitations

• Currently the application has been tested only with the ‘POSTGRES’ database, however, it should work for most RDBs.
  • On the BigID side, it supports all RDBs.
  • The Fortanix DSM plugin may need to be tested on RDBs other than ‘POSTGRES’.
• The Fortanix API calls are made using a static API-Key.
  • Future updates are planned to enable the use of service account-based authorization (and also support OAuth if possible).
• At present, data protection is limited to tokenization. Fortanix DSM may offer other protection features and these need to be tested and validated.
• Data-protection actions (calls to Fortanix plugin) are invoked at the column level.
• The App could be updated to protect entire tables or even schemas.

11.0 Best Practices

• Verify if the system requirements are present in both BigID and Fortanix DSM systems.
• Ensure the filter criteria entered into the App’s Protect Using Fortanix action are accurate.
• Set the 'Export JSON' parameter value to 'on' during initial executions. This will provide for a visual inspection of the data that is being sent to Fortanix. This value can remain set to on but is not required.

12.0 Developer Notes

02/02/2022 - v1.0

• The App has been developed using JavaScript/NodeJS.

• The App repo is hosted on BigExchange.

13.0 Disclaimer

• The Fortanix DSM-BigID application captures only data source, table, and column names for those that meet the input filter criteria. This data is then sent to the Fortanix API using POST so that it can then tokenize the corresponding source data.
• No source data is read or maintained within the App, nor sent to Fortanix DSM. Only data pertaining to where the source data resides is captured and shared.