Run the Workflow - Web Interface - Connectors and Scripts

1.0 Introduction

This article describes how to create, manage, and execute the Workflows using Connectors and Scripts in Fortanix Confidential Computing Manager (CCM) using the Workflow RUN button that allows users to start the application job and monitor it.

Using the Workflows menu item on Fortanix CCM user interface (UI), you can access visual mapping of the workflow. This mapping illustrates the interconnection of various components, including data connectors and scripts.

2.0 Execute the Script using Azure Service Principal

2.1 Prerequisites

Ensure that the Compute Clusters are configured in Fortanix Confidential Computing Manager. A compute cluster is a set of nodes that run containerized applications. Compute clusters are used to run Fortanix Confidential Computing Manager workflows. For more information on how to configure a compute cluster in Fortanix CCM and access the cluster, refer to Section 3.0: Configure the Cluster using Azure Service Principal.

2.2 Create a Cluster

Ensure that you have set up a cluster using Azure Service Principal as a worker node in the Fortanix CCM. For more information, refer to Azure Service Principal with Fortanix Confidential Computing Manager.

3.0 Configure the Cluster using Azure Service Principal

Perform the following steps to configure the Azure Service Principal credentials in Fortanix CCM:

1. Click the Infrastructure → Compute Clusters menu item in the Fortanix CCM UI left navigation panel and click the + ADD COMPUTE CLUSTER button to configure a new compute cluster.

   

2. On the Add Cluster page, enter the following details:

   • Name: Enter a required name of the cluster.

   • Description: Enter the required details about the cluster. However, this is optional.

   • Type: Select the ACI via Service Principal option from the drop down menu.

     

3. After you select the ACI via Service Principal option, rest of the parameters appear on the screen.

   • Location: The Azure region where the deployment occurs . If the required location is not available in the provided list, select the Other option and manually enter the specific location.

   • ACI configuration:

     • App ID: The Azure Active Directory application ID or client ID used for application identification.

     • App Passcode: The application secret required for authentication.

     • Tenant ID: The unique identifier of the Azure Active Directory instance, known as the Directory ID.

     • Subscription: The subscription ID where all resources are managed; it contains information related to resources.

     • Resource Group: The resource group designated for managing all containers and deployments.

       

       

4. Click the ADD CLUSTER button to save the cluster configuration.

   

The compute cluster is now successfully created.

4.0 Creating a Workflow

In this section, you will establish connections among the Inbound Connector, Scripts, and Outbound Connectors to formulate a comprehensive workflow. Within this workflow, the Inbound Connector is utilized to access the input data, which is then processed using an SQL query or Python within the Script. The resultant output is then generated and made accessible in the designated location specified by the Outbound Connector. This interconnection facilitates the seamless execution of data processing and transmission operations within Fortanix Confidential Computing Manager.

Perform the following steps to create a workflow:

1. Click the Workflows menu item in the Confidential Computing Manager UI left navigation panel.

2. On the Workflows page, click + WORKFLOW to create a new workflow.

3. In the CREATE NEW WORKFLOW dialog box, enter the following details:

   • Name: Enter a required a name for the workflow.

   • Group: Select the required group name from the drop down menu to associate the workflow with that Group.

   • Description (Optional): Enter the required short description for the workflow.

     

4. Click the CREATE WORKFLOW button to add a new workflow.

The workflow is added to the Fortanix Confidential Computing Manager application successfully.

4.1 Add Scripts and Connectors to the Workflow

Perform the following steps to create the workflow:

1. Adding an Inbound Connector:

   1. Drag the Inbound Connector icon and drop it into the working area. Click the ADD INBOUND CONNECTOR node.

   2. In the INBOUND CONNECTOR dialog box, you can either create a new inbound connector or select an existing inbound connector name. For more information on how to create a new inbound connector, refer to Inbound Connectors.

   3. Click the ADD INBOUND CONNECTOR button to add an inbound connector or select an existing one.

2. Adding an Outbound Connector:

   1. Drag the Outbound Connector icon and drop it into the working area. Click the ADD OUTBOUND CONNECTOR node.

   2. In the OUTBOUND CONNECTOR dialog box, you can either create a new outbound connector or select an existing outbound connector name. For more information on how to create a new outbound connector, refer to Outbound Connectors.

   3. Click the ADD OUTBOUND CONNECTOR button to add an outbound connector or select an existing one.

3. Adding a Script:

   1. Drag the Script icon and drop it into the working area. Click the ADD SCRIPT node.

   2. In the SCRIPT dialog box, you can either create a new script or select an existing outbound connector name. For more information on how to add a new script, refer to Scripts.

   3. Click the ADD SCRIPT button to add a script or select an existing one.

4. Establish the connections:
   Connect the inbound connectors to scripts written in either SQL, Python, or SQL Aggregate. Then connect these scripts to the SQL Aggregate scripts. Next, establish connections between these aggregate scripts with one or more outbound connectors (s) to facilitate the output.
   It is deemed invalid to have unattached Confidential Computing Manager nodes, except when they are part of a draft.

   NOTE

   Ensure the following:

   • The SQL nodes must have at least one connection originating from either an inbound data connector or a Python script.

   • The SQL Aggregate nodes must have exactly one connection originating from either a SQL node, Python node, or an inbound data connector.

   • The Python scripts must have at least one connection originating from either an inbound data connector or a SQL/Python script.

   • The Python scripts must have at least one connection leading to any script (Python/SQL/SQL aggregate) or outbound connector.

   • Only one Python script per workflow is supported for this release, along with one SQL script per workflow.

   • The Outbound Connectors must have at least one connection originating from:

     • If there is exactly one incoming connection, it may be from a SQL, Python, or SQL Aggregate node.

     • If there are multiple incoming connections, they must all be SQL Aggregate nodes.

   • All other types of inbound connections are not permitted.

   • Avoid disconnected components:

     • The Inbound Connectors, Script nodes, and Outbound Connectors must be connected to another node.

     • It is not allowed to have unattached Windsor nodes, except as part of a draft.

   • There should be no connections between legacy nodes and Windsor nodes.

   • Connections such as App to Data Connector and Dataset to Data Connector or script are not allowed.

   

5.0 Requesting the Workflow Graph Approval

After the workflow is complete, click the REQUEST APPROVAL button to initiate the approval process for the Workflow.

Perform the following steps to accept the workflow request:

1. The workflow remains in a pending state until it receives approval from all users. In the Pending menu item, click the SHOW APPROVAL REQUEST button to approve a workflow.

   

2. In the APPROVAL REQUEST – CREATE WORKFLOW dialog, you can either APPROVE or DECLINE a workflow.

   NOTE

   • A user can also approve/decline a workflow from the Fortanix Confidential Computing Manager Tasks tab.

   • The users who have approved the workflow display a green tick against their icon.

3. Approval from at least 1 admin user is necessary to finalize the workflow. If a user declines, the workflow is rejected. When all the users approve the workflow, it is deployed.

   1. Fortanix Confidential Computing Manager configures apps to access the Datasets.

   2. Fortanix Confidential Computing Manager creates the Workflow Application Configs.

   3. Fortanix Confidential Computing Manager returns the list of hashes required to start the apps.

The workflow approval is now requested successfully.

6.0 Configuring the Workflow Graph

Perform the following steps to configure the workflow:

1. Navigate to the Workflows → Approved menu item in the Fortanix Confidential Computing Manager UI left navigation panel.

2. From the list of approved workflows, select a workflow that has a single application since Fortanix Confidential Computing Manager supports only single job deployments.

3. In the detailed view of the selected workflow, you will notice the disabled RUN button. The RUN button will be disabled if you have not configured the Azure account and Location. Click the icon to configure these details and enable the RUN button.

4. In the RUN WORKFLOW window, enter the following details:

   • Deployment Type: The workflow deployment type. Select the Azure Confidential Instances (Single Job) option from the drop down menu.

   • Azure account: Select the ACI cluster (Test Cluster) created in Section 3.0: Configure the Cluster using Azure Service Principal option from the drop down menu.

   • Location: The Azure region where the deployment occurs.

     

5. Click the SAVE CONFIGURATION button to save the changes.

The RUN button is now enabled on the screen.

7.0 Running the Workflow

Ensure that you have created an image registry in the Fortanix Confidential Computing Manager UI. For more information, refer to Image Registry. A registry for the image used in the ACI application workflow is required to be created in the Fortanix Confidential Computing Manager account, so at runtime, the credentials are passed to the Azure container instance to pull the image.

Perform the following steps to run the ACI workflow application:

1. Configure the image pull secret.

2. Click the RUN button in the detailed view of an approved workflow enabled in Section 4.0 : Configuring the Workflow Graph.

   

3. In the RUN WORKFLOW window, confirm the values of each parameter and click the RUN button to run the workflow.
   Observe the running indicator at the bottom of the workflow.

   

   NOTE

   The workflow execution status is not updated in real-time and must be fetched from the cluster manually. Therefore, click the Refresh icon to get the latest execution status.

   If there is a need to halt the execution at any point, click the STOP button. This action will re-enable the RUN button.

4. If the application is executed successfully, the execution status will be displayed under the Execution Log. Click the View detail link to view the log details.
   The EXECUTION LOG window provides a detailed log of the run. You can also download the log using the DOWNLOAD button.

   

   NOTE

   Attempting to execute a workflow containing more than one application will result in the mentioned error. The Fortanix Confidential Computing Manager supports the execution of workflows with a single application only.

Additionally, users can access the output CSV file to verify the output data.