User's Guide: Configure and Run the Workflow

1.0 Introduction

Welcome to the Fortanix Confidential Computing Manager - Workflows user guide. This guide describes how to create, manage, and execute the Workflows in Fortanix Confidential Computing Manager.
Workflows play a crucial role in orchestrating the flow of data processing within the Fortanix Confidential Computing Manager environment. They act as collaborative entities where multiple users can contribute their respective objects and approvals, creating a streamlined and organized process for managing data.

Using the Workflows menu item on Fortanix Confidential Computing Manager User Interface (UI), you can access visual mapping of the workflow. This mapping illustrates the interconnection of various components, including data connectors and scripts.

The workflow in Fortanix Confidential Computing Manager defines the sequence of actions, including draft, pending, and final:

• Draft Workflows: These are in-progress works that lack approval and do not grant permissions to applications. They are still under development and are not ready for deployment.

• Pending Workflows: These represents intermediate stages where the workflow has been submitted for approval but has not received unanimous approval. During this phase, applications do not have access to datasets.

• Final Workflows: These represents versioned and protected by quorum approval. These workflows grant applications access to datasets upon receiving certificates that confirm compliance with the approved workflow. After they are approved, a final workflow can be deployed, enabling applications to securely interact with the specified data connectors and scripts.

Transitioning from a draft workflow to a final workflow necessitates approvals, which involve:

• Fortanix Confidential Computing Manager Account Administrator inviting other users to join the account.

• Users joining the account and contributing data in the form of datasets and applications/application configurations.

2.0 Creating a Workflow Graph

In this section, you will establish connections among the Inbound Connector, Scripts, and Outbound Connectors to formulate a comprehensive workflow. Within this workflow, the Inbound Connector is utilized to access the input data, which is then processed using an SQL query or Python within the Script. The resultant output is then generated and made accessible in the designated location specified by the Outbound Connector. This interconnection facilitates the seamless execution of data processing and transmission operations within Fortanix Confidential Computing Manager.

Perform the following steps to create a workflow:

1. Click the Workflows menu item in the Confidential Computing Manager UI left navigation bar.

2. On the Workflows page, click + WORKFLOW to create a new workflow.

3. In the CREATE NEW WORKFLOW dialog box, enter the following details:

   • Name: Enter a required a name for the workflow.

   • Group: Select the required group name from the drop down menu to associate the workflow with that Group.

   • Description (Optional): Enter the required short description for the workflow.

     

4. Click the CREATE WORKFLOW button to add a new workflow.

The workflow is added to the Fortanix Confidential Computing Manager application successfully.

2.1 Configuring a Workflow Graph

Perform the following steps to creating the workflow:

1. Adding an Inbound Connector:

   1. Drag the Inbound Connector icon and drop it into the working area. Click the ADD INBOUND CONNECTOR node.

   2. In the INBOUND CONNECTOR dialog box, you can either create a new inbound connector or select an existing inbound connector name. To know the detailed steps for creating a new inbound connector, refer to the User's Guide: Inbound Connectors guide.

   3. Click the ADD INBOUND CONNECTOR button to add an inbound connector or select an existing one.

2. Adding an Outbound Connector:

   1. Drag the Outbound Connector icon and drop it into the working area. Click the ADD OUTBOUND CONNECTOR node.

   2. In the OUTBOUND CONNECTOR dialog box, you can either create a new outbound connector or select an existing outbound connector name. To know the detailed steps for creating a new outbound connector, refer to the User's Guide: Outbound Connectors guide.

   3. Click the ADD OUTBOUND CONNECTOR button to add an outbound connector or select an existing one.

3. Adding a Script:

   1. Drag the Script icon and drop it into the working area. Click the ADD SCRIPT node.

   2. In the SCRIPT dialog box, you can either create a new script or select an existing outbound connector name. To know the detailed steps for adding a new script, refer to the User's Guide: Scripts guide.

   3. Click the ADD SCRIPT button to add a script or select an existing one.

4. Establish the connections:
   Connect the inbound connectors to scripts written in either SQL, Python or SQL Aggregate. Then connect these scripts to the SQL Aggregate scripts. Next, establish connections between these aggregate scripts with one more outbound connector(s) to facilitate the output.
   It is deemed invalid to have unattached Confidential Computing Manager nodes, except when they are part of a draft.

   NOTE

   Ensure the following:

   • The SQL nodes must have at least one connection originating from either an inbound data connector or a Python script.

   • The SQL Aggregate nodes must have exactly one connection originating from either a SQL node, Python node, or an inbound data connector.

   • The Python scripts must have at least one connection originating from either an inbound data connector or a SQL/Python script.

   • The Python scripts must have at least one connection leading to any script (Python/SQL/SQL aggregate) or outbound connector.

   • Only one Python script per workflow is supported for this release, along with one SQL script per workflow.

   • The Outbound Connectors must have at least one connection originating from:

     • If there is exactly one incoming connection, it may be from a SQL, Python, or SQL Aggregate node.

     • If there are multiple incoming connections, they must all be SQL Aggregate nodes.

   • All other types of inbound connections are not permitted.

   • Avoid disconnected components:

     • The Inbound Connectors, Script nodes, and Outbound Connectors must be connected to another node.

     • It is not allowed to have unattached Windsor nodes, except as part of a draft.

   • There should be no connections between legacy nodes and Windsor nodes.

   • Connections such as App to Data Connector and Dataset to Data Connector or script are not allowed.

   

3.0 Requesting the Workflow Graph Approval

After the workflow is complete, click the REQUEST APPROVAL button to initiate the approval process for the Workflow.

Perform the following steps to accept the workflow request:

1. The workflow remains in a pending state until it receives approval from all users. In the Pending menu item, click the SHOW APPROVAL REQUEST button to approve a workflow.

   

2. In the APPROVAL REQUEST – CREATE WORKFLOW dialog, you can either APPROVE or DECLINE a workflow.

   NOTE

   • A user can also approve/decline a workflow from the Fortanix Confidential Computing Manager Tasks tab.

   • The users who have approved the workflow display a green tick against their icon.

3. Approval from at least 1 admin user is necessary to finalize the workflow. If a user declines, the workflow is rejected. When all the users approve the workflow, it is deployed.

   1. Fortanix Confidential Computing Manager configures apps to access the Datasets.

   2. Fortanix Confidential Computing Manager creates the Workflow Application Configs.

   3. Fortanix Confidential Computing Manager returns the list of hashes required to start the apps.

The workflow approval is now requested successfully .

4.0 Editing the Workflow Graph

Perform the following steps to edit a workflow:

1. In the Approved menu item, click the  overflow menu for a workflow. Select the EDIT WORKFLOW option to modify the workflow.
   When a workflow is edited, it generates a new version of the workflow in the Drafts section for editing while preserving the existing one. For example, if you edit the initial version (Version 1) of an approved workflow named "Workflow 1.0," a new version (Version 2) of "Workflow 1.0" is created.

2. Update the workflow with the required changes and click the REQUEST APPROVAL button to submit the edited workflow for approval.

3. The system generates a new version (Version 2) of the workflow in the Pending state. Click the SHOW APPROVAL REQUEST button to approve this edited version.

4. Click the APPROVE button to accept the edited workflow.
   After approving Workflow Version 2, it becomes linked to Version 1. Now, you can either delete Workflow Version 1 or restore it.

   

The workflow is edited successfully.

5.0 Cloning the Workflow Graph

You must clone a workflow when you want to create a copy of an existing workflow instead of building it from scratch.

Perform the followings steps to create a workflow:

1. For an approved or draft workflow, click the    overflow menu and select the CLONE WORKFLOW option to replicate the workflow.
   When a workflow is cloned, the new workflow is created with a modified name. For example, if the approved workflow “Workflow 1.0” is cloned, a new workflow “Workflow 1.0 (clone)” is created. The user can modify the workflow name using the. Edit icon next to the workflow name.

2. Update the workflow with the required changes and click the REQUEST APPROVAL button to submit the workflow for approval.

A new workflow is created in the Pending state successfully.

6.0 Deleting a Workflow Graph

Perform the following steps to delete a workflow:

1. For an approved workflow, click the overflow menu on the right and select the DELETE THIS VERSION option to remove the workflow.

2. In the DELETE WORKFLOW dialog box, click the DELETE button to confirm the action.

   NOTE

   This action will create an approval request for deleting the workflow with the quorum. Also, the workflow will be moved to Pending state and you can no longer directly edit the workflow or withdraw the request.

The workflow is deleted successfully.

7.0 Configuring the Workflow Graph

Perform the following steps to configure the workflow:

1. Navigate to the Workflows → Approved menu item in the Fortanix Confidential Computing Manager UI left navigation bar.

2. From the list of approved workflows, select a workflow that has a single application since Fortanix Confidential Computing Manager supports only single job deployments.

3. In the detailed view of the selected workflow, you will notice the disabled RUN button. The RUN button will be disabled if you have not configured the Azure account and Location. Click the icon to configure these details and enable the RUN button.

4. In the RUN WORKFLOW window, enter the following details:

   • Deployment Type: The workflow deployment type. Select the Azure Confidential Instances (Single Job) option from the drop down menu.

   • Azure account: Select the ACI cluster option from the drop down menu.

   • Location: The Azure region where the deployment occurs.

     

5. Click the SAVE CONFIGURATION button to save the changes.

The RUN button is now enabled on the screen.

8.0 Running the ACI Application Workflow Graph

Ensure that you have created an image registry in the Fortanix Confidential Computing Manager UI. For more information, refer to User's Guide: Image Registry. A registry for the image used in the ACI application workflow is required to be created in the Fortanix Confidential Computing Manager account, so at runtime, the credentials are passed to the Azure container instance to pull the image.

Perform the following steps to run the ACI workflow application:

1. Configure the image pull secret.

2. Click the RUN button in the detailed view of an approved workflow enabled in the Section 7.0 : Configuring the ACI Application Workflow.

   

3. In the RUN WORKFLOW window, confirm the values of each parameter and click the RUN button to run the workflow.
   Observe the running indicator at the bottom of the workflow.

   

   NOTE

   The workflow execution status is not updated in real-time and must be fetched from the cluster manually. Therefore, click the Refresh icon to get the latest execution status.

   If there is a need to halt the execution at any point, click the STOP button. This action will re-enable the RUN button.

4. If the application is executed successfully, the execution status will be displayed under the Execution Log. Click the View detail link to view the log details.
   The EXECUTION LOG window provides a detailed log of the run. You can also download the log using the DOWNLOAD button.

   

   NOTE

   Attempting to execute a workflow containing more than one application will result in the mentioned error. The Fortanix Confidential Computing Manager supports the execution of workflows with a single application only.

Additionally, users can access the output CSV file to verify the output data.