Custom process apps

Overview

For any business enterprise, customized processes and data structures play a huge role in their day-to-day operations. Therefore, you require customized tools and planned processes to extract data from the enterprise system and transform and load them into the process mining applications. The custom process app feature in IBM Process Mining supports the creation of the custom tools that help to automate the ETL process for IBM Process Mining. The custom process apps provide the flexibility to edit and configure the ETL tool package per your business process requirement. In addition, you can share the process app with the stakeholders of the business process for continuous use. You can use the data output of the process apps as a data source for creating a project in IBM Process Mining.

The custom process app feature is compatible with all enterprise applications. You can edit and update the scripts and configuration in a custom process app per the data structures and requirements of the enterprise system.

To learn more about the use case of the process app in a Salesforce environment, refer to the Process App use case document.

To learn more about the tasks that you can perform in the custom process app feature, see the following topics:

Understanding types of users and permissions in custom process app

Understanding the permissions that you can give to the custom process app users helps you to manage and control the use of the custom process app in your enterprise system. The custom process app feature has the following types of users:

  • Creator

    Creators are the users who create a custom process app](#creating-a-custom-process-app). The creator has the privilege to set up the permissions for general users and importers. To learn more about the permissions of a creator, see the Permissions of creator topic.

  • General user

    General user uses the custom process app that is created by a creator to generate a process in IBM Process Mining. The creator defines the permissions of the general users. To learn more about the permissions of a general user, see the Permissions of general user topic.

  • Importer

    Importer is the user who imports a custom process app from an external or same IBM Process Mining instance. The permissions of the importer depend on the permission configuration set up by the creator while exporting a custom process app. To learn more about the permissions of an importer, see the Permissions of importer topic.

Permissions of creator

The creators have the full privilege on the custom process app that they create. 'Table 1. Creator permissions in a custom process app' defines the different permissions of the creator.

Permission type Action permission
Closed source
  • Use: Use the custom process app to generate a process in Process Mining.
  • Edit: Edit the custom process app including the Python script and configurations in the process app.
  • Export: Export the custom process app with or without the Edit permission for the importer of the custom process app in an IBM Process Mining instance.

    ⓘ Tip: See the Permissions of importer topic to learn more about the permissions of an importer.

  • Duplicate: Duplicate the custom process app.
  • Delete: Delete the custom process app.
Open source
  • Use: Use the custom process app to generate a process in Process Mining.
  • Edit: Edit the custom process app including the Python script and configurations in the custom process app.
  • Export: Export the custom process app with the Edit permission for importer of the custom process app in an IBM Process Mining instance.
  • ⓘ Tip: See the Permissions of importer topic to learn more about the permissions of an importer.

  • Duplicate: Duplicate the custom process app.
  • Delete: Delete the custom process app.

Table 1. Creator permissions in custom process app


ⓘ Tip: When a creator exports a custom process app and then imports it in an IBM Process Mining instance, the creator's role is changed to importer of the custom process app that is imported to the IBM Process Mining instance.


Permissions of general user

The permissions of a general user are defined by the Closed source and Open source tags given by the creator while configuring the permissions for the custom process app. 'Table 2. General user permissions in a custom process app' defines the permissions of a general user.

Permission type Action permission
Closed source
  • Use: Use the custom process app to generate a process in Process Mining.
  • Export: Export the custom process app with only the Use permission.

    ⓘ Tip: See the Permissions of importer topic to learn more about the permissions of an importer.

Open source
  • Use: Use the custom process app to generate a process in Process Mining.
  • Read-only: Read only the details of the custom process app, the python script and configurations.
  • Export: Export the custom process app with the full permissions given to the importer.

    ⓘ Tip: See the Permissions of importer topic to learn more about the permissions of an importer.

  • Duplicate: Duplicate the custom process app with the full permissions configured by the creator.

Table 2. General user permissions in custom process app

Permissions of importer

The permissions of an importer are defined by the permissions that are configured by the creator before exporting a custom process app. 'Table 3. Importer permissions in a custom process app' defines the permissions of an importer.

Permission type Edit status Action permission
Closed source Editable
  • Use: Use the custom process app to generate a process in Process Mining.
  • Edit: Edit the custom process app including the Python script and configurations in the custom process app.
  • Export: Export the custom process app with the Edit permission.
  • Delete: Delete the custom process app.
Uneditable
  • Use: Use the custom process app to generate a process in Process Mining.
  • Edit name: Only edit the name of the custom process app
  • Export: Export the custom process app with only the Use permission.
  • Delete: Delete the custom process app.
Open source
  • Use: Use the custom process app to generate a process in Process Mining.
  • Edit: Edit the custom process app including the Python script and configurations in the custom process app.
  • Export: Export the custom process app with the Edit permission for importer of the custom process app in an IBM Process Mining instance.
  • Duplicate: Duplicate the custom process app.
  • Delete: Delete the custom process app.

Table 3. Importer permissions in custom process app

Creating a custom process app

The creation of a custom process app includes the following activities:

Accessing Custom process app wizard

To create a custom process app, you must access the "Custom process app" wizard. Do the following steps to go to the "Custom process app" wizard:

  1. In the Processes dashboard, click the Process App tile to go to the Process App dashboard.

    Process App

  2. In the Process Apps dashboard, click the Create process app button on the upper right of the window to see the "Custom Process App" wizard.

    Add new custom process app

Creating a Python logic file

While configuring a custom process app, you must write and upload a Python logic file. The Python logic file defines and transforms the data from your enterprise application, which becomes the source of data for your process mining project. You must have the following key structures in your Python logic file:

Entry points

The entry point or the entry function is a dedicated function in the logic file that serves as the entry to the data transformation process.

   def execute(context):

ⓘ Tips: Without a defined entry point, the data transformation fails even before the process begins.


Configurable variables

The logic file requires configurable variables to compute the data transformation process. The configurable variables include dynamic variables such as token key, server or systemURL, username, and password. You can set the configurable variables in the following ways:

  • Getting configurable variable as environment variables

    You can access and retrieve the configurable variables from an environment variable within the logic file. In other words, you must set each configurable variable from an environment variable.

    For example, if you want to configure username as a configurable variable within the logic file, you must use the following command:

       username = os.environ["username"]
    

    ⓘ Tips: The configurable variable is case-sensitive. Therefore, you must ensure consistency of letter cases while writing the Python file. For example, if you define username in lowercase, you must use the same letter case, username, elsewhere in the logic file.


  • Getting the configuration variables from the context object

    The configuration variables are also available with a Python dictionary that you can retrieve by using the context.config syntax. In the following example, you define the execute function as a context parameter:

       def execute(context): 
          config = context[“config”] 
          # example retrieving an input property named ‘URL’ 
          url = config[“URL”]
    

ⓘ Tips:

  1. In the custom process app, you must configure the input variables that is given in the logic file to establish connection between the logic file and the process data. For more information about defining user input variable, see the Define the user inputs topic.

  2. If the raw data source is in .CSV format, you can configure the Python logic file to transform the data to be the source for your process mining project. Before uploading the data, the process app user must transform the file from .CSV format to .zip format. The Python logic file accesses the .zip file to perform the necessary data transformation in the same directory location. The logic file can also retrieve the name of the uploaded .zip file either from an environment variable or from the context JSON that is passed into the execute function of the logic file. The name of the key for the uploaded logic file name is fileUploadName and the value is the name of the uploaded .zip file.

    For example, if the name of the uploaded .zip file is “Input_sample.zip”, the logic file can retrieve the name of the .zip file by using any one of the following commands:

       myFileUploadName = os.environ["fileUploadName"]
    

    Or

       def execute(context): 
          # Example retrieving the name of an uploaded zip file 
          myFileUploadName = context[“fileUploadName”] 
    

    For more information about uploading the raw data, see the Define the user inputs topic.


Output data format

The entry point, which you define in the logic file, returns the output of the data transformation process. The output format must be a python DataFrame object, which is a data structure to manipulate and store data in Python. The output of the data transformation process in the logic file must be either Pandas or Polars, which are the two DataFrame modules. In the following example, you configure the DataFrame object:

   def output(event_list): 
      return pd.DataFrame(event_list) 
   def execute(context): 
      extract_and_transform(context) 
      return output(event_list)

ⓘ Remember: In the example, you need to change the functions per your requirements for the logic file.


Supported Python libraries

In a custom process app, you can use the Python libraries in the logic file to transform the data:

Python libraries Version
jsonschema 4.17.3
numpy 1.24.1
pandas 1.5.3
polars 0.16.8
PyGithub 1.57
python-dateutil 2.8.2
python-dotenv 0.21.1
pythonhub 0.3.1
requests 2.28.2
types-requests 2.28.11.8
types-urllib3 1.26.25.4
typing_extensions 4.4.0
urllib3 1.26.14
ibm-cos-sdk 2.12.2
ibm-cos-sdk-core 2.12.2
ibm-cos-sdk-s3transfer 2.12.2
oracledb 1.2.2
gitdb 4.0.9
GitPython 3.1.34
confluent-kafka 2.0.2
pymongo 4.3.3
boto3 1.26.100
impyla 0.18.0
elasticsearch 8.6.2
sqlalchemy 2.0.7
google-cloud-bigquery 3.8.0
redis 4.5.4
salesforce-api 0.1.45
servicenow 2.1.0
servicenow-api 0.12.0
simple-salesforce 1.12.3
SOAPpy 0.12.22
splunk-sdk 1.7.3
hbase 1.0.0
neo4j 5.6.0
couchbase 4.1.3
pyexasol 0.25.2
pysolr 3.9.0
pymemcache 4.0.0
pysnc 1.1.4

Handling exceptions in the python code

To inform the process app user about invalid credentials or lost connection to an external system, you can configure the python code to send a dedicated exception named ProcessAppException in the process app user interface. The following code snippet gives an example of ProcessAppException configuration:

   from process_app import ProcessAppException 
   def execute(context): 
      # Example raising a ProcessAppException 
      raise ProcessAppException("cannot connect to the system")



Creating a Summary card for a process app

In the Summary card step, do the following steps:

  1. Enter the name of the process app in the Name field.


    ⓘ Tip: You cannot use the same name of an existing process app for the new process app.


  2. Enter a description about the process app in the Brief description field.


    ⓘ Tip: The maximum number of characters that you can type in the Brief description field is 120.


  3. Switch the toggle to Closed source or Open source to set the permissions for the process app users. You can choose one of the following options per your requirement:

    • Switch the toggle to Open source if you want to give the Use, View, Export, and Duplicate permissions to the process app users.

    • Switch the toggle to Closed source if you want to give only the Use and Export permissions to the process app users.

  4. If you want to customize the icon and add feature highlights and documentation link on the summary card, expand the Customization section.

    Customize Summary Card


    ⓘ Tip: The Customization section is optional. Therefore, if you want to skip customizing the icon and adding the feature highlights and documentation, you can click the Next button.


  5. You can change the summary card icon by selecting an icon from the pack of icons in the Icon section. You can also upload your own icon by clicking the Add icon (Add icon).

    Change icon


    ⓘ Tips:

    • The maximum file size that you can upload is 200 KB.

    • You can upload only the SVG, JPG, JPEG, or PNG format.


  6. If you want to display the feature highlights of your process app on the summary card, add them in the Feature highlights field in the Feature highlights and documentation section.

    Feature Highlights


    ⓘ Tip: You can add maximum three feature highlights for your process app.


  7. If you want to display the documentation URL link for your process app on the summary card, add them in the Link to external documentation field in the Documentation section.

    Documentation link


    ⓘ Tip: You can add only one documentation URL link for your process app in the summary card.


  8. Click the Next button to go to the Upload your Process App logic step.

Uploading your Process App logic

By uploading a process app logic in the "Custom Process App" wizard, you define the task of the process app. The process app logic files are customized Python scripts that connect your data sets and process logic to extract and transform the required data from the source application. You can use the data output from the execution of the Python script to generate a process in IBM Process Mining. In addition, you can reuse the process app logic file and apply it in multiple processes.

In the Upload your Process App logic step, click the Drag and drop file here to click to upload link to select and upload the python script from your system. After uploading the Python script, click the Next button to go to the Define user inputs step in the "Custom process app" wizard. (See Video 1. Uploading your Process App logic)

To learn more about structuring your Python script, you can download the template document that is provided in the Upload your Process App logic step by clicking the Download template link.


ⓘ Important: You must validate and verify the Python script for accuracy before uploading the Python file. If there is any exception or error message while running the Process App logic, you can view the error status in the Generating your process stage.


Process App Logic file upload

Video 1. Uploading your Process App logic

Defining the user inputs

In the "Custom process app" wizard, you can configure the options for the users to input data that the Python script must transform for process generation. It helps you to provide the process app users the flexibility to select the data that they want to analyze in the event logs of the business process. You can configure the input options in the following ways:

  • Allow the users to upload a data file to transform into a valid data source for process mining.

  • Allow the users to define the individual parameters for the Python script to generate a data source based on their business requirement.


ⓘ Remember: The configuration that you set up in the Define user inputs step determines the availability of the Load your data and Configure log events steps in the process app during the process generation. For example, if you set the File upload required switch to Required, the Load your data step is made visible for the process app users during the process generation. Similarly, if you add the Custom input field, the Configure log events step is made visible for the process app users during the process generation.


If you want to allow the users to define the user input, do the following steps in the Define user inputs (optional) step:

  1. In the File upload section, set the File upload required switch to Required if you want to allow the users of the process app to upload a file with raw data that needs to be transformed.

    When you set the File upload required switch to Required, the Load your data step is made visible for the process app users during the process generation.

    If the file upload is not mandatory for the process app user, set the File upload required switch to Not required. When you set the File upload required switch to Not required, the Load your data step is hidden for the process app users during the process generation.

    User file upload


    ⓘ Tip: If you want to configure the Python logic file to transform data as the source for your process mining project, you must compress the raw data to create a .zip file before you upload. For more information, see the Tips in the Configurable variables topic.


  2. In the Custom input fields section, expand User input 1 to see the fields to define the user inputs.

    Custom user input

  3. Set the input required switch to Required if the input field is mandatory for the users. When you set the Custom input field to Required, the fields in the Configure log events step is made mandatory for the process app users during the process generation.

    If the input field is not mandatory for the users, set the input required switch to Not required. When you set the Custom input field to Not required, the Configure log events step is optional for the process app users during the process generation.

  4. Define the following input fields for the users of the process app:

    • Label field

      The field to define the label on the user interface for the input field.

    • Variable name field

      The field to define the name of the variable that the user must enter based on the logic of the Python script. For example, Enter the Password

    • Input type

      The field to define the input type of the input field, which can be a String, Dropdown, or Password.

      ⓘ Tip:

      • If you want to include multiple input fields for the process app users, click the Add custom input button to add more custom input fields.

      • If you add the Custom input field, the Configure log events step is made visible for the process app users during the process generation. If you do not add the Custom input field, the Configure log events step is hidden for the process app users during the process generation.

  5. Click the Next button to go to the Provide user guidance step.

Providing the user guidance for the process app

In the "Custom process app" wizard, you can create user guides for the custom process app. The Provide user guidance step is optional.


ⓘ Remember: The configuration that you set up in the Provide user guidance step determines the availability of the Documentation step in the process app during the process generation. For example, if you do not set up the Provide user guidance step with the documentation information, the Documentaion step is hidden for the process app users during the process generation.


To create the user guide for the process app, do the following steps in the Provide user guidance step:

  1. In the Guidance text section, enter the heading of the text in the Text heading field.

  2. Enter the main body of the user guidance text in the User guidance body field (maximum 2000 characters).


    ⓘ Tip: You can add multiple description block by clicking the Add description block link.


  3. In the Documentation and other links section, you can add the documentation URL link in the URL field and the label name of the link in the Link label field.


    ⓘ Tip: You can add multiple documentation link by clicking Add another link.


  4. Click the Next button to go to the Process settings step.

Setting up a process for the process app

In the Process setting step, you can upload a process back up file in the .idp format or duplicate settings from an existing process in IBM Process Mining. The process app uses the mapped data columns, the process settings, filters, and dashboards from the process backup file to generate a new process.

To upload a process back file, do the following steps:

  1. Select the Upload process backup file option in the Process setting step.

  2. Click the Drag and drop file here or click to upload link to upload the process backup file.

  3. Click the Save button.

To duplicate the settings of an existing process, do the following steps:

  1. Select the Duplicate settings from a process option in the Process setting step.

  2. Select a process from the Find a process dropdown.

  3. Click the Create button.

Editing a custom process app

To edit a process app, do the following steps:

  1. Click the menu icon on the upper right of the Process App tile to view the menu options.

  2. Click the Edit Process App option to view the "Edit Process App" wizard.


    ⓘ Tip: Only the process app creators can edit their process app in the same IBM Process Mining instance.


  3. In the Edit process app wizard, follow the steps in the Creating a custom process app topic to complete your edits.

Importing a custom process app

In your IBM Process Mining instance, you can import a custom process app from another IBM Process Mining instance. Importing of a process app helps you to reuse and apply an existing process app logic to your process. Before you import a custom process app, you must ensure that you have already downloaded the process app file in the .pma format from the source IBM Process Mining instance. To import a custom process app, do the following steps:

  1. In the Process Apps page, click the Import process app button to see the Import Process App from file dialog.

    Import Process App

  2. In the Import Process App from file dialog, drag or upload the file in the Upload files section.

    Upload Process App Import file


    ⓘ Tip:

    • You can import only the process app file in the .pma format.

    • When you import a process app, the name of the new process app gets the suffix, Copy-<timestamp-id>, along with the original process app name.

    • If you want to import a process app that is created in IBM Process Mining 1.14.0 to IBM Process Mining 1.14.1, you must do the following steps:

      1. Upgrade the IBM Process Mining instance, from which you want to export the process app file, to version 1.14.1.

      2. Export the process app file from the upgraded IBM Process Mining instance.

      3. Import the process app file in the target IBM Process Mining instance of version 1.14.1.

    • For more information about permissions of the importer per different permission types that are configured for the process app, see the Permissions of importer topic.


Exporting a custom process app

You can export a process app to save, reuse, and share the process app configuration. To export a process app, do the following steps:

  1. Click the menu icon on the upper right of the Process App tile to view the menu options.

    Export Process App

  2. Click the Export option to run one of the following actions per your existing configuration:

    • If the process app is Open source, the process app is downloaded in the .pma format.

    • If you are the creator of a Closed source process app, you see the How would you like to export the process app? dialog. On the How would you like to export the process app? dialog, switch the toggle to Editable or Uneditable per your requirement and click the Export button to download the process app.


      ⓘ Tips:

      • If you switch the toggle to Editable, the importers get the Use, Edit, Export, and Delete permissions.

      • If you switch the toggle to Uneditable, the importers get the Use, Edit name, Export, and Delete permissions. However, the importers cannot change the Python script for extracting and transforming the data.


    • If you are a general user of a Closed source process app, you see the Warning dialog. On the Warning dialog, click the Yes button to download the process app.


ⓘ Tips:


Duplicating a custom process app

To duplicate a process app, do the following steps:

  1. Click the menu icon on the upper right of the Process App tile to view the menu options.

    Duplicate Process App

  2. Click the Duplicate option to duplicate the process app.


    ⓘ Tip:


Deleting a custom process app

To delete a process app, do the following steps:

  1. Click the menu icon on the upper right of the Process App tile to view the menu options.

  2. Click the Delete option to view the Confirm delete dialog box.

  3. In the Confirm delete dialog box, enter the process app name in the Delete confirmation field.

  4. Click the Delete button.


    ⓘ Tip: Only creators and importers can delete their process app. For more information about permissions in process app, see the Understanding types of users and permissions in custom process app topic.


Generating a process by using the custom process app

You can generate a process in IBM Process Mining by using the custom process app. The custom process applies all the process logic and configurations that you set up in the process app during the creation of the process app. To generate a process by using the custom process app, you do the following steps in the process app:

Setting up the process details in Process Apps

To set up the process details, do the following steps:

  1. Click Get Started on the Process App tile in the Process app page to see the Process details step.

  2. In the Process details step, enter the name of your process in the Name field.

  3. You can choose to link or not to link your process to an organization. If you do not want to link your process to an organization, click the Next button to go to the Documentation step. If you want to link your process to an organization, use one of the following options:

    • Link to an existing organization: If you want to link your process to an existing organization, select the Use existing organization option and select an organization name from the list in the Organization name field.

    • Link to a new organization: If you want to link your process to a new organization, select the Create new organization option and enter the organization name in the Organization name field.

  4. Click Next to go to the Documentation step.

Viewing the documentation

The availability of the Documentation step depends on the configuration that is set up in the Provide user guidance step in the "Create process app" wizard. For more information on configuring the Provide user guidance step, see Providing the user guidance for the process app topic. In this step, you can view the documentation for generating a process by using the process app. You can click the Next button to go to the Configuring event logs step.

Configuring event logs

The availability of the Configure event logs step depends on the configuration that is set up in the Define user input step in the "Create process app" wizard. For more information on configuring the Define user input step, see Defining the user input topic. In the Configure event logs step, you input the fields based on the configuration in the process app. Depending on the process app configuration, you must input the fields that are marked mandatory and click the Next button to go to the Load your data step.

Loading your data

The availability of the Load your data step depends on the configuration that you set up in the Define user input step in the "Create process app" wizard. For more information on configuring the Define user input step, see Defining the user input topic. To load the extracted data from your enterprise application to IBM Process Mining, do the following steps:

  1. In the Load your data step, drag or upload the data file in the .zip format in the Add files by using drag and drop section. The process app then validates and uploads the data.


    Tip:

    • The maximum file size that you can upload is 125 MB.

    • You can upload only .zip file.


  2. Click the Next button to go to the Generate your process step.

Generating your process

The process app starts generating the process when you click the Next button in the Load your data step. You can see the progress of process generation by viewing the status, namely PENDING, IN PROGRESS, COMPLETED, and ERROR, of the following steps in process generation:

  • Transforming your data
  • Generating event log
  • Applying default analysis
  • Finalizing the process

You can click the Finish button to go to the IBM Process Mining Processes dashboard while process generation happens in the background. When the process app completes the process generation, you can view the new process in the process list on the Processes dashboard.