Syncing Salesforce leads with an on-premises sales system by using an on-premises callable flow

Learn how to use IBM® App Connect to create an event-driven flow that invokes a callable flow in IBM App Connect Enterprise to create a record in an on-premises sales system with a file-based interface whenever a lead is created in Salesforce.

This tutorial shows how data and processing is shared from on-cloud SaaS applications with on-premises activities through an on-premises App Connect Enterprise callable flow. For example, by using App Connect Enterprise callable flows, non-technical or line-of-business (LoB) users of IBM App Connect Enterprise as a Service can create flows that make simple calls to enterprise capabilities. These flows can interact securely with data in on-premises systems and packaged applications, without the need to define special interfaces or expose the on-premises systems directly to the public internet.

Scenario

Your company is starting to use Salesforce for its sales reps to record customer leads that they find. In the past, all customer-related sales information was stored in an on-premises custom sales application. The on-premises system still needs to have a record of all the leads that are found by the sales reps, but they don't want to have to enter the data in both systems. Eventually, the Salesforce system will take over more of the on-premises application processing but for now, both applications need to include the lead information. To integrate these two systems, a hybrid integration approach is adopted, where some of the integration function runs in the cloud, and some must run on premises.

First, find or create everything you need

  • A Salesforce business or Developer Edition account.
  • An App Connect Enterprise as a Service instance.
  • App Connect Enterprise on your local machine.

    If you don't have App Connect Enterprise, you can develop and test integration solutions without charge by using IBM App Connect Enterprise (ACE) for Developers.

    This tutorial was created and tested by using App Connect Enterprise for Developers 11.0.0.8 on Windows.

  • An on-premises integration server that is configured and running.

    You can create and start an integration server in the IBM App Connect Enterprise Toolkit. A work directory is created in your workspace that has the same name as your integration server. For example, in the Integration Explorer pane of the Toolkit, right-click Integration servers, then click Create a local integration server. Enter a name for the integration server, such as myServer, change the options if required, then click Finish.

    Alternatively, you can create the working directory for an integration server by using the mqsicreateworkdir command, as shown in the following example: mqsicreateworkdir c:\myaceworkdir\myServer

If you want more information before you start, you can read more about App Connect Enterprise for Developers and how to use App Connect with Salesforce on the following pages:

Next, create your integration solution:

You can use the App Connect Designer to create your flow from a template, or from scratch if you want to explore the steps to create a flow.

  1. Configure the integration server to connect callable flows:
  2. Import the enterprise integration project into App Connect Enterprise, then deploy the provided callable flow
  3. In App Connect, create and start a flow to invoke the callable flow

Configure the integration server to connect callable flows:

In this section we use App Connect on IBM Cloud® to configure callable flows, to enable secure connectivity between flows running in App Connect on IBM Cloud and in the integration server in your on-premises App Connect Enterprise.

Note: If you have already configured callable flows between App Connect on IBM Cloud and the integration server in your on-premises App Connect Enterprise, you can skip this section.
  1. Sign in to IBM App Connect on IBM Cloud.
  2. On the Dashboard, click the Callable flows icon Callable flows icon
  3. Click Connect callable flows.
    Figure 1. App Connect Dashboard, showing the Callable flows features
    App Connect Dashboard, showing the Callable flows features

    (Click image to view full size.)

    This opens the Synchronize your on-premises agent dialog.

    Figure 2. The Synchronize your on-premises agent dialog
    The "Synchronize your on-premises agent" dialog

    (Click image to view full size.)

  4. Click Download the configuration; this opens a dialog to save the agentx.json file.

    Leave the Synchronize your on-premises agent dialog open (note the dialog contains instructions for installing the agent into the work directory of the integration server in App Connect Enterprise).

  5. After saving the agentx.json file, copy the file to the work directory of the integration server in App Connect Enterprise: workdirectory/config/iibswitch/agentx
  6. In your on-premises ACE software, if the integration server is not running, start it; for example, in the toolkit right-click the server and then click Start, or in the console run the IntegrationServer command like the following example:
    IntegrationServer --name myServer --work-dir c:\myaceworkdir\myServer

    Check the console log (To view in the toolkit, you might need to Refresh the server entry under Independent Resources.) You should see messages that indicate ...The integration server component 'agentx' has been started; for example:

    component starting: "agentx"
    Starting agentx with config folder: 'C:\Users\IANLarner\IBM\ACET11\workspace\myServer\config\iibswitch\agentx'
    2020-06-29 13:31:51.082740: The connection agent for remote callable flows has established a connection to the Switch server with URL 'wss://ibm-sw-aaaahbbbcccciqiz.eu-gb.ace.ibm.com:443/'. 
    2020-06-29 13:31:51.088324: The integration server component 'agentx' has been started. 
    component started: "agentx"
    
  7. Switch back to the App Connect Designer window and then test the agent connection:
    1. Click Test your agent

      You should see at least 1 agent found. Secure communications for callable flows has been established between IBM App Connect on IBM Cloud and the integration server in your local IBM App Connect Enterprise.

    2. You can now close the Synchronize your on-premises agent dialog.

You have established secure connectivity between flows running in App Connect on IBM Cloud and in the integration server in your on-premises App Connect Enterprise.

Import the enterprise integration project into App Connect Enterprise, then deploy the provided callable flow

All the resources that are required for this tutorial are provided in a project interchange file.

  1. To download the project export file, click project interchange file, and save the file to your local machine.
    1. Open the Toolkit.
    2. Import the project into the Toolkit.

      Click File > Import, select IBM Integration > Project Interchange, select the downloaded file, then click Finish. A new project called AppConCloudToACE is added in the Application Development pane of the Toolkit. If you expand the project, you can see the flow entry called WriteDataToFileAsXML. If you want to explore the message flow, double-click the flow entry. You can click each message flow node to see the configuration details (double-click the Compute nodes to open the SQL file).

      Figure 3. The WriteDataToFileAsXML flow
      The WriteDataToFileAsXML flow

      (Click image to view full size.)

      The flow is a basic callable flow that performs the following actions.
      • The CallableInput node registers the flow to be triggered when the endpoint WRITE_TO_FILE is called. In this example, the calling flow passes two local environment variables (FileName and DirectoryName) to this node to give the correct file and directory name of the local file to be written to.
      • The Construct XML Message node converts the incoming data into an XML format, including the addition of a new unique identifier. The node also passes the input local environment variables to the output terminal for use by the following file node, and sets another local environment variable (OutputLocalEnvironment.Variables.Id) to be returned for use by the calling flow.
      • The Write to file node appends the XML data directly to a file on the file system (which is identified by the local environment FileName and DirectoryName variables), adding a new line for each new entry.
      • The Modify response node sets a JSON data property (SET OutputRoot.JSON.Data.status = 'done';) to be returned for use by the calling flow.
      • The CallableReply node sends a response to the original callable node request, including the extra data that is created in the flow: the unique identifier (Id) as a local environment variable and the status (status) as a response JSON data property.
    3. The project contains a BAR file called AppConCloudToACE.bar, which contains the callable flow, ready to be deployed.

      In addition to the callable flow, the project includes the YAML definition file for a flow that is exported from App Connect, called: Salesforce Lead to on-premises.yaml. This file is used later to create a flow in App Connect.

  2. Deploy the BAR file to the integration server that you intend to use for the tutorial.
    1. Check that the Toolkit is connected to your integration server. If it is connected, it is shown in the Integration Explorer view:

      Integration Explorer view, showing the integration server myServer running and connected to the toolkit

      Tip: If the integration server is not shown as connected, complete these steps to connect the server…
      1. In the Integration Explorer view, right-click Integration servers, and then click Connect to an integration server. In the Connection details dialog, enter the host name and port for the integration server. Ensure that the port matches the value of the adminRestApiPort property that was specified in the server.conf.yaml file (in this example, the port is 7600, and the host name is localhost). If the integration server is secured, you must also specify the user name and password.
      2. Click Finish. The connection for your integration server is now displayed in the Integration Explorer view in the toolkit.
    2. Deploy the callable flow to your App Connect Enterprise integration server.
      For example, drag the AppConCloudToACE.bar file from the Application Development pane, then drop it onto your integration server in the Integration Explorers pane. In the console log, you should see messages that AppConCloudToACE was deployed successfully, as shown in the following example.
      
      2020-06-29 11:31:38.493712: Configuration message received. 
      2020-06-29 11:31:38.599276: About to 'Initialize' the deployed resource 'AppConCloudToACE' of type 'Application'. 
      2020-06-29 11:31:38.603516: File node 'Write to file' in message flow 'WriteDataToFileAsXML' cannot write to output directory ''. 
      2020-06-29 11:31:38.603816: File node 'Write to file' in message flow 'WriteDataToFileAsXML' has no valid filename specified as property ''. 
      2020-06-29 11:31:38.650964: About to 'Start' the deployed resource 'AppConCloudToACE' of type 'Application'. 
      2020-06-29 11:31:38.651508: Deployed resource 'WriteDataToFileAsXML' (uuid='WriteDataToFileAsXML',type='MessageFlow') started successfully. 
      2020-06-29 11:31:38.651592: Application 'AppConCloudToACE' has been created successfully. 
      2020-06-29 11:31:38.651640: The source 'AppConCloudToACE.bar' has been successfully deployed. 
      2020-06-29 11:31:38.652264: Integration server finished with Configuration message.
      Note: You can ignore the messages about ...cannot write to output directory and ...has no valid filename specified as property, which indicate that the properties of the Write to file node are not set. The local environment variables (FileName and DirectoryName) for use by this file node are set by the preceding Construct XML Message node to values that are passed from the calling flow.

In App Connect, create and start a flow to invoke the callable flow

Create the flow by importing the flow definition file that is in the imported Toolkit project, AppConCloudToACE. The flow definition file is in the Other Resources subfolder and is called Salesforce Lead to on-premises.yaml. Import the flow by completing the following steps.

  1. In the App Connect Designer, click New > Import flow.
  2. Copy the YAML file from the Toolkit into the App Connect dialog box, then click Import.
    The imported flow (Salesforce Lead to on-premises) opens in the App Connect flow editor.
    Figure 4. Flow diagram for the imported flow
    Flow diagram for the imported flow

    (Click image to view full size.)

    After you import the flow, click each action node to check the configuration and connect to your own application accounts. For the Slack Create message action, select a channel (or user) where you want to post the message.

    The flow consists of following message flow nodes.
    • Salesforce New lead event: The flow is triggered whenever a new lead is created in Salesforce.
    • Callable flow Invoke action: When the flow is triggered, this first action calls the message flow that is running in App Connect Enterprise. The map in the action takes data from the trigger (Salesforce lead fields) and writes it to the message tree that is sent to the callable flow. The map also sets the file name and file directory that the flow will write the file to. These values are currently set to c:\temp and demo2.txt.
      If necessary, change the file name and file directory to values that are valid for the machine where App Connect Enterprise is running.
      Figure 5. Callable flow / Invoke node, showing the fields used for configuration
      Callable flow / Invoke node, showing the fields used for configuration

      (Click image to view full size.)

    • Salesforce Update lead action: This action updates the new lead in Salesforce with data that is sent back from the called flow that is running in App Connect Enterprise. This data is mapped from the returned local environment and JSON data from the callable flow.

      In this example, an If (conditional) node is used to update the Status field of the Salesforce lead based on the status value that is returned from the on-premises callable flow. If the returned status equals done, the Salesforce status is changed from Open to Working.

  3. After you check everything in the flow, start the flow and return to the Designer dashboard.

    Open the options menu [⋮] in the banner, then click Start flow. When the flow status is "Running", click Dashboard.

Test your flow

To test the flow in App Connect, you create a new Salesforce lead, then see the effect of the callable flow on premises. (In Salesforce, the lead description is updated with the ID that is generated by the on-premises callable flow, and the lead status is changed to Working).

Create a new Salesforce lead.

  1. Log in to Salesforce with the same account that's connected to App Connect.
  2. Create a new lead, specifying values for First Name, Last Name, Company, and Email, but leaving the Description and Status with their default values, as shown in the following example.
    Figure 6. For a new Salesforce lead, specifying values for First Name, Last Name, Company, and Email
    For a new Salesforce lead, specifying values for First Name, Last Name, Company, and Email

    (Click image to view full size.)

  3. Save the new lead.

Result

An entry is written to the file that is specified in the Callable flow invoke action (for example: c:\demo2.txt).


<lead><name><first>Joe</first><last>Yeoman</last></name><emailAddress>j.yoeman@email.com</emailAddress><company>Merry Men</company><id>Yeoman477e0cab-e691-4176-8af1-b1ce5aaf485f</id></lead>
In Salesforce, the new lead Description field is updated with the ID that was generated in the callable flow. The Status field is changed from Open to Working because the callable flow's response included the JSON data property status = 'done'.
Figure 7. Salesforce record showing fields updated by App Connect
Salesforce record showing fields updated by App Connect

(Click image to view full size.)

If the flow fails for any reason, any errors are displayed on the flow tile on the Designer dashboard.

Conclusion

You created a flow that's triggered when a new Salesforce lead is created. The flow calls an on-premises integration flow that's running in App Connect Enterprise. The on-premises flow transforms the message to an XML format, adds a unique identifier, and writes the data to a file, ready for an on-premises application to process. The flow then sets a status value and sends a response back to the App Connect flow that contains the new ID and status. The ID is inserted into the Salesforce lead, and the lead's status is changed conditionally, based on the status in the response. The example uses Salesforce as the triggering application and the application on which actions are performed, but you could change it to use any of the application that are supported by App Connect. The App Connect Enterprise flow on premises completes basic file processing but you could use any of the transports and formats that App Connect Enterprise supports. For example, you could try some of the following scenarios.

  • If you have IBM MQ available, try replacing the File Output node with an MQ Output node to send the message to a queue instead of a file.
  • Try adding other actions to the on-cloud flow, such as a Slack "New message" action or a Log node (to write a message to the App Connect log).
  • Try replacing Salesforce with a different application like Insightly.