Using IBM App Connect Enterprise on IBM Cloud to invoke REST operations on a data collection in MongoDB

Learn how to use IBM® App Connect on IBM Cloud® (with a plan that provides enterprise capabilities) to expose a REST API to a MongoDB collection.

Scenario

You want to store customer details in a database and would like to be able to invoke REST operations to create, retrieve, update, and delete customers. You’ve chosen MongoDB as the database and want to construct a REST API to allow controlled access to the customer data.

To achieve this, we’ve provided a BAR file for you to import into App Connect on IBM Cloud, to deploy an integration solution that implements a REST service to create, retrieve, update, and delete customer data in a MongoDB data source. In the integration solution, a LoopBackRequest node is used to issue synchronous requests through a LoopBack® connector for MongoDB, to perform CRUD operations on a data collection in MongoDB. You’ll need to configure the LoopBackRequest node to add a data-source stanza that defines the MongoDB instance to be accessed, and to provide a LoopBack model for the data source. You’ll also need to specify security credentials that the LoopBackRequest node can use to connect to the MongoDB data source. You can define the data-source stanza, the model of the LoopBack object, and the security identity by creating policies in App Connect on IBM Cloud.

First, find or create everything you need

  • A MongoDB instance (and cluster) that is accessible from the public internet. This tutorial uses a MongoDB Atlas instance, which is offered as a free tier for trying out MongoDB. (For reference, we’ve also provided some instructions for setting up a MongoDB instance on premises using Docker at the end of this tutorial.)
  • An IBM Cloud account with an IBM App Connect service instance that provides enterprise integration capabilities; for example, the Lite or Custom Enterprise plan.
  • Optionally, you can also install IBM App Connect Enterprise for Developers if you want to look at and change the deployed flow. It is not necessary for running the tutorial, but it is useful for examining the flow to understand how it works. Note: The enterprise integration project for this tutorial was developed with the IBM App Connect Enterprise Toolkit, but can be examined with the Integration Toolkit of IBM Integration Bus V10.
  • If you want a bit more information before you start, you can read more about the App Connect Enterprise Developer edition on the following pages:

Import the enterprise integration project into App Connect Enterprise or IBM Integration Bus, and examine the message flow

You do not need to complete this step unless you want to look at or change the flows. A separate BAR file is also provided for deploying the integration into App Connect on IBM Cloud, as described in a later step.

  • All the resources required for this tutorial are provided in a project interchange file named appconnect_mongodb_tutorial_pi.zip.

    The steps are the same for App Connect Enterprise V11 Toolkit and IBM Integration Bus V10 Toolkit (the toolkit):

    1. Download the project interchange file, by clicking the link above and saving the file to a local directory.
    2. Open the toolkit.
    3. To import the project into the toolkit, click File > Import, expand IBM Integration, select Project Interchange, and then click Next. Browse to select the downloaded project interchange file and then click Finish.

    A new project called CustomerMongoDBV1 is displayed in the toolkit.

    This project contains one flow called CustomerMongoDBV1.msgflow, which implements a REST service with the following resources:

    (Click image to view full size.)

    (Click image to view full size.)

    As shown in the last two images, the REST service has two main resource paths:

    • /customers for creating a customer and retrieving all current customers
    • /customers/{customerId} for retrieving, deleting, and updating individual customers

    The resource paths provide five operations that each have their own generated subflow for interacting with MongoDB. The subflow constructions are very similar and they need the same configuration to interact with MongoDB.

    Let’s look at one of these subflows in detail. The subflow named getCustomer.subflow implements the retrieval of a customer from MongoDB by using an ID as the key.

    This subflow includes the following actions:

    • Set Id: Contains a few lines of ESQL to set the ID to use for the lookup:
    • Retrieve customer: Uses the MongoDB LoopBack connector to perform the retrieval:

      (Click image to view full size.)

      • The data source name links the LoopBackRequest node to the definition entry (or data-source stanza) that needs to be created in the datasource.json file in the App Connect Enterprise work directory. This file then contains details for connecting to MongoDB; for example, the host name and port. In App Connect on IBM Cloud, the MongoDB definition entry will be created for you based on a policy that you define in the web UI. More details on this later…
      • The security identity similarly provides a link to the database user name and password that you use to access your MongoDB cluster . In App Connect Enterprise, security identities are created using the mqsisetdbparms command and, again, this step will be completed for you in App Connect on IBM Cloud based on a policy.
      • The final thing required is the model for the LoopBack object, which needs to be copied to the data source directory. App Connect on IBM Cloud will do this for you based on any models that you attach to your data source policy.
    • Generate 404 if no customer: Checks the response from the LoopBack connector and returns a 404 error if no doc is found.

    When moving the Rest API to the cloud, the important parts of the preceding subflow are the details given on the LoopBackRequest node. This will require policies to be created in App Connect on IBM Cloud to define the data source, the model of the object, and the security identity. Once these are defined, the flow can run in the cloud without any modifications to the flow itself.

Get the connection details from mongodb.com

To allow App Connect on IBM Cloud to connect to MongoDB, you need to get connection details.

  1. Log in to your MongoDB instance then click the CONNECT button in the Clusters view.
  2. From the "Connect to cluster" panel, click Connect Your Application.
  3. Click the Short SRV connection string option to display the SRV address.

    (Click image to view full size.)

    In our example, the SRV address is shown as mongodb+srv://user1:@cluster0-bjrkj.mongodb.net/test?retryWrites=true. The LoopBack connector requires only the host name portion of this address. Make a note of the host name (in our example, this is cluster0-bjrkj.mongodb.net). You’ll also need the database user name and password that you use to access your MongoDB cluster. This database user must have sufficient access to create databases and collections.

Configure the integration in App Connect on IBM Cloud:

  1. Download the CustomerMongoDBV1.bar file.
  2. Sign in to App Connect on IBM Cloud.
  3. From the App Connect on IBM Cloud dashboard, click New > Import a BAR file and then select the BAR file that you downloaded. Then click Import.

    The integration server is displayed as a tile in the dashboard.

  4. To open the Policies view, click the Policies icon .
  5. Click New policy
  6. Create a MongoDB (Loopback) policy with the following details (to configure the data source and model):
    • Policy name: Enter a name; for example, mongodb.com datasource.
    • Datasource name: Enter mongodbds. (Must be set to mongodbds to match the value given on the LoopBack node.)
    • Host: Enter the host name value from the SRV address in your MongoDB instance; for example, cluster0-bjrkj.mongodb.net.
    • Port: Accept the MongoDB default value of 27017 or specify your configured port.
    • Database name: Enter CustomersDatabase.
    • Authentication source: Accept the default value of admin or specify your preferred value for the authentication database in which the MongoDB user is created.
    • Add model: Use this boxed area to attach a model for the customers object. Download and add this definition of the model: customers.json.
    • Connect to MongoDB via the public internet: Select this check box.
    • Protocol: Select mongodb+srv.
    • Use SSL: Select this check box. Note: The Protocol and Use SSL fields are displayed when you select the Connect to MongoDB via the public internet check box. If you do not select this check box, App Connect will instead attempt to connect to an on-premises MongoDB system.

      (Click image to view full size.)

  7. To save the values for the Policy properties, click Save. You are returned to the Policies view.
  8. Click New policy to create a second policy of type Generic (Security Identity). This is needed because the LoopBackRequest node also requires a security identity.

    The policy name is unimportant (in our example, we’ve used mongodb security identity), but the security identity name must be mongodbsi to match the value given on the LoopBackRequest node. Specify the security identity type as loopback, and then specify the database user name and password that you use to access your MongoDB cluster .

    (Click image to view full size.)

  9. To save the values for the Policy properties, click Save. You are returned to the Policies view.
  10. To attach the two policies to the integration server, complete the following steps for both the mongodb.com datasource and mongodb security identity policies:
    1. Click the 'Used by' cell of the row for the policy that you want to attach.
    2. In the Apply policy dialog, select the integration server (CustomerMongoDBV1) that you want to attch the policy to, and then click Apply.
  11. Return to the dashboard and then start the integration server.

When the integration server shows Running, it is ready to use.

Finally, test your integration:

Now try creating a customer and retrieving the details:

  1. From the dashboard, open the integration.
  2. Click Show API Explorer.
  3. Choose addCustomer from the five operations shown. Then click Try it.
  4. Click Generate to produce test data for a new customer.
  5. Click Call operation and success should get returned.
  6. Check the CustomerCollection collection in the CustomersDatabase database in MondoDB to verify that a new customer has been added.
  7. Go back to the API Explorer view in App Connect. This time, choose the getAllCustomers operation and then click Try it.
  8. Click Call operation and success should get returned with details of the customer you added.

If all of these steps work, then you have the tutorial up and running. If it fails, then an error message with the reason will be returned. Check the integration log for any errors ; you can access the log from the integration in the dashboard.

Using a Docker container running on premises rather than a mongodb.com instance:

To run MongoDB on premises, you need to have Docker running on a local machine that has App Connect Enterprise installed. Then run the following command, substituting the user name and password with your own values:

docker run -p 27017:27017 --name mongodb -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=secret -d mongo:latest

You now have MongoDB running locally and you just need to configure App Connect on IBM Cloud to have access to your local machine.

  1. Change the mongodb.com security identity policy to use the username and password for the docker container.
  2. Change the mongodb.com datasource policy to use a host name of localhost and to not connect to the public internet.
  3. Save the policy and refresh the policy page. You should now see that the policy has a warning against it asking you to update the agent.
  4. Click the policy warning and follow the instructions given. After this is done, a green tick should appear against the policy.
  5. Stop and start the integration to pick up the new configuration.
  6. Now repeat the test from the previous section. It should work the same as before, but using your local MongoDB rather than your mongodb.com instance.