November 11, 2016 By Nick Maynard
Ilene Seelemann
3 min read

Building a DevOps pipeline for an API Connect and Microservices architecture

There are many lessons to learn from a DevOps implementation in a greenfield development project, where 20 new cloud native microservices were built and deployed to Bluemix. Our client’s business partners created end-user applications, driving mindshare and loyalty to our client’s brand.

In this post, we’ll talk about how we used API Connect to socialize, document, and govern access to these microservices consumed by business partners. We’ll examine how we used them to build high value web and mobile applications that leveraged the data and function provided by these microservices. We’ll also show how we evolved our development practice using Agile and DevOps techniques to streamline the deployment of these microservices and vastly reduce the time to production of updates while reducing risk.

Understanding the challenges to manage the solution

We dealt with a few issues in order to manage the solution:

  1. Management of build artifacts

  2. Testing microservice versions for promotion

  3. Configuring microservice environments

  4. Deploying artifacts to live environments while maximizing uptime

  5. Packaging APIs into meaningful, controllable sets

  6. Managing versioning and microservice compatibility (interoperability)

  7. Aligning the presented API layer with the microservice deployments

  8. Monitoring state

The surface area of the solution included approximately 20 microservices. These were deployed across the four environments, databases in each environment, and API Connect with its configuration of APIs and products. While 20 microservices may seem relatively small, the number of microservices and APIs across the environments exceeded 100 moving parts to be managed consistently in a repeatable and predictable manner. With the project’s rapid changes, this couldn’t be done efficiently without automation—even with a dedicated, skilled operations team.

Topology of microservices and API Connect on Bluemix

Our architecture consisted of three main layers:

  1. API Connect providing governance and a consistent access point for all microservices

  2. A set of microservices providing domain-specific features for clients and interfacing with third-party services

  3. Data storage in dashDB and IBM Cloudant

Get started with IBM API Connect.

This architecture was completely replicated in four environments:

  1. Development: Tracking the latest “development” code versions

  2. QA: The latest code release for integration test runs

  3. Pre-production: For partner integration and testing

  4. Production

We modeled these environments as separate Bluemix spaces within a single Bluemix organization. We provided project members with per-space permissions appropriate to their roles, allowing isolation of concerns and confidentiality of production information.

We provisioned a single API Connect instance in a space isolated from the environment-specific spaces. This allowed us to handle all API configurations, management, deployment, and monitoring from this central instance of API Connect. The API products defined in API Connect were parameterized so that they proxied to microservices in each of the environments. As microservices moved through the development life cycle, the APIs that provided access to the microservices aligned with and pointed to the microservices.

How we structured our DevOps toolchain

Our toolchain consisted of the following technologies:

We used Flowdock for team communication and for build and deploy notifications. We used Rally for tracking all work items for the team. Every microservice had its own GitHub repository. Commits to a repository kicked off a Jenkins build, which created both microservice artifacts (WAR files, Bluemix manifests) and API Connect configuration artifacts (YAML files). UrbanCode Deploy (UCD) deployed microservices and APIs to Bluemix, which provided the runtime environment. We used Runscope for health monitoring of the APIs, microservices, and some downstream components. API Connect provided valuable insight into the stability and performance of the microservices, which highlighted opportunities for improvements around response time and latency.

In future articles, we will focus on the details of the Code, Deliver, and Run steps in our toolchain and show how they addressed the challenges listed.

This is a complex system. To help you better understand it, we’ve split up what we’ve learned. Stay tuned for the next in a series of articles covering the following topics:

  • Versioning and Swagger for documentation and code skeleton generation

  • Deep dive into API DevOps

  • Deep dive into microservice DevOps

Learn more about microservices on Bluemix.

Sign up for Bluemix. It’s free!

More from Cloud

Hybrid cloud examples, applications and use cases

7 min read - To keep pace with the dynamic environment of digitally-driven business, organizations continue to embrace hybrid cloud, which combines and unifies public cloud, private cloud and on-premises infrastructure, while providing orchestration, management and application portability across all three. According to the IBM Transformation Index: State of Cloud, a 2022 survey commissioned by IBM and conducted by an independent research firm, more than 77% of business and IT professionals say they have adopted a hybrid cloud approach. By creating an agile, flexible and…

Tokens and login sessions in IBM Cloud

9 min read - IBM Cloud authentication and authorization relies on the industry-standard protocol OAuth 2.0. You can read more about OAuth 2.0 in RFC 6749—The OAuth 2.0 Authorization Framework. Like most adopters of OAuth 2.0, IBM has also extended some of OAuth 2.0 functionality to meet the requirements of IBM Cloud and its customers. Access and refresh tokens As specified in RFC 6749, applications are getting an access token to represent the identity that has been authenticated and its permissions. Additionally, in IBM…

How to move from IBM Cloud Functions to IBM Code Engine

5 min read - When migrating off IBM Cloud Functions, IBM Cloud Code Engine is one of the possible deployment targets. Code Engine offers apps, jobs and (recently function) that you can (or need) to pick from. In this post, we provide some discussion points and share tips and tricks on how to work with Code Engine functions. IBM Cloud Code Engine is a fully managed, serverless platform to (not only) run your containerized workloads. It has evolved a lot since March 2021, when…

Sensors, signals and synergy: Enhancing Downer’s data exploration with IBM

3 min read - In the realm of urban transportation, precision is pivotal. Downer, a leading provider of integrated services in Australia and New Zealand, considers itself a guardian of the elaborate transportation matrix, and it continually seeks to enhance its operational efficiency. With over 200 trains and a multitude of sensors, Downer has accumulated a vast amount of data. While Downer regularly uncovers actionable insights from their data, their partnership with IBM® Client Engineering aimed to explore the additional potential of this vast dataset,…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters