A modern approach to enterprise hybrid integration
The case for agile integration
Why have microservices concepts become so popular in the application space? Because they represent an alternative approach to structuring applications.
Rather than an application being a large silo of code running on the same server, microservices can help you build applications that are composed of a collection of smaller, completely independently running components.
A microservices architecture enables three critical benefits:
With those benefits in mind, what would it look like if we re-imagined integration, which is typically deployed in centralized silos, with a new perspective based on microservices architecture? That’s what we call an “agile integration.”
People and process: Decentralized integration ownership
SOA tended to result in the creation of centralized integration teams and infrastructure to implement the service layer. The central team knew their integration technology well, but often didn’t understand the applications they were integrating, so translating requirements could be slow and error prone. This dependency created ongoing friction given the pace at which projects could run.
Many organizations would have preferred the application teams own their own integration, but the technology and infrastructure at the time didn’t enable that. The move to agile integration with more fine-grained, cloud-native deployment, makes it possible to distribute the responsibility for creating and maintaining integrations. This works together with API management, enabling application teams to manage the exposure of their own APIs.
Furthermore, microservice-based applications often prefer to communicate internally using events. The increased decoupling further improves the team’s autonomy to make changes independently. Agile integration also considers how best to enable teams to publish and consume event streams within and across application boundaries.
Architecture: Fine-grained integration deployment
Traditional integration is characterized by the centralized deployment of integrations in the ESB pattern. In this model, all integrations are deployed to a single heavily nurtured pair of integration servers and can result in a bottleneck for projects. Any deployment to the shared servers runs the risk of destabilizing existing critical interfaces. Furthermore, teams running individual projects can’t independently choose to upgrade the integration middleware to gain access to new features.
Using the same concepts as a microservices architecture, we could break up the enterprise-wide ESB into smaller, more manageable and dedicated pieces. These “fine-grained integration deployment” patterns provide specialized, right-sized containers that improve agility, scalability and resilience, and look very different than the centralized ESB patterns of the past.
Fine-grained integration deployment draws on the benefits of a microservices architecture. Let’s revisit what we listed as microservices benefits in light of fine-grained integration deployment:
Technology: Cloud-native integration infrastructure
Agile integration requires that integration capabilities be deployed using cloud-native principles. This enables a cloud-native platform such as one based on Kubernetes to perform much of the deployment, scaling and availability tasks in a standardized way. Agile integration normalizes the integration skillset required to set up and look after a given capability and ensures that capability is portable across a multicloud landscape.
Adopting a cloud-native approach impacts the ways your DevOps teams will interact with the environment and the solution overall to increase efficiency as more solutions are moved to lightweight, portable architectures.
In addition, the cloud-native approach and technology simplifies automation of build and deployment pipelines and enables cleaner setup and tear down of testing environments. These benefits contribute to much shorter build-release cycles and a significant increase in productivity.
Moving to a cloud-native approach is a significant change. Some high-level characteristics include:
If the above principles are adhered to, container orchestration platforms like Kubernetes provide auto scaling, load-balancing, deployment, internal routing, reinstatement and more in a standardized way to significantly simplify platform administration.
Modern integration portfolios must be able to embrace the core principles and standards of a cloud-native approach. Integration runtimes, API management, event streams and messaging capabilities must all leverage container orchestration platforms to provide the portability and operational consistency required by today’s solutions.