IBM Event Streams offers a fully managed Apache Kafka for the Enterprise.
We are pleased to announce a wide-ranging set of updates to our IBM Event Streams Enterprise Plan offering, including the following:
- Self-service Cloud Service Endpoint and network access restrictions
- Updates to consumption metrics, scalability, and a price reduction
- Cross-region replication
- Schema registry support
This blog post is broken down into sections, so you can skip right to the specifics of greatest interest to your business. We provide the appropriate links to dive deeper and get started with these new features.
Cloud Service Endpoint and network access restrictions
An extremely common requirement across nearly all our Enterprise clients is the ability to restrict their network traffic to non-public interfaces. We have previously supported this via request, but have enhanced this capability to now be entirely driven by our GUI and CLI.
As a client, you have three options available to you:
- Public network
- Public and private network
- Private network
The option to connect over both the public and private links simultaneously allows those users who have started on public-only interfaces to migrate to private-only with no downtime to their applications.
The use of the IBM Cloud private network via Cloud Service Endpoints has the following two main advantages:
- An enhanced security and compliance posture.
- Cost optimisation. If you expose your IBM Event Streams instance using the private network, you will not accrue any bandwidth charges for data consumed from Event Streams to applications that reside within IBM Cloud. At any level of scale, this can provide significant cost savings when using Event Streams at the heart of our event-driven Architecture within the IBM Cloud.
For more details, see “Restricting network access using the Enterprise plan.”
Consumption, scalability, and pricing
As we look at the evolution of our clients using Event Streams at the centre of their architectures, we see many consumption patterns, including the following:
- A few users up to thousands of users sharing a cluster.
- A cluster utilised by a single team/business unit, to a cluster shared across multiple organisations.
- Use cases where the events are highly transient, to use cases where the events require persistence.
As a result, we have revisited the way you consume the service, how you scale, and our price point. We have reduced our entry price point for a fully managed Apache Kafka service by approximately 45%.
We have strived to make this as simple as possible for our clients to consume—with no hidden charges—so that you are able to make a clear, informed decision about the capacity of the managed service you consume.
Our IBM Event Streams Enterprise offering will have four metrics:
- Base capacity unit
- Additional capacity unit
- Additional storage
- Gigabyte transmitted outbound
Each base or additional capacity unit includes the following:
- 50MB/s of data ingress and egress capacity.
- 2TB of storage available for your data retention. Event Streams stores three replicas of your data to ensure the highest level of resilience across three availability zones. When you select 2TB of storage with Event Streams, this is equivalent to deploying 6TB of storage if you are running your own Apache Kafka cluster with the same replication policy enabled.
For example, selecting a base capacity unit, one additional capacity unit, and 4TB of additional storage would provide you with the following:
- 100MB/s of data ingress and egress capacity.
- 8TB of storage available for your data retention.
If you utilise our Cloud Service Endpoint capability, outlined earlier in this blog, then you can avoid any additional outbound bandwidth charge by setting your interfaces to private.
The pricing for these metrics will be the following:
- Base capacity unit-hour: $6.85
- Additional capacity unit-hour: $6.85
- Additional storage terabyte-hour: $0.6
- Gigabyte transmitted outbound: $0.08
This capability will be generally available beginning September 1, 2020.
For our existing clients, your current deployment will be automatically migrated from using ‘Instance-Hour’ to using ‘Base Capacity Unit-Hour’ as the charge metric. This provides the same capacity as you receive today, but with enhanced scaling options and our new features. With Cloud Service Endpoints enabled and set to private network, this provides a 45% cost reduction. Your client success manager will be in touch directly to discuss this transition and the impact on your invoicing in more detail.
Cross-region replication
As our clients move mission critical applications to the Cloud, disaster recovery is a key part of their resilience planning.
There are multiple ways you might look to achieve this pattern, but ultimately, the desire is to have your data replicated from the primary site to the secondary site, including the relevant metadata the enables you to fail-over your applications in the unfortunate event of a disaster.
We will support fully managed cross-region replication between two Event Streams Enterprise clusters. This is charged at an additional flat rate in a 1:1 relationship to the number of capacity units you have deployed with your Event Streams cluster you will be replicating the data from. There is no additional network bandwidth charge.
The pricing for this capability will be the following: Mirroring capacity unit-hour: $1.37
For example:
- 2 x Enterprise clusters with a base capacity unit provisioned in each = $1.37 per hour to enable replication
- 2 x Enterprise clusters with a base + one additional capacity unit provisioned in each = $2.74 per hour to enable replication
This capability will be generally available beginning on August 7, 2020 and you will see this referred to as ‘Mirroring’ throughout our documentation.
Schema registry
As our clients build their event-driven architectures, they are striving to truly decouple their teams from each other, both organisationally and from an implementation perspective.
Utilising a schema registry is a common way to help further decouple your teams and reduce friction to sharing Event Streams pervasively across your organisation.
Previously, our clients would deploy or connect their own schema registry to help enable these patterns, but starting today, you no longer have to worry about running another component separate from consuming your managed infrastructure. Our fully managed schema registry is available by default and free of charge for all of our Enterprise Plan users.
For more details, see “Event Streams schema registry.”
We appreciate you taking the time to read this post, and we hope you are as excited as we are about the value these changes can bring to your business.