Archive

The benefits of integrating your own CI environment into OpenStack

Share this post:

In a previous post, I provided an overview of the processes and tools used for continuous integration in OpenStack. Here I’d like to take a moment to talk about the benefits of creating and integrating your own continuous integration (CI) environment into OpenStack.

Why would you create your own CI environment if OpenStack already has one available?

The first reason involves your own components and systems. OpenStack projects, Tempest tests and DevStack can accomplish very broad tasks.. However, some components depend on physical characteristics such as the architecture, operating system (including specific distributions) and hypervisor in order to run tests. For instance, if you need integration for a PowerKVM hypervisor, you need PowerKVM in your environment. A distribution may want to test OpenStack against the different versions of their system and a network card manufacturer may ensure Neutron will not break on their cards.

The services for CI (Zuul, Jenkins, Nodepool) used by the OpenStack Infrastructure community are widely accessible, therefore they can be used to build your own environment.

Let’s talk in general about the tools, knowledge and skills you’ll need in order to do this.

What do you need in your environment?

Basically, you will need a cloud environment (OpenStack cloud, of course) with the components and systems you want to test. If you are using PowerKVM as your hypervisor, you’ll need a Power-based cloud with PowerKVM installed on each of the nodes. You’ll also need a pool of servers (such as virtual machines outside or inside the cloud) where you are going to install the services you need.

How large should it be?

First, think about which OpenStack projects you want to check and how many types of jobs you will run. Suppose you are going to verify tests against Nova and you want to cover unit tests, Tempest tests against MySQL, and PostgreSQL. That makes three different sets of tests you want to run. You can get a better understanding of those by looking at a patch review in Gerrit. For example, you may see the following:

Openstack testJobs running on official OpenStack CI by Jenkins—source: http://review.openstack.org

If you want to check every patch submitted for those three jobs, you will need enough VMs to run those checks. Considering we have an average of 90 patch sets for Nova per day, each will run three jobs, and the average duration (considering failures) would be 90 minutes.

In a Gaussian distribution, at our peak, we would need around 50 virtual machines to consecutively run the jobs needed. Since we have the Zuul service that will handle the queue of changes, we don’t need to consider the peak, but that can give us an idea of the size of the cloud we need. The duration of jobs can also change according to the third-party components being tested. The closer this is to the actual community values, the better.

Openstack test summary

Source:  http://stackalytics.com/report/contribution/nova-group/30

In the next and final post of this series, I will talk about the main services used for continuous integration and how you can put it all together. I will also point out some tutorials for next steps.

Can you think of any other considerations when creating your own CI environment? Feel free to contact me on Twitter  or LinkedIn, or leave a comment below to continue the conversation.

More stories

Why we added new map tools to Netcool

I had the opportunity to visit a number of telecommunications clients using IBM Netcool over the last year. We frequently discussed the benefits of have a geographically mapped view of topology. Not just because it was nice “eye candy” in the Network Operations Center (NOC), but because it gives an important geographically-based view of network […]

Continue reading

How to streamline continuous delivery through better auditing

IT managers, does this sound familiar? Just when everything is running smoothly, you encounter the release management process in place for upgrading business applications in the production environment. You get an error notification in one of the workflows running the release management process. It can be especially frustrating when the error is coming from the […]

Continue reading

Want to see the latest from WebSphere Liberty? Join our webcast

We just released the latest release of WebSphere Liberty, 16.0.0.4. It includes many new enhancements to its security, database management and overall performance. Interested in what’s new? Join our webcast on January 11, 2017. Why? Read on. I used to take time to reflect on the year behind me as the calendar year closed out, […]

Continue reading