A series of successes

Several years ago, Cullen and his team began to automate software testing with the goal of improving the speed with which they could roll out new releases of their core digital wealth management platform, Figaro. It worked—they were able to shave days off their test cycles.

But the testing team was sharing a logical partition (LPAR) with the development team, and over time their successful testing strategy started to slow down important software development. Cullen and team tried to address this by running their tests after business hours, but that proved to be less than ideal.

“We would make a change, wait for the test to run overnight, find out we’d introduced a bug, fix it, wait for the test to run overnight … over and over, to the point where we were starting to slow down our own processes—the opposite direction from where we wanted to go,” explains Cullen.

Figaro is a very large system, with millions of lines of code, thousands of tables and about 25,000 program objects. Historically, it has run on IBM i on Power servers, so the move to IBM Power Systems Virtual Server was natural. But there’s more to Cullen’s choice than finding the right combination of server and operating system.

With IBM Power Systems Virtual Server and CSI’s test-as-a-service offering, Cullen and team have as much compute power as they need, when they need it. “With this setup, we can request a virtual server with the required version of Figaro and the appropriate data set, run our tests against it, then delete the virtual server,” Cullen says.

The ability to access the right version of Figaro is key. FNZ’s current release schedule calls for new versions of the software every quarter, with patches every two weeks. For testing to be efficient and cost-effective, Cullen’s team needs to be able to spin up an accurate, up-to-date version of the software on an as-needed basis. “We don’t want a situation that in order to run a test, which might only take 15 minutes to execute, we have to install reams and reams of patches to get a working environment—it all has to be prebuilt,” he says.

To accomplish this, the team relies on a Docker-style approach, which use layers to build images. “You have a known starting point, and you add layers on top. We took the stock IBM Power images and gradually layered on more and more configuration software until we had a fully working environment,” explains Cullen.

Now, Cullen’s team has automated the entire image building process to keep everything up to date, using a watcher program to identify newly published artifacts. “Say we want to upgrade from version 1.2 to 1.3 of our system. When we push the version 1.3 patch to the artifact repository, the watcher sees that, then spins up the previous version on a virtual machine, installs the version 1.3 patch, saves the results and deletes the virtual machine.”

As a result of this process, FNZ always has a test-ready environment. “All we have to do is find the right image, spin it up, run the tests and we’re done,” says Cullen.

Plus, with the Docker-based approach, if anything in the intermediary layers changes, they don’t have to rebuild any of the surrounding layers. “It’s quite efficient,” notes Cullen.

The CSI solution also takes advantage of the IBM Cloud Pak® multicloud management technology, which runs on Red Hat® OpenShift® (link resides outside of ibm.com). Specifically, the offering’s cloud automation management capabilities help FNZ organize, templatize and parameterize Terraform system definitions, while Red Hat Ansible® (link resides outside of ibm.com) automation scripts fully deploy the application.