July 9, 2013 | Written by: Paul DiMarzio
Share this post:
If you’ve been following my blogs, you know that I’m a huge advocate of running business analytics on System z. One of the most exciting technologies that we currently offer in this space is the IBM DB2 Analytics Accelerator, a Neteeza-based workload-optimized appliance that dramatically improves the ability to derive complex analytical insights from DB2 z/OS data.
Our marketing materials claim that the accelerator can run complex queries “up to 2,000x faster.” I know that this is accurate, as I recently reviewed a client’s data showing that the run time of one of their reports was reduced from just under three hours to six seconds using the accelerator—a measured improvement, in a production environment, of 1,908x. And this is pretty consistent with results that I’ve seen from other clients.
So, 1,908x sounds like a lot, right? I think so, but the statement “1,908x faster” doesn’t fully illustrate just how game-changing this appliance can be for organizations looking to turn data into insights. So I’ve decided to put on my creative hat, do a little Internet surfing and work up an analogy to share with you.
Let’s imagine that I run a business in my hometown and keep all my historical data at Grand Central Station in midtown Manhattan—roughly 70 miles away. A customer comes into my shop and wants to know about his past purchases. For some reason (I’m taking a little creative license here) I determine that I have no way of getting that information other than walking, so I ask my customer to take a seat, and I slip out the back door and begin my 140 mile round-trip trek to Grand Central.
Most humans walk at a rate of three to three-and-a-half miles per hour (mph). Assuming I am able to walk at this pace for a sustained period without stopping, I could be there and back in 40 hours. Of course, by that time my customer will have (understandably) left and taken his business elsewhere. In any event, let’s use this number of 40 hours | 2,400 minutes | 144,000 seconds as a baseline for this story. What would I have to do to improve this result by 1,908x?
As it turns out, Metro-North runs trains from my town right to Grand Central! These trains average 60 miles per hour with peak speeds of 80 miles per hour. Assuming I can get a train to run flat-out and make no stops, I would be able to make the round trip in 105 minutes. This is an improvement of 23x over my walking baseline. Good, but I need to do better.
One of the fastest passenger trains on the planet, the Shinkansen bullet train in Japan, can run at a peak speed of 200 miles per hour. Should Metro-North ever offer such service, and make no stops, I’d be able to make the trip in 42 minutes and show an improvement of 57x over the baseline. At this point it’s becoming obvious to me that conventional ground transportation will never get me to my goal of 1,908x.
Thinking out of the box, I turn my attention to firearms. Internet sources tell me that an average .45-caliber handgun can shoot a bullet at a speed of around 661 miles per hour. Even better, I find out that a Winchester .270 rifle shot can travel at 1,924 miles per hour—two and a half times the speed of sound! Assuming I had some preternatural ability to ride a Winchester shell to Grand Central and back, with no loss of velocity, of course, I’d make the trip in 262 seconds. Sounds impressive, right? Still, this improvement of 550x is not even close to the results that I’m after.
After a little more digging, I came across reports of a cool technology called the electromagnetic railgun. In one report, the US Navy announced the ability to fire a 40-pound projectile at 5,600 miles per hour using a railgun! Although I do weigh a bit more than the 40 pound projectile tested, assuming that someday I will be able to ride a railgun projectile to Grand Central and back at this speed I can get my travel time down to 90 seconds. Even this scenario only gets me to an improvement of 1,600x over my walking baseline—still 15 percent short of my goal.
I hope that by now you’re gaining an appreciation of how big a 1,908x improvement really is. In my analogy it represents the difference between walking and riding a railgun to a distant destination. Most people would view a sustained brisk walk of 140 miles to be impossible, and the technology to ride a railgun doesn’t exist.
In the real-life world of information management, it represents the difference between doing nothing and achieving business breakthroughs. There are many complex analytics-reporting projects sitting on the shelf because they are viewed as impossible to generate in a timely and cost-effective manner; like the 140 mile walk, these projects are not attempted. Fortunately, the data management equivalent of railgun technology does exist; it’s called the IBM DB2 Analytics Accelerator, and it’s helping clients achieve truly dramatic results right now. Check out this video from Swiss Re describing how the accelerator is helping them manage their business.
I personally know the team that achieved the 1,908x improvement. They’ve told me that based on these results, the business units are beginning to reevaluate projects that had been previously killed. They’re riding the rail and reaping the rewards. So why are you still walking?