In case you didn't know already, Informix Warehouse Accelerator (IWA) is a state-of-the-art in-memory database (IMDB) that provides a Google-like experience for analytic queries. With its columnar storage capability and extreme scalability using clustering, it is a disruptive technology that provides “speed of thought” analytics to organizations' operational and historical data.
Apart from these, the availability of an ELT tool (SQW) and IBM Smart Analytics Optimizer (ISAO) studio for quick & easy data mart creation & loading along with IWA installable edition makes it even easier for organizations. They neither have to worry about data loading and transformation, nor need warehousing experts to build & load data marts in IWA.
My first chance to really try out IWA in a real-world situation was when I worked on a proof of concept (PoC) showcasing IWA for an Indian stock trading company. They needed to gather stock market movement information, identify the trends, and accordingly advise their clients what stocks to purchase and what to sell. With their current setup they had a 24 hour lag. They would run analytical queries that would run overnight to create summary tables. And then, for business to understand the trends, analytical queries would have to be run against these summary tables. This caused multi-fold problems.
Long query processing times (summary table creation time + analytical queries against these summary tables)
Ad-hoc access to data unavailable
Any change request from business was difficult to implement and time consuming
If summary table creation failed overnight, it caused a 24 hour delay
Apart from that, they had a single system setup that caused scaling up (customer expects data size to increase manifold over the years) a cumbersome and expensive proposition.
When we ran the PoC showcasing IWA, we could show how they could (for one) eliminate need for the summary tables and run analytical queries against near-real-time data, and which performed up to 35 times faster while using only 1/4th of the system resources. Also, with the cluster support we showed how scalability is not an issue anymore.
In another PoC, the customer was an India based healthcare company that wanted to run analytics to best understand the usage patterns of its customers (i.e. patients). Here we saw even more spectacular comparative numbers. Queries in IWA ran at up to 1200 times faster while using only 1/4th resources. In one instance, a query that ran for 1 hour on their current system, was able to run (on IWA) in 3 seconds!
Leveraging the spectacular successes of these two PoCs, we have been able to reach out to more customers. We expect a few more such proofs of concept lined up, keeping me busy and learning more on the technology. The most gratifying part is to be able to run ad-hoc queries performing 100x to 1000x times faster than the competition and watching the pure amazement on the faces of the customers – especially the technology guys. Priceless!