The Informix development team has put a lot of efforts over the last year or so to continue to improve the product capabilities.
We strongly believe that this new release will help everyone, customers and partners alike, address the challenges and changing needs of data management.
Will it be faster? Will it be easier to manage? Will it include new functionality? Will it be smarter to accommodate a smarter planet?
What about big data and analytic?
You're in for a treat! Here is the webcast information:
The New IBM Informix: It's Simply Powerful
Date: Tuesday, March 26, 2013
Time: 10:00 AM PDT
Don't miss it.
I dare add to this, to me, the new IBM Informix, it's simply wonderful!
I've been saying for quite a while now that smart meters represent BIG DATA and that Informix TimeSeries is the optimal solution for an operational data store.
We can complement the Informix capabilities with other IBM products. When it comes to real-time processing of huge amount of data. The IBM solution is InfoSphere Streams.
It happens that Streams can interface with Informix as a data source or as a target (sink).
If you want to know more in this area, go take a look at the new information added to the Smart Meter Central wiki on Streams.
Two pages were added. One on a quick overview of Steams (with a youtube video) and another on setting up the environment.
The exact pages URLs are:
More to come as we go deeper into BIG DATA!
I arrived in Vegas Sunday mid-afternoon. Already, the activities have been going on for a day and a half. The expo floor looks good with Informix demos at multiple locations including the blade server with Informix and the theater presentation showing, at least, the clustering capabilities that include SDS, HDR, RSS, and ER.
The evening reception was in two parts: one in the expo and a second one for specific section of the Information management portfolio.
This year I decided to stay at the Luxor, next to the Mandaly Bay. You can walt from one hotel to the other without going outside. To go from my room to the registration desk takes a little over 15 minutes. On my way, I passed 3 Starbucks. I guess a lot of attendees need that to go through the long hours we'll have this week.
The IOD conference is less than a week away. I received an email about a blog entry that lists all the book signings that will happen at IOD. A total of 10. I happen to be one of them.
I wrote a short book titled: "Informix Dynamic Server Application Development - Getting Started". It is a free book that will be available at the conference. My book signing session is as follows:
Tuesday 12:00 pm - 1:00 pm
Location: Mandalay Bay Registration Desk South
Since I'm giving up my lunch for this, please stop by and say hi. For more information on all the book signings at IOD, please see the following blog entry:
(Short URL: http://bit.ly/KB8zy)
I'm currently in Paris in the second week of a business trip. For a two-week trip it is pretty common to have some clothes laundered otherwise this makes for a lot of stuff to lug around.
I took a look at what was offered at my hotel: To launder one shirt (men), they charge 8.50 euros (around 12.37 US dollars). As I was leaving the hotel, I saw a hotel employee with a laundry bag in her hands. Looking at the size of the bag, I could just imagine the small fortune spent by the guest.
As I was walking to the IBM office, I passed a dry cleaner that advertized the cleaning and pressing of men shirts for 2.20 euro per shirt for 5 shirts. The price at the hotel was over 3.8 times that price. With a little knowledge a a 5 minute walk, the hotel guest could save a significant amount of money: for 5 shirts the price goes from 42.50 euros to 11 euros. For a company with a lot of employees that use that type of service, this can add up to significant savings.
Of course, that made me think of Informix. It is well known that IDS provides a high level of performance and scalability and require minimal resources for its administration. In some cases, one database administrator can manage thousands of instances. Of course it is much easier to go with a safe choice, use as much hardware as needed, and hire as many employees and consultants as the situation requires for the management of the environment and business application development. This is simply the cost of doing business...
It seems to me that with a little knowledge and a little effort, that cost of doing business could be greatly optimized.
I recently received a note about the IOD conference, October 25-29, at the Mandalay Bay in Las Vegas. If you register by August 31, you can get the early bird hotel rate!
Please go to the Conference Site to learn more about the IOD conference and register. Here are the top reasons provided to attend:
- Turn your information into a strategic driver of innovation, business optimization and competitive differentiation
- Learn how to transform data into a trusted strategic asset using an information agenda
- Improve business performance by applying data analytics and optimization techniques, e.g. save $423 million and realize 95% improvement in on-time delivery
- Accelerate information intensive projects for immediate ROI, e.g., increasing daily deliveries by 100% and spotting new trends in seconds - not weeks
- Optimize your existing information infrastructure to achieve higher availability and improved ROI
- Add value to your organization by building your skills and knowledge
You can find Informix-specific information on the conference at Informix at Information On Demand
. Here are the top 5 reasons on the Informix side:
- The best in IBM technical training
- Hands-on-labs provide in-depth training
- Immediate return on investment
- First-hand experiences shared by customer speakers
- Everything your company needs in one location
The general session started with an example of context computing and an interview with Captain Phillips.
All that was pretty exciting but what stole the show is the announcement of the partnership
between IBM and Twitter for analytics.
Then I went on my way to attend Streams sessions talking about use cases.
The first one i attended is about a partner, Voci, that has a appliance that converts audio to text.
In addition, it adds additional metadata such as the type of voice, accent, sentiment.
This solution can be augmented with InfoSphere Streams and BigInsight to take actions in real-time.
The next session was a panel of expert on geospatial analytics.
In the afternoon, I attended a session on the features of the new Streams beta that was announced last Friday.
You can find more information at http://ibm.co/streamsdev.
I followed with a session on context computing used to counter fraud. I finished my day
with a panel of users.
The conference is winding down with the last day tomorrow.
Another full day.
It started at 7:00 with a breakfast meeting and was followed by a conference call.
I then went to the conference bookstore for a book signing activity and moved on to a customer lunch.
As I mentioned in other blog entries, my new book is now out, at least at the conference:
"The Power of Now: Real-Time Analytics and IBM InfoSphere Streams"
My afternoon was taken by a Streams and text analytics lab.
I went back to the conference floor and had interesting conversations with many technical people
from different world regions. The conference sure provides great opportunities.
I'll be able to catch up on some Streams sessions Tomorrow. I can't wait to hear about some customer/partners stories
Also, I heard through the grapevine that there my be a big announcement at the general session.
I'll make sure not to miss that either.
After walking by 3 different Starbucks, I arrived at the conference breakfast hall.
I thought I would have a quiet breakfast by myself when I saw Bruce Brown, a big data partner expert.
Soon after, I was sitting others joined us: They were long time InfoSphere Streams experts. That was a great opportunity to talk shop and exchange information.
Then it was time to attend the general session that started at 8:15.
The session started with Jake Porway and Jeff Jonas talking about context computing.
The session was so packed with information that it is impossible to summarized properly.
Lets just say that Bob Picciano talked about three imperatives:
Data is the new natural resource, basis for business advantage
Systems of engagements
Multiple speakers expanded on these themes.
I particularly likes the line: "Geospatial data will become analytics superfood".
There were many interesting sessions to choose from but because of multiple engagements, I only attended
the Joy Global session where they described the real-time analytics they while monitoring mining equipment.
There was so much, if you are not at the conference, you may want to look for InsightGo to be able to attend some general sessions remotely.
Now it's time to move on to Tuesday!
The event went as planned at the Mandalay Bay convention center with presentation on:
Internet of things
Informix gateways and Informix capabilities for the internet of things
IBM Internet of Things foundation
Real-time analytics with Streams in the context of an internet of things architecture
Many people attended and were engaged in the presentations. Overall a success.
The Insight conference officially started with the opening reception.
We are getting ready for a great week of learning and networking.
We're up and going.
The conference is still being setup but there are events happening this Saturday.
This morning I was participating in the "Big Data and Analytics EdCon". This is part of an education session for faculties
offered under the IBM Academic Initiative. This was a hands on session introducing InfoSphere Streams and it was full!
All sorts of other sessions are taking place in other areas of the Mandalay Bay convention center.
Tomorrow, I'll be part of the "Internet of Things Deep Dive" as I mentioned in my previous blog entry.
The deep dive goes from 11:00am until 5:30. There is still time to register for it:
If you are already in Las Vegas for the Insight conference, this would be a good use of your time.
Finally, Sunday evening, the Insight conference officially starts with the Solution EXPO Grand Opening Reception
starting at 6:00pm.
I'll post comments on the conference daily so, stay tuned!
We are barely more than two weeks away from the Insight conference.
As I mentioned in my previous blog, lots of interesting sessions on Streams. Still there is more.
As you know, Streams is excellent at providing real-time analytics. It can be used with other
products to provide a solution in many domains. One of them is the Internet of Things (IoT).
It happens that I'll be participating in an IoT deep dive on Sunday October 26.
I'll be joining the main speakers:
Michael Curry, Vice President, WebSphere Product Management, IBM.
Jerry Keesee,Director, Real-Time Context Computing, IBM.
Jeff Jonas, IBM fellow and chief scientist, context computing
The technical section is divided in three parts:
Kevin brown talking about sensors and gateways
Peter Crocket telling us about the IBM IoT Foundation
Jacques Roy covering data-in-motion with Streams
You can register for the event at: http://insight-deep-dive.eventbrite.com
Don't forget to come see me at Insight in my sessions and labs as well as a book signing
session on Tuesday October 28 at the Insight Conference book store between 9:30 and 10:30.
The book is: "The Power of Now: Real-Time Analytics and IBM InfoSphere Streams"
See you in Vegas!
Ok, this is probably not news to you but there is information you should know.
The Insight conference, formerly known as Information on Demand (IOD), is going on Oct 26-30.
This is only 35 days from now! There is a lot of good content. Fro me, it starts on Sunday with an IoT deep dive call/meeting.
From there, I'll go to the demo ped to spend my evening. Please come visit
For the week, I am particularly interested in the Streams sessions such as:
Just to name a few. I am involved in a few sessions:
LCI-4252A: Hands-on lab "Streams and text analytics" on Tuesday afternoon (2:00pm)
LCI-5454A: Hands-on lab "The Internet of Things and Geospatial Analytics Powered by InfoSphere Streams", on Thursday morning (10:00am)
IIS-7096A : Expert Exchange "How to Harness the Internet of Things"
The other exciting part for me is that I am coming out with a new book:
"The Power of Now: Real-Time Analytics and IBM InfoSphere Streams"
I am doing a book signing on Tuesday between 9:30 and 10:30.
The Insight conference provides many excellent learning opportunities on many subjects including Cloud, mobile/Social, security, analytics, and more.
It is also a great opportunity to network with experts from IBM, partners, and other customers.
I'm looking forward to see many of you there at the Mandalay Bay in Las Vegas.
For more information on the conference, please go to the following web site:
A while back, I started reading a book called "Thinking, Fast and Slow" from Daniel Kahneman.
Daniel Kahneman is a professor of psychology who won a Nobel prize in economic.
I have to admit, I am not done reading it. I need more "plane" time
What I read so far is fascinating. This is the type of book that can be read multiple times.
Today, I just want to relate some parts of chapter 14 where he put together a test to see how people would classify individuals
based on some personality descriptions. Here is the description:
"Tom W is a high intelligence, although lacking is true creativity.
He has a need for order and clarity, and for neat and tidy systems
in which every detail finds its appropriate place His writing is
rather dull and mechanical, occasionally enlivened by somewhat
corny puns and flashes of imagination of the sci-fi type. He has a
strong drive for competence. He seems to have little feel and little
sympathy for other people and does not enjoy interacting with
others. Self-centered, he nonetheless has a deep moral sense."
After reading the description, the subject was asked to figure out which field of study Tom was most likely in.
The description was actually designed so people should rank computer science among the best fitting
because of 'hints of nerdiness ("corny puns")'.
I laughed out loud when I read that part. I immediately though of one of my co-worker, Robert U., that
reminds me regularly that I make corny jokes during my presentations. And yes, I graduated in computer science.
For those who read this blog, if you make corny jokes/puns and graduated in computer science rejoice.
Embrace your nerdiness. You picked the right major
The book is full of interesting information including the fact that even statisticians can misuse/misinterpret statistics.
One I really like is:
"you dispose of a limited budget of attention that you can allocate to activities. . .
You can do several things at once, but only if they are easy and undemanding."
My conclusion: if someone tells you he/she's multitasking, they do trivial work.
When we talk about processing data in real time, it is easy to just write a program and be done with it.
The problems start piling up when we add analytics and volume.
A program is easy to write when it can process records sequentially. Once you reach the limit of this sequential processing, you start adding complexity that may represent the bulk of your work: You start by using multi-threading and eventually you need to also go to multi-processing to take advantage of multiple machines. It is much easier to use a framework to reduce those issues.
Still, a framework may give you the ability to distribute your processing but how easy is it to do? Now you want proper tools to assemble the many operations that you want to link together. Then, you also need to have the tools to easily identify bottlenecks so you can parallelize you operations. What about all the standard operations you would expect to be able to do?
This is where a platform comes in. It gives you the foundation for distributed processing but also gives you pre-built capabilities to interact with the outside world (files, message queues, databases, and so on) and also analytics so you don't have to reinvent the wheel.
For a more complete discussion on the subject, take a look at my two articles on the IBM Datamag site: part 1 and part 2.
InfoSphere Streams is starting to engage the open-source community to provide additional capabilities to its real-time analytics platform.
This is still very early in the process and we can assume we'll see evolve quickly. That may also be a way to consolidate
the offering of the most popular open-source toolkits currently available on the Streams Exchange.
One of the projects is under the name resourceManagers.
The current available resource manager that is available to support Streams is Yarn!
Learn more about what is available for Streams on GitHub by looking at the newest page from the InfoSphere Streams playbook:
Streams on GitHub.
Anyone remembers this cartoon? I think the first time I saw it was in the '80s. Still, it keeps coming back.
This used to apply to IT requests. It can also be applied to all sort of things, including how quickly you want to go from data to actionable information.
In Today's world, it seems that we need to get insights now. This is one reason for the rise of the interest in "data in motion".
Real-time analytics apply in many industries including medical, telecommunication, and security. You can find additional examples in the
following article: Big Data in Motion Where? Everywhere.
There is a special need in processing machine data. The data can be generated at such a rate that we need machines to analyze all that data.
You can find more information on machine data examples in the ebook: The Rise of Machine Data: Are You Prepared.
Data in motion processing is here to stay. It is a great approach to solve many business problems. Of course, this approach does not work in a vacuum.
It is a great complement to new and established systems based on data at rest. Here, I mean systems that use data repositories such as operational
data stores, data warehouses, Hadoop (BigInsights) and other NoSQL repositories.
The IBM solution for data in motion is InfoSphere Streams. You can download a free copy of the software to learn about it.
It is called the InfoSphere Streams Quickstart Edition. Visit the streamsdev site to download a copy of it and access an introductory lab (under Docs).
Do you know about IBM Data Magazine? It is the regular newsletter based on ibmdatamag.com that many people receive in their inbox
every few weeks (or is it weekly?).
This online magazine contains articles related to: Big Data and Warehousing, Databases, Information Strategy, Integration and governance.
There are multiple regular columnists and I am now one of them. I am covering Data in motion in a monthly column.
My first article got published on January 31st and is titled: "Getting the big data ball rolling".
You can find it at: http://ibmdatamag.com/2014/01/getting-the-big-data-ball-rolling/
I have put together a plan for a series of articles. When it gets more in depth, I will complement the articles with
my blog entries. I will also continue to cover other subjects and likely more technical subjects in this blog.
Hopefully this will get me to write a blog entry a bit more regularly than I've done lately.
Until next time...
I have to say, these are busy times!
With TimeSeries PoC and multiple activities around Streams, time flies by quickly.
It's been a while since I updated the InfoSphere Streams Playbook. This was overdue. There are new videos, training material and capabilities that were not reflected in the playbook. Here's what I updated:
In this section, I updated the databases supported and support for MQTT
There is now a link that should provide the complete lists of available videos dynamically. Also, I cleaned up the tutorials and added a brand new series of tutorials.
Video use cases
Some new youtube videos that show interesting use of Streams
With the end of the year so close, we can expect everyone to prepare for the new year. Looks like 2014 will be another fun year!
The other day I ran across an article on Infoworld.com: Cloudera pitches Hadoop for everything. Really?”
Of course, the article starts by mentioning the expression about hammers and nails. This is an old story and it appears that it is getting ready to repeat itself. Like it’s been said: “those who forget the past are doomed to repeat it”.
Hadoop has been the biggest star of the big data story. I have to say that it is revolutionizing data processing and for good reasons. Many seem to point to the use of cheap clusters based on commodity hardware. I personally prefer to attribute it to the large amount of data that has different requirements from traditional data processing.
The traditional data processing needs are still there and still growing. Getting rid of “silos” of data has proven extremely difficult. It also relies on getting rid of years of investments and re-writing many proven applications.
Instead of trying to fit everything into Hadoop, it is much better to have an overall strategy that takes into accounts the different needs of different data sets and make sure the overall architecture accommodates exchange of information between all of them.
Cloudera want to become the “enterprise data hub” powered by Hadoop. Like the article mentions, “Hadoop i still seen on all sides as a bucket of parts..”. Maybe it is a bit early to talk about an enterprise data hub based on Hadoop.
Of course, if all you have is a hammer, everything looks like nail
There is now a new resource for Streams: https://www.ibmdw.net/streamsdev/
The Streamsdev site includes articles, blog entries, videos, and intro labs. You can also get to the download the latest quickstart edition of Streams from there. This way, you can download either the product or a vmware image with it and do the lab at your leisure.
This site is put together by developers for developers. Still, if you are new to InfoSphere Streams, you can find something there for you too. Just go to the getting started section under "Docs".
Since the IBM Information on Demand (IOD) conference starts this weekend, you can also find information on the activities (labs, presentation) on Streams during the conference. You can see the next few acticities on the mainpage or a more complete calendar under events.
This site is evolving. You should go look at it at least once a week to see what's new.
Hopefully many of you are going to the IOD conference next week. Enjoy the conference and learn a lot!
Last week, on October 22, IBM announced a new version of InfoSphere Streams: version 3.2.
This follows version 3.1 that was announced on May 21.
The new version includes some nice improvements such as remote development, Rest API for data access, and improved toolkits.
Over the next few blog entries, I'll go into more details on these features. In the meantime, you can find information on
InfoSphere Streams 3.2 at:
If you are interested in trying Streams, IBM provides the quick start edition that you can download as native product or
as a VMWare image. you can download it at:
Of course, you may need more information on how to use Streams. You can start by browsing through the InfoSphere Strreams Playbook at:
If you have questions, don't hesitate to drop me a note or comment on my blog entries.
Until next time!
If you've been following my blog over the last few years, you can notice a few things lately:
I have not blogged in a few months
My blog's name has changed
The significant part is really the name change. It went from "Informix and Computing" to "Big data in motion".
Let me first address the Informix part. Yes, I am still involved with Informix activities. In fact, I am currently working on a proof-of-concept for Informix TimeSeries that involves technologies such as Java, kafka, zookeeper, fastjson, messagePack, and more. So, Informix continues to be involved in "Big Data" and its use with other current technologies.
Will I continue to talk about Informix? Probably. It all depends if I believe I have something interesting to say on the subject. As long as I have activities with Informix I have opportunities to find interesting information.
Now. What about "Big data in motion"?
A while back I decided to go back to my old team: Worldwide Technical Sales and Enablement.
My main focus is now on InfoSphere Streams. This has already been an interesting ride. I've worked on multiple projects that include putting together an extensive training session, work on PoCs, writing DeveloperWorks articles, and more. I've even put together a DeveloperWorks wiki that centralizes all sort of resources related to InfoSphere Streams. I called it the InfoSphere Streams Playbook.
InfoSphere Streams is part of an overall "Big Data" architecture. There are many ties between Streams and the BigInsights platform and any other technologies that help getting big data under control. Yes, that includes Informix. It also includes many other technologies.
My focus may be mainly on "in-motion" data but the entire "Big Data" solution stack eventually interacts with it. That explains the new blog title.
As usual, I want to continue "casting a large net" so I can be free to talk about anything I find interesting.
So, drop me line, post comments. Let's continue a dialog that will help everyone (including me) learn new things and continue to have fun with our technological challenges.
A few years ago, IBM started talking about a smarter planet: Instrumented, interconnected, intelligent.
We are seeing more and more uses of sensors starting from your smart phone ant its many sensors (GPS, proximity, temperature, barometer, etc) to electric meters at your house. Add to that all the other sensors used in many industrial plants and even sensors on rails!
How can we convert this deluge of data into information?
This leads to issues related to two ways to handle data: in-motion and at-rest.
It happens that IBM has a mix of products that can handle these two "states" of the data:
For data in motion, we can use InfoSphere Streams for real-time analytics based on more in depth analysis on historical data (analytics models).
For the data at-rest, there are problems of how fast we can store it and how fast we can retrieve the information, specially when it concerns many users making requests. This would be an operational data store environment. Then, of course, there is the issue of "in-depth" analysis that requires fast access of large amount of data.
Informix has the combined solution with its TimeSeries capabilities and the Informix Warehouse Accelerator.
Learn more about the use of Informix to solve this big data problem in the following webcast:
Solving the Big Data Challenge of Sensor Data
Date: June 26, 2013
Time: 1:00 PM EDT / 10:00 AM PDT
Register at: https://event.on24.com/eventRegistration/EventLobbyServlet?target=registration.jsp&eventid=641115&sessionid=1&key=AA3293E3AC9715CF3D602D0DEAE4D52B&sourcepage=register
The new Informix, version 12.10 was announced last week. It is time to start talking about the new features in TimeSeries.
The Informix team has added a public version of a fast loading mechanism. It allows to load into existing TimeSeries that are defined as part of a container.
This loader API was previously undocumented. It was only available to use as part of the Tooling. A lot of work went into it since its internal implementation. You should not try to use the older internal version since it disappears in 12.10 in favor of this new one.
You can find a description of its use in the "Informix Smart Meter Central" in the page Loading fastest with the loader API
You should also refer to the Informix documentation for more details.
Since the Loader API is an SQL API, it can be used by any clients including InfoSphere Streams.
For more information on how to use Streams with the loader api, please see the Informix Smart Meter Central wiki: Streams and the TimeSeries Loader API
More to come. Don't forget, the IIUG conference is just around the corner. This is the perfect place to learn about all the new features in Informix 12.10: Simply powerful.
We are seeing more and more interest in using both InfoSphere Streams and Informix together.
This is in the context of "Big Data".
InfoSphere Streams is a platform that allows you to add operators as you see fit.
In our case, there are already a few operators that can be used to read from or write to Informix from InfoSphere Streams.
There is a new DeveloperWorks article that describe how this could be done. With these basic examples you should be
able to integrate Informix in a Streams environment (or vice versa) in no time.
I'm always looking for interesting information to stimulate my thinking.
My morning routine usually starts at around 5:30am and I use my tablet to look at news, blogs, tweets, and some web sites.
As part of the tweets I get, it includes some from a site called TED. I've talked about TED before. Take a look at my blog entry for January 2011: Happy new year!
In this blog entry, I recommended no less than four TED presentations.
For people that don't know TED, it is an organization that organizes conferences on all sorts of subjects. The presentations used to be have to be 17 minutes.
Now, you can find presentations that can also be much shorted. TED's tagline is: "Ideas worth spreading".
So, in the morning, I often check what's new on TED to see if there is something interesting to watch during breakfast (of course, when I have breakfast alone...).
I recently came across one that I thought was interesting considering everything we've been hearing over the last 4-5 years about the global economy.
Of course, the fact that it talks about complexity and emergence is just a bonus.
Here is the link to this presentation: Who controls the world?
Happy new year everyone!
The informix team is always hard at work improving the Informix products.
It turns out that, while working on V.next, a feature escaped and made it into version 11.70.xC5 and above (xC6 being the current release as of October 2012).
It concerns loading data into TimeSeries using a relational view of a TimeSeries (also known as VTI interface). To take advantage of this new feature, you simply
use the TS_VTI_ELEM_INSERT (128) flag when you create the relational view with the TSCreateVirtualTab() procedure.
A simple test showed that this feature loads data 3.6 times faster than previously. Of course, your "mileage" will vary depending on your environment. To know more on how you can
use this new feature, consult the following link from the Informix Smart Meter Central wiki:
Another year is coming to an end.
All in all, not a bad year. Informix released 11.70.xC5 and 11.70.xC6 while continuing to work on the next major version of the product. You can find the latest Informix release notes at: Informix 11.70 Information center
We continue to see more acceptance of features like IWA and TimeSeries. The Informix group also delivered many presentations and demo that the IIUG and the IOD conference. We can ad to that support for regional Informix users' group, new redbook, and so on.
Well... stay tuned. 2013 is lining up to be another good one for Informix. But what about ourselves. Are we improving over time like good wine or...
Here are some of my new year resolutions:
- Get back in shape.
In 2012, I neglected this a bit but I am already getting back to it by running regularly and continuing to train in Brazilian Jiujitsu.
- Learn new things
Informix does not operate in a vacuum. It needs an ecosystem. For me, I need to look into what it takes to integrate Informix more with the IBM BigData products. I already started. You can find some information on using Informix with InfoSphere Streams in the SmartMeterCentral wiki.
I'm also slowly started using twitter (@jroy58). I re-tweetted two tweets and I will put in my first tweet as soon as I'm done with this blog entry.
What about your new year resolutions?
For one, are you using the best Informix you could use? Resolve to upgrade to Informix 11.70.xC6 as soon as possible
There is a new redbook now available for people that want to get into the use of the TimeSeries feature.
Other resources to help you include:
Let me know what you are doing with Informix TimeSeries!
We are now living in a world that is more and more instrumented, intelligent, and interconnected. That is actually the IBM definition of a smarter planet.
This opens the door to many possibilities to better use natural resources and improve many things.
Like "uncle Ben" said: "With great power comes great responsibilities" (I'm sure you know where that comes from ). There is always a way to misuse capabilities.
Recently I ran into an interesting video on Ted called Tracking the trackers.
It is basically about how many different sites that you may have never visited can track you and your information and you can't say anything about it.
You can find this video at: Tracking the trackers
It is about a basically unregulated industry doing what is called "behavioral tracking". Apparently it is a $39B industry!
the video is a little under 7 minutes. Enjoy.
It is easy to jump from stealth tracking to security concerns: What is going on in your network?
If your systems are accessible through the network, you should take a close look at how secure they are. Make sure your INFORMIXDIR files all have appropriate access control.
What about your users log ins? Have you setup your environment in case of denial of service attacks?
Maybe it's time to review this URL: http://publib.boulder.ibm.com/infocenter/idshelp/v117/topic/com.ibm.sec.doc/SEC_wrapper.htm
This title could refer to a lot of things. What about countdown to summer vacation?
That's too far in the future. The countdown I am referring to is to the IIUG conference.
I'm really looking forward to seeing some old friends and find out what's been happening lately. I'm also looking forward to fantastic sessions.
What about The keynote sessions? They rock!
- The Informix Way - Forever relevant for a smarter planet
Jerry always has some interesting to say and good information to share with everyone. I can't wait to hear it.
Of course, I believe my group is doing its part in the smarter planet strategy: TimeSeries for smart meter management.
TimeSeries can be applied in many areas of a "smarter planet" and smart meter management is a no brainer.
- Big Data keynote
I must admit that the title does not give out much
David is Program Director, Strategy and Emerging Internet Technologies at IBM
Internet technologies... We can easily see big data there with all the blogs, tweets, images and so on.
It also ties to the smarter planet since an instrumented and connected smarter planet sure generates a lot of data.
Did I mention TimeSeries? Maybe I should talk to David to show him he has a need for it...
- IBM - An Investment in Informix - Innovation for Champions
Arvind Krishna and Martin Wildberger
We have the big guns out for the last keynote address: The general manager for Information Management and the VP of development for Information Management.
These guys know exactly what's going on between IBM and Informix. It will be great to get there insights on what is happening and, hopefully, also get a glimpse
at future plans.
And those are only the keynote presentations. The conference has eight parallel tracks with six sessions each per day. Lots of learning to be done.
And let's not forget the parties!!!
See you there Sunday night!
First, let me put an end to the rumor that the IIUG conference was moved to San Diego to accommodate me.
It is true, I live in that area. It is also true that I am presenting my fair share of material but I can assure you that not even one passing thought on my location was part of the decision
This being said, the conference is approaching quickly. One more week in March and then a few weeks in April and we're there.
As usual the conference organizers are trying to outdo themselves year after year. This year is no exception. What happened since last year?
For one, Informix 11.70.xC3 was just out then. Since we've seen xC4 come out. Can we hope for xC5 soon?
On my side, I am giving four sessions on various subject:
- Dummies guide to TimeSeries
You want to get started with TimeSeries, come to this session.
- Informix applications uncovered on iOS
Yes, you can teach an old dog new tricks. At least this old dog is trying to prove it.
- PHP and Informix
Web applications have established themselves as mainstream. If you don't know PHP and the web, come see me.
- Update on Infomrix and open-source
Some progress there, come to this session and let's have a discussion.
I think these are interesting subjects. You should look for a lot more interesting sessions at the conference.
Take a look at the list of sessions and hands-on labs at: http://www.iiug.org/conf/2012/iiug/sessions.php
See you there!
Just a quick note to say that I've updated the "Informix Smart Meter Central" wiki.
The changes are in the following page:
I updated the section on how to convert TimeSeries into relational format. That should give you a good example on how to use the Transpose function.
The examples use the stores demo database with TimeSeries data. This way, you can run the examples yourselves.
I added a new video on the "Informix Smart Meter Central" wiki. It is a recording of a live demo accessing the TimeSeries data provided in the stores demo database in Informix 11.70.xC3 or higher. Some of you may have already installed that demo in their environment since the code is available on the wiki but there is a new twist. I added a section that shows the use of spatial and TimeSeries. It is demonstrated using a google map of the bay area (San-Francisco bay area in case you wonder). You can find this video with other videos on the collaterals page:
I think that makes for a nice addition to the demo. It will eventually make its way to the demo code on the wiki.
Take a look at the videos, I hope you enjoy them.
I'm traveling this week to talk to multiple people about TimeSeries. It should be a good week!
I've been silent for quite a while. That does not mean I have not been busy!
A lot of efforts has been put on TimeSeries over 11.70.xC3 and 11.70.xC4 and we are still going full steam ahead. We continue to improve its performance, scalability, usability and functionality.
I wanted to put together a repository of information so people can find it all (or most of it in one place. For this purpose, I put together a wiki on developerWorks that is dedicated to The smart meter support. It is still a work in progress but I believe it is a good start. you can find it using the tinyurl: tinyurl.com/InformixSmartMeterCentral
Let me know what you think.
In the last few blog entries, I've been talking about TimeSeries. This time, I'd like to diverge a little for a change. Still there is a tie to TimeSeries
About a year ago, I went to a E&U conference. As you may know, Informix is making a push in this industry due to the advantages that TimeSeries can provide to this industry. In one of the sessions I attended, the presentor mentioned in passig the "Did you know?" video on youtube. Just the context when it was mentioned made me pay attention. I took a note and decided to look it up later. Last time I checked, it had had over 14 million viewing!
"Did you know?" starts with a global view of the world ("If you are 1 in a million in China, there are 1300 people just like you") and continues to talk about the evolution of the impact of technologies on our lives and its impact in the future.
Some other highlights:
- the top 10 in-demand job in 2010...did not exist in 2004
- (dated!) 200 million register users in MySpace would represent the 5th largest country in the world (What about facebook?!?)
- It too the radio 38 years to reach an audience of 50 million people, it toook television 13 year and facebook 2 years
- It is estimated that 4 exabytes of unique information will be generated this year. This is more than the previous 5000 years
Like it says in the video, we are living in exponential times.
Take a look at it, it's only 5 minutes of your time: http://www.youtube.com/watch?v=cL9Wu2kWwSY
We left off with an insert through the virtual table view. We created a container, a row type, a table, and a virtual table. What if we could simplify this? What if we could avoid creating a container?
One reason why you don't want to create containers could be that you have a lot of data to load and you would need a lot of containers. Would it be nice if Informix could help you with that? Informix can! In the Informix 11.70.xC3 release, we added a capability that does just that.
The new feature if referred to as auto create container. When you insert a new time series in a table and no container is specified, Informix will create one for you if needed. For example, let's take the following table:
CREATE TABLE jroy (
loc_esi_id char(20) NOT NULL PRIMARY KEY,
) LOCK MODE ROW;
WE can insert a new TimeSeries without specifying a container:
INSERT INTO jroy
VALUES(1, "origin(2010-11-10 00:00:00.00000),calendar(tst15min),threshold(0),regular,");
If there is no container available, a container is created as we can see in the tscontainertable table:
SELECT * FROM tscontainertable
partitiondesc autopool00000000 datadbs 16 16 4194538
This features goes a few steps further. If the table is partitioned over multiple dbspaces, Informix will create one container per dbspace and put them in a pool called autopool. It is possible to have the following inserts go through the pool in a round robin fashion to evenly distribute your time series over multiple container and dbspaces.
If you prefer to manage your containers tourseld, you can create your own containers and ut them in a specific pool so you can take advantage of a container pool. You can even create your own policy to decide where new time series should be located.
There is more to know about these capabilities. You can find out more in the information center starting at:
Informix TimeSeries is a specialized storage and retrieval mechanism that optimizes the processing usually done on this type of information. For this reason, it includes specialized storage called "container". A container is created in a dbspace. In fact, multiple containers can be created in a dbspace. A container is created using the TSContainerCreate procedure:
EXECUTE PROCEDURE TSContainerCreate(’meter_cont’, ’datadbs’,’meter_data’, 0, 0);
This command creates a new container called meter_cont in the datadbs dbspace. It is created specifically for time series elements of type meter_data (row type). Since we are talking about a row type, it could include anything a row type accepts. The only restriction is that the first column has to be a datetime year to fraction(5). Here's a simple row type that could be used:
CREATE ROW TYPE meter_data (
tstamp datetime year to fraction(5),
The last two arguments represent the initial space allocation and the growth space allocation. This is similar to initial extend and next extend. A value of 0 resolves to the default of 16KB.
With this in place, we can create TimeSeries in a table. Let's start with the following definition:
CREATE TABLE ts_data (
id int PRIMARY KEY NOT NULL,
We can insert a row in a table with an empty TimeSeries as follows
INSERT INTO ts_data VALUES(1, "origin(2011-07-27 00:00:00.00000),calendar(ts_15min),container(meter_cont),threshold(0),regular,");
We now have a row in the table with an empty TimeSeries column. This is different from
NULL: The TimeSeries exist but has no elements in it.
Now, you may say: "Whoa! How do I insert data in that TimeSeries? Must be difficult".
The TimeSeries functionality includes a way to create a relational view on a table that contains a TimeSeries column. If the table were to include multiple TimeSeries column, you could create multiple "views", one for each TimeSeries column. This capability is provided through an Informix feature called the Virtual Table Interface (VTI). This is a capability that allows Informix users to make something look like a standard relational table. At this point, there is no need to describe this interface further. The Informix TimeSeries provides a stored procedure that facilitates the creation of that virtual table. For example, we can create a relational view on out ts_data table as follows:
EXECUTE PROCEDURE TSCreateVirtualTab('ts_data_v', 'ts_data',
'origin(2010-11-10 00:00:00.00000),calendar(cal15min),container(raw_container),threshold(0),regular', 0, 'raw_reads');
This creates a virtual table called
ts_data_v on the table
ts_data and referring to the TimeSeries column named
raw_reads. The character string expression in the middle starting with
origin provides information to the interface about what to do if a new row is inserted and a TimeSeries needs to be created. After the executio of the procedure, you will have this new table that effectively has the following definition:
CREATE TABLE ts_data_v (
id int PRIMARY KEY NOT NULL,
tstamp datetime year to fraction(5),
If you want to insert into a timeseries, you simply use a standard insert statement. If the row does not exist, it gets created, if it is there, the TimeSeries column gets updated. Here's a simple insert example:
INSERT INTO ts_data_v
VALUES (2, "2011-08-10 08:25:00.00000"::DATETIME YEAR TO FRACTION(5), 12.5);
A simple standard SQL insert... How easy can it be?
We have a lot more to talk about. Next time, we'll start introducing some 11.70.xC3 capabilities. This is starting to get exciting! See you next time
In 11.70.xC3, we added some new time series capabilities. Why would you care?
Time series are found everywhere. It is simply data that is collected over time. It could be changes in stock price and transaction volumes. It could also be reading of your house electric meter. Readings could be done every 15 minutes for example to provide a much more accurate picture of how electricity is being used. Other time series examples include weather information, network traffic, thermal readings in a large data center, and so on.
One key characteristic of time series is that the processing always include a time component. For example, you want to get all the meter readings for one month for a specific customer. With this data, you can calculate daily consumption, running averages, etc. To do this type of processing, you need quick access to the specific range of data you want to analyze and you also need to get it in time order.
Informix provides a data type that is used specifically to optimize time series data. It also comes with a extensive set of functions used to manipulate these time series. The Informix TimeSeries provide three major benefits:
- Space savings
In a standard relational database, each time series element must have an identifier and a time stamp because the time series element are stored in a separate table from the object it refers to (such as an electric meter). You then need an index on the identifier and the time stamp so you can joint it with the table that allows you to select what identifier and time range you want to operate on.
In contrast, Informix TimeSeries stores the time series in the same table and row that represents the object. You don't need the additional index on the identifier to join tables. Also, the data is kept in time order in the time series. In the case of regular intervals, you don't even need to keep the time stam since the position gives you the time
In customer tests we have regularly seen that Informix TimeSeries takes one third of the disk space.
- Performance benefits
Just by having less data on disk gives the Informix TimeSeries a significant performance benefit but it does not stop there.
The data is ordered. This means it is much faster to get to the exact subset of data that you want to process. By being ordered, This means you will likely find the next record that you are looking for on the same page as the last one. In a relational system, all this data could be scattered over a large number of pages without specific ordering. This would cause a lot more I/O operations to be executed than in the case of Informix TimeSeries. You also don't need to go through an additional sorting step before you process your data.
In customer tests we have seen as much as 60X performance improvements over standard relational in some queries.
- Simpler development
Informix TimeSeries comes with a set of functions that allow you to manipulate the time series. For example, if you need to group ou r reading to go from a 15 minute interval to a hourly interval, it can be done in a simple statement. Similarly, if you want to calculate a running average: simple statement. This means you don't need to write specialized code to provide this processing. It is built into the Informix TimeSeries capabilities.
The end result can be simpler code, less maintenance, and faster time to market
Informix TimeSeries also provides the ability to create relational views on top of your time series data. This opens the door to the use of standard off the shelf products to do things like reporting.
With this very brief introduction, we are now ready to talk about the improvements made in 11.70.xC3. This will have to wait until next time
Informix often adds features in fixpacks and xC2 and xC3 are no exception. I strongly suggest that you take a look at the list of new features that are listed in the release notice. You can find it at:
Release notice 11.70.xC3
In my next few blog entries I will not cover all the new features. I will limit myself to two main areas:
- Compatibility functions
- TimeSeries improvements
For anything else, see the release notice and the Informix documentation. The easier way to do this is to use the information center that can be found at:
Information center 11.70
This release added a few compatibility function that makes it easier to move application to informix. They include:
- CHARINDEX(substring, source_string, [start_position])
Search for a string in another one and return its starting position. You can start at an offset (start_position) in the source string
Convert a radian value to degrees
- INSTR(source_string, substring [,start [,count]])
Search for an occurence of a substring starting at a specific offset
- LEFT(source_string, len)
Return the "len" leftmost characters
Synonym for LENGTH
Convert a degrees value to radians
Return a string in reverse order
- RIGHT(source_string, len)
Reverse the "len" rightmost characters
Return a string of "len" spaces
- SUBSTRING_INDEX(string, delimiter [, occurence])
Returns the substring that ends at a specific occurence of a delimiter
Take a look at the details of these functions. I'm sure you will find a good use for them.
July 1st was the 10 year anniversary of the IBM acquisition of Informix. Since the acquisition, Informix has releases version 9.3, 9.4, 10.0, 11.10, 11.15, and 11.70. A few days ago, we releases 11.70.xC3. Other recent addition include the Informix Warehouse accelerator that introduce game changing technology for the data warehouse/data mart area. Add to that the Informix-Genero for fast application development and mobile applications.
So much has happened in these 10 years. Go take a look at Informix. Download one of the free edition and give it a try. For people that think they know Informix, go take a look at the large number of improvements we've added to it over the years. Go visit www.ibm.com/software/data/informix and find out more about what is going on. The IBM Information on Demand conference is coming up. This is the best way to learn about the latest capabilities and network with Informix partners, customers, and IBMers. The conference is held in Las Vegas October 23-27.
Now that my major deliverables are done in xC3, I'll be back regularly to talk to you about these major improvements and how many people can take advantage of them.
Ten years with IBM and going strong. There is so much more to come. Stay tuned!
Hello everyone, I've been buries in development activities for the last many months. That explains why I've neglected my blog lately. I think I see the light at the end of the tunnel. Hopefully it's not a train...
One thing that will make me come out of my cave is the IIUG Informix conference.That's a great event that takes place this year in Overland Park, KS between May 15 and May 18. As usual, this is the perfect opportunity to network with other Informix enthusiasts and the Informix lab people. The architects will be there in force, make sure to let them know what you like and what you want. Also, we should not forget the great sessions you can attend. A lot of effort has been put into this. I'm sure everyone will enjoy the conference.
While you wait for the conference to learn more abot Informix, you could also read some of the technical articles published recently such as:
There is more so stay tuned.
It's hard to believe that we are already at the end of March. Seems like it should still be January. According to my blog, it still is January! I better get on with it!
Informix just came out with version 11.70.xC2. No big deal you may think. Wrong. It is a big deal! With xC2 we are making available a new edition: IBM Informix Ultimate Warehouse edition.
I'll be talking about this at the France Informix Users group next Monday. With this, queries that take hours now could take minutes. Some queries end up perfroming 100 times faster! For more information Look at:
And that's not all. We've been looking at ways to help our 4GL customers modernize their environment for years. We want customers to get more value out of their 4GL code and new application developments. the result: Informix Genero. Find out more at:
Stay tuned. The year is barely starting.
2010 has been a great year with the release of Informix 11.70 and 2011 is lining up to be a busy year with plenty of activities and execution on the plans of v.next.
I also hope that in 2011 we'll see even more participation from the Informix community to continue to make Informix and the solutions around it better and more exciting.
Part of making Informix and its ecosystem system better is to share ideas and be exposed to ideas that may or may not be related to Informix. If you remember, in my blog entry of October 9, I talked about where good ideas come from. It is time that I divulge the source of these comments. It came from a site called TED. That specific presentation is Where good ideas come from
Here are a few other presentations I enjoyed from www.ted.com:
There are many more interesting talks in there. I hope you'll enjoy these short presentations. Who knows, by exploring these presentations and others, you may come back with a new outlook on how we can use Informix to make our world a smarter planet.
Someone asked me the following question:
"How do I keep passwords in the database so nobody can get them?"
It means that we cannot keep the the passwords in plain text in the database. Informix has a few functions that can be used for encryption: ENCRYPT_AES and ENCRYPT_TDS. It would be easy to create a table and encrypt the column that contains the passwords.
The next statement that came up was: "..but, if someone has the encryption password, he can get all the passwords. We need to protect the passwords from internal access".
This means that we need to use a different password to protect each password in the table. The solution I proposed was to use the password to encrypt itself. Let's look at an example:
CREATE TABLE passwd (
INSERT INTO TABLE passwd VALUES(1, ENCRYPT_AES("Jacques", "Jacques"));
INSERT INTO TABLE passwd VALUES(1, ENCRYPT_AES("Lance", "Lance0"));
INSERT INTO TABLE passwd VALUES(1, ENCRYPT_AES("Daniel", "Daniel"));
INSERT INTO TABLE passwd VALUES(1, ENCRYPT_AES("Umut", "Umut01"));
The values inserted look as follow:
SELECT * FROM passwd
I can now test f someone has the right password for user 1 by using the password value to decrypt itself:
SELECT col1, DECRYPT_CHAR(col2, "Jacques") FROM passwd WHERE col1 = 1;
If I use the improper password, I receive an error:
SELECT col1, DECRYPT_CHAR(col2, "Jacques") FROM passwd WHERE col1 = 3;
26008: The internal decryption function failed
One more thing. Note that the encryption password must be at least six-character long. This is why in the example I padded some encryption passwords. An easy way to work around it would be to always add padding to make sure we meet that minimum size. Keep in mind that the maximum size of an encryption key is 128 bytes.
With this approach, we can keep passwords in the database and keep them secret.
Wednesday started with an Informix "eat and meet" breakfast followed by nine different Informix sessions spread throughout the day. My favorite session was: "How Hildebrand and IBM bring smart metering to homes across Britain". It was very interesting to see a real-time system where people can see their power consumption and compare it to a pool of similar housed to see how they are doing. The system does not only measure the total consumption at a home but can break it down to specific outlets. For example, some people were able to find out that their energy consumption was greatly impacted by their use of hair straitening devices. Another person could find out that they spent around 250 pounds per year to run their old refrigerator. Buying a new one for 200 pounds made it pay for itself pretty quickly.
Of course, the other presentations were also interesting. They covered areas such as building data warehouse, grid-based replication, Informix in the cloud and more.
An additional 11 sessions were held on Thursday to wrap up the conference.
The one thing that is hard to measure at a conference like this is the value of the interactions with other people. Discussions on different interests and new challenges, and also how Informix has been used. This ties into what I mentioned in this blog on Oct 9. Good ideas come form people interactions. The conference provided a good environment for that. This was a great conference and you can expect interesting things coming out of the Informix lab in the future. I'm sure we'll have a lot to say next time we meet: The International Informix Users Group (IIUG) conference in Overland Park, Kansas, that will be help between May 15 and 18, 2011.
Another year, another conference. It has been so busy that I have not had the time to write a short blog entry for each day. Here is my quick update.< /p>
It all started Saturday morning with the business partner council and the customer advisory council on Saturday. I attended the customer advisory council and I found it interesting and full of good discussions.
The conference was kicked off with an opening reception on Sunday night and we were off to the race. There were eight Informix sessions on Monday including presentations on how IBM helps Cisco, open source, hands on lab on high-availability, another one on the new features of Informix 11.70, bests practices for virtual environments, and performance enhancements. Of course, the most popular session was from Jerry Keesee titled: "Informix at IBM: The next decade".
The day ended with an Informix reception at the Mandalay Bay beach casino for an Informix 11.70 launch celebration and to start looking forward to the next decade of Informix at IBM.
Tuesday started early with an Informix "eat and meet" breakfast at 7:00am, followed by nine Informix sessions throughout the day. The sessions covered areas such as upgrade, new features, Informix warehouse, application development, 4GL, embeddability, flexible grid, and more. It was also interesting to hear about how Informix is used to run a steel plant.
The day ended with a beach party reception. Now it is on the Wednesday with another full agenda.
Yes, a new version of Informix is now available: Informix 11.70.
There are a lot of great features in this release. I could talk about the flexible grid that allows you to manage many machines like one and support rolling upgrades. I could talk about the new analytics features where we've seen speed up of warehouse-type queries of around 50%. I could talk about storage provisioning, improved installation and embeddability features. Yes, I could talk about all this but at this time, I want to talk about some features that should interest application developers.
I have to admit I am a little biased since my group is called application development services. However, the features I want to talk about were either requested by customers or have had a very positive reception in early mention under non-disclosure or during the beta period.
The first one will facilitate porting schemas from other databases to Informix. Let me first show an example:
CREATE TABLE tab (
col1 int NOT NULL default 0,
col2 int NULL,
col3 integer REFERENCES tab1(col1) CONSTRAINT tab1_c1
ON DELETE CASCADE
The first improvement is the ability to change the order of constraints and default values. Before Informix 11.70, the col1 definition would have returned an error since the default clause had to be located before the NOT NULL constraint.
The second improvement is the ability to explicitly say that a column can accept NULL values. Before, it was implied if the NOT NULL constraint was not there.
The last improvement shown in the example above shows that we can add "ON DELETE CASCADE" after the constraint name.
Another improvement in the DDL area is the ability to conditionally execute CREATE and DROP statements. Here are two examples:
CREATE TABLE IF NOT EXISTS tab ( . . .);
DROP PROCEDURE IF EXISTS my_proc();
If, for example, you want to make sure a table is re-created, you could always say:
DROP TABLE IF EXISTS tab;
If you want to make sure that you keep the table if it already exists, then don't do the "DROP IF EXISTS" and simply use "CREATE TABLE IF NOT EXISTS".
Finally, here's another DDL feature that was in great demand. It is not really an application development feature but it has been requested a lot: The ability to define the EXTENT size in a CREATE INDEX statement:
CREATE INDEX myidx tab(col1) FIRST EXTENT 8 NEXT EXTENT 8;
Don't forget to read the release notice since there are many other improvements on the INDEX capabilities.
On the DML side, we are now able to use expressions in the COUNT aggregate function. This can be useful if you want multiple aggregates in one statement:
SELECT COUNT(*) total, count(CASE WHEN sex = 'M' then 1 else NULL) males
COUNT(CASE WHEN sex = 'F' then 1 else NULL) female FROM tab;
Without this capability, you would have to solve this problem with three separate statements. For example:
SELECT * FROM
(SELECT COUNT(*) AS total FROM tab ),
(SELECT COUNT(*) AS male FROM tab WHERE sex ="M"),
(SELECT COUNT(*) AS female FROM tab WHERE sex="F");
These are just a small part of the new improvements in Informix 11.70. Make sure you read the release notice to learn more about Informix 11.70 at:
I listened to a presentation on this subject recently.
What I found interesting is that the research found that good ideas do not come from a Eureka moment. For example Darwin recounts his Eureka moment in his auto-biography. Further study of his personal journals show that Darwin had the full theory of natural selection many months before his stated Eureka moment.
According to research, most good ideas come from discussions:
- The rise of coffee houses is credited for the Enlightenment period in England
- In research labs most ideas come from the weekly lab meeting when people share their mistakes, issues, etc.
Another interesting point was that many good ideas come from the connection of people that share their thoughts to form a complete idea that is worth pursuing.
How can we generate good ideas we can act upon and make our environment better?
We need to interact with people in a situation that is conducive to generating these ideas. We have such an opportunity in just a few week: The Information on Demand (IOD) conference in Las Vegas, October 24-28.
Think about it: we will be with a bunch of people that have technical problems to solve around the use of technology in general and Informix in particular. We'll listen to presentation on new features, solutions in different industries, best practices, bird of a feather sessions, mingling in social settings such as the Informix celbration on Monday night.
Let's take advantage of this great opportunity! See you in Las Vegas!
Here's another one:
This whitepaper summarizes a comparison study in which the total cost of ownership of IBM Informix is shown to be 31+ percent less than Microsoft SQL Server. This is of great interest to mid-size businesses that face considerable challenges in their IT environments to reduce costs while gaining competitive advantage.
You can download the report here: http://ht.ly/2Ln0d
A new report is now available on the economic impact of using Informix. The study is based on a large international retailer with over 4,000 instances of Informix.
The report concludes that over a three year period, the retailer saved over $1 million dollars. The study was done in May 2010. I am convinced it would be even better if they were to do it again with Panther. You can find the report at:
There is so much going on!
As you surely know, we've been doing a closed beta of the next version of Informix. We have received a lot of great feedback and we keep on working on this release.
We still can't talk about it. It is just a matter of time before we can do so stay tuned.
On other fronts, I am working on a follow up to my Application development short book. I've received a lot of positive feedback on this book and I am excited about continuing on the subject. When will it be ready? I'm hoping sometime this year.
Finally, do you realize that we are barely more than a month away from the Information on Demand (IOD) conference? I hope to see you there.
As you know, the IIUG has a page devoted to open source products that can run on Informix. There has been a new addition recently. It is a patch to have Drupal version 6.16 run on Informix.
Drupal is a popular product that provides capabilities to publish, manage and organize a wide variety of content on a web site. To run it with Informix, you need:
- Informix version 11.50 or higher
- PDO_IBM (and the IBM common drivers to compile it)
You still need to download the code from the drupal site (www.drupal.org) and then apply the changes provided at www.iiug.org/opensource.
In my blog entry of February 17, 2010, I had to put out a retraction about the common drivers. As I said then, I had to start lobbying for their inclusion with Informix.
I am happy to report that the Informix client SDK version 11.50.xC7 includes these drivers. Note that they are not included in Informix 11.50.xC7. I would expect that future releases of Informix will include a CSDK that has the common drivers.
There is a bit of work to do after the CSDK installation to complete the common drivers installation. I'll cover that in an other blog entry later. If you need information on how to use the common drivers, I suggest you download:
"Informix Dynamic Server Application Development: Getting Started":
If you are interested in application development, the following URL is also of interest:
Until next time...
Here's something you may want to act on:
IBM Informix Survey for Continuous Availability White Paper
Complete the Survey to win an Apple iPad! - 1 week left
Every response matters. - Start Survey here!
IBM Informix is the database software voted #1 in customer satisfaction. Clients choose Informix because it is reliable, low cost, and hassle free. Solution providers choose Informix for its best-of-breed embeddability.
Yet, there are some people that still don't 'get' Informix, or realize the many benefits of deploying it. Help us gather data to support this claim.
- Informix is exceptionally hardware efficient, which means that (in the REAL world) you need to spend MUCH less on hardware to get the same performance as other products.
- Informix is exceptionally reliable, which means that (in the REAL world) you don't need to pay lots of people to make sure it stays 'up'.
- Informix is exceptionally scalable, which means that (in the REAL world) it can be idling one moment and then processing thousands of transactions the next with no apparent stress.
Advanced DataTools is working with Oninit, to gather data about what happens in the REAL world to support these assertions with empirical evidence in the REAL world.
The data collected will be used to compile a report that will be made available to every CTO, IT Director and IT Manager. Along with this, they will receive a list of all the major application vendors that are now porting their applications to Informix V11.5 and a document outlining the key reasons to choose Informix.
Every response matters. - Start Survey here!
Win an Apple iPad. One randomly selected participant in the data collection phase of this research will win an Apple iPad, provided by Advanced DataTools Corporation, an Advanced IBM Informix Business Partner.
Note: Public Sector Employees are not eligible
I recently ran into the mention of INT8 and, by association, SERIAL8 by Informix engineers and a recent redbook. I want to make a quick comment on that.
These two types were added a long time ago to support the eight-byte integers (64 bits). They are defined as being a 10-byte structure that includes two "standard" integers. It was done this was so eight-byte integers could be supported on 32-bit operating systems. Now it appears that most operating systems support a native 64-bit integer. For this reason, new data types were added to Informix version 11.50 (fixpack 1). The new types, BIGINT and BIGSERIAL, take less space and perform better. Here is what the release notice says:
Improved Query Performance for Large Integers and Serial Data
The BIGINT and BIGSERIAL data types, which are provided as alternatives to the INT8 and SERIAL8 data types. can provide better performance than the INT8 and SERIAL8 data types.
So, let's forget about INT8 and SERIAL8 and let's use BIGINT and BIGSERIAL available in Informix 11.50.
Just a quick note to say that I'll resume blogging soon.
I've had some medical issues that took me out for over a month but I'm coming back.
Great things are hppening with Informix. Stay tuned!
The day started with a Q&A with IBM excutives: Alys Passarelli, Inhi Cho Suh and Rob Thomas. There were a number of good questions and it was an opportunity for the excutives to state their commitment to Informix and describe many of the efforts in progress.
Once again, a good mix of sessions on subjects from customer case studies, database administration, security, best practices, and use of tools such as Eclipse.
The day ended at 4:30 after the delivery of 25 sessions.
This was a great conference with lots of good information and fantatic networking.
I did forget to mention the key note presentations of the evening of Monday. Rob Thomas gave us his view of the Informix business and a glimpse at his plan for continuing successes. It was followed by a presentation by dr. Arvind Krishna, general manager of IBM Information Management. Great information!
Tuesday, sessions started at 8:00 AM. I was the moderator for the session on disk level encryption. It was about a flexible product that can protect your database data at rest. It works for file-based dbspaces (cooked files) or raw devices. All that, transparently to Informix. Mark Jamison did a great job at presenting the product in a clear and concise manner.
Theere were many sessions on application development in cluding sessions on Groovy, Perl, Python, PHP. Of course, there were also more sessions on duifferent aspects of tuning and also presentation on embeddability, replication , warehouse and so on.
With the sessions starting at 8:000 AM, we had a total of 35 sessions for the day.
This was followed by a casino night. What a full day!
The day started with a keynote speach by Jerry Keesee. I believe the key to the presentation is that the Informix is taking the next step to insure continuing growth of the Informix product. There are some exciting things happening on that front.
The first session I attended was "Data Modeling" given by Jack Parker. There were a lot of interesting examples how a data model greatly impact your production system. I have to agree with Jack that the greatest performance gain in with the data model, not tunig the database after the fact. This is why database experts should be involved at the beginining of a project. They can then also take advantage of database extensibility; something I could talk about for a long time...
There were five sessions going at once for a total of 30 sessions for Monday. There were a lot of good sessions on subjects including database administration, application development, programming, security and more. there wer demos of all sort and great conversations to be had. Someone told me that one of the benefits of this conference is that they can solve in minutes problems that have benn bugging them for weeks.
Of course the evening event was also worth attending with a bunch of engineers coming from the lab just to talk to everyone. All in all a fantastic day!
Sunday was the beginning of the conference. Even though there were tutorials during the day. The real beginning was the welcome reception. The exhibitors booths were ready and the attendees were cheerful. IT was a great networking opportunity with good food and plenty to drink.
I have to admit I missed a lot during the reception because I was in multiple intense discussions.
All in all, a great start to the conference.
I ended my last entry with the IBM statement about smarter planet: instrumented, interconnected, intelligent. I'd like to comment a little more on the instrumented part.
In my last entry, I mentioned the utilization of active RFID to monitor a large data center. Of course, this monitoring compares data points at a specific point in time. Most likely, the readings are done at specific time intervals. The same would be true if you were monitoring the energy consumption and temperature of houses in different neighborhood in a city, traffic information (cars or packets), or any other thing you'd want to collect.
There are two main concerns in this type of processing:
- How do we quickly ingest the large amount of information generated into a database?
- How do we efficiently process the information?
The first concern is to be able to ingest the information without falling behind and with cycles to spare for analysis. In most cases, the information must be kept for further analysis or for historical comparisons and analysis.
The second one is about the analysis itself. Informix has addressed these issues with time-based data with the TimeSeries datablade. It provides an efficient way to store and process large amount of data very quickly. If the ingestion rate is a concern, Informix also has the TimeSeries Real-Time Loader. As its name implied, you can ingest a large amount of data and make it available for analysis virtually immediately.
For a smarter planet, keep these efficient tools in mind. Informix is likely to be the answer to your needs.
I saw the cover of Computer world the other day with a title of "Swinging toward centralization". I'm not one to be jumping on trends but I think this idea has merit. To me, it ties into virtualization, possibly cloud computing, and also the IBM concepts of the smart planet.
Centralized IT could mean first the optimization of hardware resources. The best approach is to use virtualization so all the hardware resources can be used optimally. For example, instead of having, let say 100 computers running at 50%-70% utilization, you can centralize and use virtualization and either reduce the number of computers to around 70 or use the extra capacity for growth. This is a pretty conservative example. Just consider this quote from Computer World, April 20, 20009:
"Austin Energy: With a new virtual environment, applications run on 150 servers instead of 600"
Centralization gives you this opportunity. Note that I'm talking about centralizing the hardware resources. If you centralize processing for one large application, you'll likely need the help of advance features such as IDS Continuous availablity (CAF) and the integrated replication capabilities (HDR and ER).
Centralization does not mean that the personnel must also be centralized. Today, network access is pretty much a fact of life (I so wanted to use the word ubiquitous!). All the application and system management can be done from anywhere. For IDS, just consider the Open Admin Tool for IDS (OAT) or management tools from our partners such as AGS and CobraSonic. Managers can consider these resources as part of a "cloud".
What a nice segway to my next point
We hear a lot about cloud computing. You can buy time on some machines in the cloud. We could also mention software as a service like in the case of LotusLive (see https://www.lotuslive.com/en/) or the IBM cloud offering. This does not mean that you have to go outside to have a cloud. You could create a cloud from your centralized data center and provide capacity on-demand based on resource optimization.
When we talk about a large centralized data center, the server consolidation is only part of the savings. the saving in energy can be significant. The other day, I listened to a presentation by an IBMer that manages a large data center providing services worldwide. Here are the type of things he did:
His team installed active RFID sensors to monitor the temperature and humidity levels in different areas of his data center, including multiple locations in the racks, and at different times. With this information, he was able to clearly identify machine needs. At one point, he was able to identify that if he installed a (raised) floor tile with holes at a specific location, he could eliminate his "hot spot" without increasing his air conditioning needs. He even figured out the correlation between applications and machines heat output. So he can regulate the room temperature based on which application is running!
Talk about a great example of a smarter planet: instrumented, interconnected, intelligent (devices).
I ran into a simple problem the other day: I got an error while creating an index because the key was too big to fit in my index. As you may remember, the maximum size of an index key on a standard Unix/Linux system is 387 bytes.
Why do we have this limit?
This is a function of the page size and the way a B-tree index works. With the limit of 387 bytes on a 2K page, we can have at least 5 keys per page. This way, we divide the data in at least 5 parts at each level. the end result is eliminating comparisons to get to our our result faster. If we had only one key per page, it would be the equivalent of doing a sequential scan so the index would be useless.
In IDS version 10.0 (2005), Informix introduced the configurable page size. from that point on, it is possible to create DBspaces with page sizes of up to 16KB in size. the page sizes available has to be a multiple of the basic page size: 2KB or 4KB.
These larger pages can provide better performance when you have a wide table where the row size could be, let say 12KB. This way, you can fit an entire row in a page instead of using page chaining to support these larger rows. The savings in I/O could make a noticeable difference in performance in many situations.
Coming back to my indexing problem, I can fix it by using a larger page size. According to the documentation, the maximum index key size is as follow for each page sizes:
max key size
If your key fits in a 2KB page (shorter than 387 bytes), you could still use a larger page size for your index. The difference is that more keys would fit in one page so the index will not be as deep so it could provide additional performance.
Why not simply use the 16KB page size everywhere?
The short answer is that you could waste space on the page used for a table. A page can include a maximum of 255 rows. If your page size is 16KB and your row contains only two integers (2 x 4 bytes), you could, in theory, have over 2000 rows in that page. Since we are limited to 255 rows, we are wasting over 14,000 bytes.
Why not use four or five different page sizes?
Each page size requires its own buffer pool. We have to decide how much memory to allocate for each of these pools. Our decision may not result in the optimal memory allocation. The result is that some pools will have too much memory and others would benefit from more. Bottom line, this would make system administration more complex.
I would suggest to limit ourselves to two page sizes. The default page size and another one. The second page size depends on the environment requirements. I would also look at the size of the I/O on the particular machine and how many requests do multiple I/O on sequential data.
If you haven't looked at the configurable page size in IDS, maybe it is a good time to do so now.
Since I've been on a common driver kick lately, might as well keep on going...
There was a chat with the lab on Feb 25th that talked about the common Java JDBC driver (referred as the JCC driver): Top 10 reasons to consider IBM Data Server Driver for JDBC and SQLJ for IDS
You can use the JCC driver with IDS when connecting using the DRDA protocol. Some of the benefits include:
- Better integration with WebSphere
- Ability to use the capabilities PureQuery
- Better tracing and debugging
- Full IDS clustering support
- Superior performance over the Informix JDBC driver
All this is significant:
- PureQuery can increase the performance of SQL statements by analyzing the usage and make changes transparently from the application. For example, it can detect the use of the same statement with different literals and convert that under the cover into a prepared statement.
- Full IDS clustering support includes working with the connection manager to automatically and transparently connect to an alternate server when the primary fails.
- Superior performance: It provides a 5% to 10% performance boost over the Informix JDBC driver.
If you are using Java, maybe it is time to start looking into the JCC driver. You can download it from the IBM site at (10MB):
For more information on this chat with the lab:
Here's Where you can find information on this chat with the lab:
Looks like I jumped to conclusion too quickly. I won't give you any details or attenuating circumstances. I simply did not check properly. It looks like we do hve something on the Windows platform but not on the others.
I simply have to statrt lobying for the data server drivers as part od CSDK on all platforms. In the meantime, you can download the common drivers starting at this URL:
The one you want is the IBM Data Server Driver Package (DS Driver). On Linux, it is a 24MB download.
More on how to use it later.
There was a big change for me this year: I left the Informix CTE group to lead a new group. I am now a manager... and architect.
My new group is called Application Development Services. This mean that my group looks at IDS from a programmer point of view. Let me give you an example of what that means. Let's look at the major features included in IDS 11.50.xC6:
Backup from an RSS server
Dynamic listener threads
View event alarms
Basic Text Search enhancements
MERGE statement enhancements
I care about these features but I my attention goes to a feature of the new Client SDK that deserved a one line mention in its release notice:
"When you install Client SDK or IConnect, you have the option to install IBM Data Server Driver version 9.7. For more information, see the Client Products Installation Guide."
As you may remember, the long term direction for client applications is to use the DRDA interface to IDS. With this one line statement, I can now write programs using CLI (ODBC) without having to have to figure out where to get the driver. Since IBM has multiple packages available, I could have easily made the mistake of thinking that I need to download the entire DB2 client (about 600MB) to get this functionality.
In addition, this is all I need to build PDO_IBM for PHP applications or IBM_DB gem file for Ruby and Rails development.
As far as what my group will do, we can start by figuring out and prioritizing what features will make Informix more attractive to developers/programmers. It's not just features in the server. It has to consider everything. Even documentation.
I'm sure I'll have more to say about this later this year. Hopefully I'll have interesting results to report by the time I see some of you at the IIUG conference in April.
There's a children book that I used to read to my kids. It was about a boy that was laying around on the grass when he say a fly go by. What followed was a bunch of animals chasing each other.
Lately, I took a break from blogging (I hope you've noticed!), like a little boy laying on the grass, enjoying a sunny day. During that time, IDS 11.50xC6 came out. Here are a few interesting features:
- External table: an SQL interface to files to allow for very fast load and unload.
- XA transactions on secondary servers
- Backup on the RSS server: You can make an archive of an instance on an RSS server
- Dynamic listener threads: You can start, stop, and restart listener threads for the soctcp or tlitcp protocols without interupting existing connections.
There are also enhancements to the MERGE statement and the attach/detach capability among other thing. You can find out more about the xC6 release in the release notice
Some of you may remember that Lester Knutsen (Advanced DataTools Corporation) had a "fastest DBA" contest at the IIUG conference last April. when I was at the IOD conference, I picked up a copy of the Data Management magazine and found an article from Lester summarizing the tuning approaches. You can find the article on the web at:
I did not noticed a session on Wednesday. Luckily, I went to it Thursday morning. It was: "Tuning Informix in a Sandbox Environment" by Russell Glancy from GSN Digital.
Russell covered in details how a product from exactsolutions, iReplay, allows him to test new configurations, versions, and tuning in a safe environment using the same workload as his production machine. this way, he is knows exactly what will happen when he makes the changes to the production environment.
I also co-presented the session "Keeping costs low and maximizing flexibility for Jamaica using IDS" with Walt Brown, senior manager at FSL Jamaica. My role was mainly to introduce Walt and let him present his environment. Walt went into details about their environment and that they basically run all the Jamaican government systems, including tax collections that was even active and used during a hurricane.
There were several other sessions including:
A deep dive into the IBM Informix 4GL Service Oriented Architecture Feature, Gaga Mahesshwari, IBM
Dimensional modeling for IBM Informix warehouse users, Fred Ho, IBM, Sandra Tucker, IBM
Managing IDS configuration ans performance with server studio and sentinel, Keshava Murthy, IBM, Anatole Vichon, AGS Ltd
And several more... All that on the last day of the conference!
The conference is over. It is now time to go back to work.
Once again, another full day. There were Informix sessions on embeddability, virtualization/cloud computing, security, and zero-downtime upgrade. We also heard a great presentation on database tuning from Rick Rabe and Tom Girsch from Hilton Hotels.
Great sessions altogether. Now on to Thursday.
In Arvind Krishna feature keynote titled "Reduce Your Data Management costs with Workload-optimized System", we heard about Cisco Systems. They mentioned that they chose Informix a few years ago after looking at all possibilities for embedded databases including open-source ones.
I spent some time with Walt Brown (from FSL) and Cathy Elliott to fine-tune his presentation. More on that Thursday.
There were several interesting sessions Today:
- SOA Enablement on IBM INformix 4GL, Gagin Maheshwari, IBM
- Building Data Warehouses with Infomrix, Lester Knutsen, Advanced Data Tools
- Hands on lab on end-to-end security with Informix, Ted Wasserman, IBM
- Open Admin Tool for IDS, John Miller III, IBM
- All About IDS CAF, Conection Manager, and Failover, Ron Privett, IBM
- Using Informix in Telecommunications, Kevin Brown, IBM
- Secure and available public finances with IDS continuous availability, Cesar Jiminez, Jalesco Mexico Government
And, of course, demos, discussions and food on the expo floor and in the networking event in the evening.
We had many good presentations on Monday and of course several impromptu meetings all over the place. Time is running short so I have to keep this entry to a minimum.
In One session I heard about Choice Hotel that has 6000 properties in 10 different brands. They strongly depend on IDS to run their business. In another one, I heard about peapod online grocer that also relies on Informix IDS to run their business. Finally, we also heard about the new IBM system x bundles for Informix where they provide tested configurations in "T-shirt" sizes (small, medium, large, X-large) to fit any businesses. They mentioned that such a configuration showed much better price performance than a Sun system running Solaris.
More to come on Tuesday.
I remember seeing something like this title in some Informix marketing material many years ago. I think it was related to the fact that IDS has extensibility features that allow developers to adapt IDS to their business requirements as the technology and needs evolve.
The "future built-in" idea came back to me as I was reading a computerworld article titled: "the desktop traffic jam" (see: http://www.computerworld.com/s/article/342870/The_Desktop_Traffic_Jam). In there they were talking about a new feature in Windows 7 (User Mode Scheduling) that lets thread multiplexing take place in the application instead of in the kernel. They go on to say: "Handling this multiplexing in the application instead of in the operating system kernel makes thread scheduling more efficient.
I know it's not quite the same but it is similar to the idea that IDS decides its thread scheduling, making it more efficient since it is will not re-schedule a thread that is in a critical section of code. This way it avoids having threads that get scheduled to find out that they have to wait. All that making the threading model more efficient. I wonder how difficult it would be to take advantage of thread multiplexing onto cores. Could it be as simple as having one CPU VP per core wih some "core affinity" of the operating system supports that? Then IDS would already be there... with the future built into it.
I was joking around about security-related events in a previous blog. It looks like what goes around comes around...
I was in Montreal last week visiting partners and customers. To make it there I went through Los Angeles and had the joy of being selected for additional security screening. It is not your father's additional screening! They did not only go through my bag and looked at all my papers - luckily they did not take the time to read everything or I would have missed my flight - They went through my wallet, looked at all my credit cards and IDs. For the first I was starting to wonder about privacy-violation. They also took the time to go through my passport and look at the different countries I visited.
I'm all for good security measures but I fail to understand why going through my credit cards makes planes safer. In fact, they asked me if this procedure made me feel safer. My parents told me I should always tell the truth...
Over the last few years we regularly heard stories about information leaks. Information like credit card numbers, email accounts with password and so on. The problems are numerous, going from disgruntled employees to lost backups and different security access problems.
If you have needs for security and data privacy, make sure to review what IDS 11 provides. It includes communication encryption, backup encryption, column-level encryption, database roles including security officer, and label-based access control (LBAC). Of course we could add to that the resistance to denial-of-service attacks and other features that can be sued to assist in securing your database system.
Make sure you discuss your security and privacy needs with your IBM-Informix technical specialist.
When I was in France, I met two partners/resellers: VMark and Frame. Both partners are strong Informix partners and supporters. It is always good to meet partners of this caliber.
In addition to them, I must give a particular mention for ConsultiX's Khaled Bentebal that went the extra mile and re-started the France's Informix users' group with a meeting on September 30th. The meeting was well attended with over 20 people despite some scheduling issues that greatly reduced the advertising for it. There was a mix of roadmap, positioning and technical presentations that were enthusiastically received by the audience.
This year we have seen several countries starting Informix users groups. the one in France is the latest that shows that Informix is growing and doing well. I wish Khaled and the France Informix users' group all the best.
Last Wednesday, Terri and I went from Brussels to Roosendaal (Netherland) to visit our Informix partner Informa. The first thing we saw when we arrived at their building was a 4 feet tall informix logo sign, the original blue logo.
We had a great meeting talking about the Informix roadmap and the state of the Informix business as they see it at Informa. Bertino and Rob also told us that they found two Informix customers that did not know that IBM had bought Informix!
We were back in Brussels between 5:30 and 6:00 in the evening and went out for dinner. When it came to pay, it turned out we could not use our credit cards since our cards do not include a chip. Here, in Europe, they all use smart cards.
That reminds me of the Informix conference in Chicago back, I believe, in 1997. At that conference, attendees would get a smartcard that was used for multiple purposes. One of them was that they could go to a PC, insert their card in the reader and take a conference survey. After the survey was submitted, they could go claim a T-shirt as a thank you prize. Before they could get the T-shirt, The smart card would be checked to make sure the attendee completed the survey and then mark their card with the fact that they had received their T-shirt. As it happens, I wrote the application that took care of the survey.
The survey was done through a web browser running on a windows machine (16-bit windows at the time). A smart card reader was attached to the PC. The attendee would insert their smartcard in the reader and invoke the survey URL. This request would execute a program on the server that would call back the PC using the PC internat address and a pre-defined port number to read the smart card and fill out the basic information on the form such as name and address. Once the survey was submitted, the application would again access the smartcard to turn on the indicator that said that the survey had been completed.
I'm sure that the capacity of smart cards has greatly increased over the last 10 years or so. A lot could be stored on those. We could store a biometric key as password (fingerprint) and all sort of personal information such as medical records and medical activities, including prescriptions. With this always up-to-date record, it could reduce risks of errors, drug abuse, and so on. The update could be done to the smartcard at the point of service and also sent to a national database through, let say, web services.
IDS can handle millions of transactions per second. It has a proven track record of reliability and scalability and is used to stringent response time requirements. Fro example, IDS is able to handle the employee badge of IBM employees worldwide to give them access to different areas of IBM. IDS also handles large streams of financial information and makes them available for analysis almost instanteneously. Handling the medical record updates would be no problem since we can easily scale out through the distribution of the data over multiple machines either through the continuous availability feature (shared disks). In the medical record case, I would likely look at Enterprise Replication (ER) as my first enabling feature.
All that to say that smart card may be worth another look. The use of smart card with IDS could open the door to many new capabilities in all sort of emerging market and new application.
I think Terri is pulling my leg. She is apparently receiving concerned emails about what happened in Brussels. It was a humorous situations that I wanted to relate in a fun way. I guess I have a future in fiction writing :-).
Really, nothing happened. She took a picture, the police courteously told us that the American embassy did not want people to take picture. Terri deleted the picture from her camera while having a pleasant time with the officers. We then left and laughed about it.
So, don't worry, Terri is doing fine and we all had a good time in Brussels. I strongly encourage people to come and visit.
I am currently in Belgium, Brussels with Terri Gerber. Last Tuesday, we had a successful meeting with an Informix customer. After the meeting, in late afternoon, Patrick Billens took us around to show us the sites of Brussels including among other things the royal palace and the "grande place". Little did I know that soon, Terri would almost cause an international incident with the digital equipment she was carrying.
It happened soon after we saw the king's working palace. The flag at the top of the palace indicated that the king was currently there working. We turned the corner after the palace and Patrick pointed to another building with an American Flag in front of it. Terri took out her digital camera and quickly took a picture. Within seconds, two Belgium police officer were in hot pursuit and quickly caught up with us. Before all h**l broke loose, Patrick intervened, avoiding the worse. I could only imagine what could have happened: Two officers tackling the red-jacket wearing culprit followed by a struggle to get to the digital camera.
Terri did not surrender her camera. Instead, she wrestled with its ease of use and deleted the offending picture in front of the officers. Satisfied with the action taken, we were free to go on our way.
At this time, Terri is on her way back home to Massachusett. Is the picture really deleted? I'm sure it would be possible to un-delete it. Hopefully this won't be a concern for the department of homeland security. If that becomes an issue, I'm sure Terri would be willing to share information with homeland security on how much Informix could help them make the country safer.
Please see the following story Trafficmaster and IBM Develop Solution for Smarter Driving to learn more about how Informix makes life easier for drivers in the UK. Here's a quote form the article:
"Trafficmaster is able to provide drivers with real-time route planning and more accurate estimated arrival times than ever before with the help of the IBM Informix data base technology," said Stuart Berman, Executive Director, Trafficmaster.
IDS 11.50.xC5 became eGA on July 24th. It includes several new features including the "CONNECT BY" syntax and the MERGE command. There were other improvements in multiple areas such as administration and usability, and in the continuous availability including Enterprise Replication (ER).
For more information look at:
I'd like to come back to the book "The Goal" I mentioned in my last blog entry.
This book focuses on manufacturing environments but the interview at the end of the book mentions that the concepts of the theory of constraints (TOC) can be applied to other fields. Looking back in teh book, I found that they ask three basic questions about the impact of changes:
- Did you sell more?
- Did you reduce the number of people on the payroll?
- Did you reduce inventory?
We can easily see that this makes sense to a financial person in manufacturing. Let's see how we can look at it when our concern is running a database.
Did you sell more?
That could be a tough one because sometime it is difficult to tie what we do to the company sales. that reminds me of a need analysis I did early in my career. The drafting department wanted to get a CAD system. At the time, that represented an investment of around one million dollars. I asked: "What happens if the plans are late?". I got blank stares as a reply. I should have talked to their customers to find the answer. We shold always ask what happens if we take longer to do something or if we don't do it. Here's a great quote:
"The cheapest, fastest and most reliable components of a computer system are those that aren’t there"
Gordon Bell, Encore Computer Corporation
Did you reduce the number of people on the payroll?
That's a question we always try to avoid but the bottom line, this is a question that is considered. Don't forget that if we can sell more with the same number of people, that's the same as reducing the payroll.
I've met many customers that have a mixed environments where we see a 10-1 ratio of Informix personnel compared to the personnel for the competitor's platform. Why not bring that up to the appropriate people. I'm sure your local IBM representative will be happy to help.
Did you reduce inventory?
Dr. Goldratt (author of "The Goal") says that investment is the same as inventory. So, what investment is made to increase sales? What is the return on investment? This seems to be a great opportunity to talk to people that use other DB products: How much are you investing in people to run these systems? What could you save there? How much are you investing in hardware? Could that be reduced? How much in software? I've heard that people that add Informix to their environment can get significant discount from their other DB vendor. That represents a reduction in the investment.
I think these three questions are worth exploring no matter which environment you're in. That can be good for your company, for you, and for all the people that invest their efforts into the Informix products.
These are two concepts I've been reading about lately in a book from Eliyahu M. Goldratt (The Goal).
It's interesting to read that a system throughput is determined by its slowest component. Of course, that's something we are familiar with in database management: we want to optimize the I/O to get better performance. What I found more interesting is that when an event is delayed, it can have a direct impact on the overall system throughput. For example, if the slowest component is delayed, it represents a direct loss to the system. In other cases, other components can take a long time to catch up after a delay.
One key to all this is to look at improving the entire system and the way to do it is to find out where the bottlenecks. Once they are found, we must figure out how to make sure they are not idle waiting for something to happen and that they don't do extra work.
This seems to be a lot of what an Informix DBA does when there are performance questions. I could easily point to disk fragmentation by expression, use of prepared statements and so on. The thing is that I've also seen other situations where people point to the database as the source of the bottleneck to find out that it is outside the database. I've seen issues of network and recently I was told by a customer that they must have a specific response time because the transaction already takes 3 times that before outside of IDS. IDS has to sprint because the other components jog.
In another situation, I found that what the customer was seeing as one database requests turned out to be over 100 SQL statements. The kicker was that most statements were unnecessary.
Next time people point to the database as the problem, make sure to get the complete picture from end to end.
Lately I've had a lot of internal discussions about features, benefits, and qualities of Informix Dynamic Server version 11. Two characteristics tht came up were the fact that IDS can be invisible and that Informix is everywhere. Humm... everywhere and invisible... we could do a lot of jokes about that... but I don't want to be in the doG house :-)
We just closed the second quarter of 2009. A lot happened during that period: we had the Informix conference, the release of IDS 11.50.xC4, Informix Warehouse, and Storage Optimization with deep compression. Of course that does not even include IDS 11 training sessions given around the world, proof of concepts, customer discussions, many upgrades, and multiple production implementations of the latest features.
So, what do we have in store for the second half? I can't really tell you :-(.
There are two things that are pretty obvious that I can mention: IDS 11.50.xC5 will likely come out in the second half and the end of support for IDS 7.31 is September 15. That should not come as a surprise since IDS 7.31 has not been sold since September 2008. It had quite a long life (IDS 7.31. was released in 1999, last millennium!).
I'm hoping that all 7.31 customers are already working on the upgrade (you can upgrade directly from 7.31 to 11.50). Since IDS 11.50 is a superset of 7.31, that's should provide a minimum of inconvenience. Then they will be able to take advantage of all the performance improvements and all the new features that makes IDS even easier to use and manage. I really believe that people that are happy with IDS 7.31 will be impressed with all the improvements in 11.50.
People on IDS 10.0 should start thinking about moving to 11.50.
If you have any questions about going from 7.31, 9.4, or 10.0 to IDS 11, don't hesitate to contact your local IBM Informix expert.
If you've read my last entry, you know what happened with my watch while I was in Strasbourg. Considering that I have had that watch for a long time, I decided that it was time to replace it with a newer model so I spent $35 for the latest version of the same watch. My watch had run its time.
You must be wondering why I'm talking about a watch in an Informix blog :-). Informix (IDS) being a software product keeps on running. The bits don't wear out. Or do they?IDS 7.31 end of service: September 2009IDS 9.40 end of service: April 2009!IDS 10.0 end of service: September 2010IDS 11.10 end of marketing: September 2009
So maybe the bits wear out... There are many other reasons why you'd want to go to IDS 11.50. They include the improvements in the areas of performance, ease of administration, business continuity, security, and application development. You also need to consider the new features and the integration with other IBM solutions.
To give you an idea about what you are missing, you can read the following slightly outdated document: Why upgrade to IDS 11.50
Staying up-to-date with IDS is another way to increase your business advantage. Look into it as soon as you can.[Read More]
Last week I stayed at a quaint hotel in Strasbourg. Since the room did not have an alarm clock, I decided to use my watch to wake me up on Monday morning. Considering that there is an eight-hour timezone difference between Denver and Strasbourg, using an alarm is a good idea.
I woke up on Monday 30 minutes before the alarm was supposed to ring. That's long enough to make it worthwhile falling asleep again so I did. I woke up again with a start, picked up my watch and looked at the time: the display was blank!I needed to find out what time it was in a hurry. Maybe I was late for the start of the class! Luckily for me, it turned out that it was the time I was planning to get up at. I guess my brain kept track of the time as I was sleeping. It has worked in the past but I don't find this method the most reliable. At this point, I started using my phone as my alarm clock.
Later that week, when I was in Paris, I had to go visit a partner. The sales specialist send me the information. I wrote the address down on a piece of paper and went to grab a taxi. The taxi driver could not find the place even with the use of a GPS device. I did not have access to my email with my laptop, I did not write down the partner's phone number and I had no way to contact anybody. I was about to tell the driver to turn around when I remember that I get my emails on my phone. Luckily, there was a phone number and we were able to get to the right location.
Twice in one week! Since I had to leave my hotel on Saturday at 5:00am, I did not want to take any chances: I setup a wake up time on both my phone and on the television/alarm clock. Surely at least one of the two would work. It turns out that both worked that morning and 20+ hours later I was back at home (ahh! the glamor of travel). Now my laptop appears to act a little strange. I better do a backup...
That made me think: Do all Informix DBAs have a contingency plan? What happens if something goes wrong? How much does it cost the business for each hour of downtime?
IDS offers a lot of capabilities that can address the needs of a business environment. It starts with online backup either full or incremental and adds to it through the following:
- Continuous Availability Feature (CAF): This provides the ability to share the disks so instances on different machines (or blade) use the same database space. This is great to quickly recover from machine failure since the database is accessible through another machine. Since all machines access the data simultaneously, it can also provide horizontal scalability
- High-Availability Data Recovery (HDR): Provides disaster recovery through the replication of the data to another instance.
- Remote Secondary Server (RSS): Adds to the HDR story by supporting additional copies of the data into other instances
- Continuous Log Restore (CLR): You can automate the restore of IDS into another instance so that instance is ready to be put online if needed in the case of a disaster on the production machine
- Enterprise Replication (ER): Gives you the ability to distribute the data over hundreds of instances to have distributes work and redundant data.
All these options work together. Talk to your local IBM-Informix IT specialists if you want to know more about these capabilities.
The machines configurations caused problems in using Data Studio with WAS CE, I already mentioned that yesterday. This also meant that we could not do the web services lab. To work around this problem, I spent a few minutes showing the students what was involved in creating a web service using the vmware image on my laptop. Of course, it took a lot less time than would be required to do the lab since everything was already setup.
The rest of the class went well. It included a review of the enterprise features such as backup, SDS, HDR, RSS, CLR, ER, CDC (Change Data Capture), and MQ integration. I think we should add a lab on shared disk and HDR since the labs appear to be very well received. They are more fun than just sitting there listening to a speaker. The class ended with a prsentation on cloud computing.
I went through the evaluation and found that the class was a success. I know there are a few adjustments but it was a good start. All in all, it was a good few days.
I took the train to Paris. It takes around 2 hours 15 minutes to cover the 500 kilometers between Strasbourg and Paris. That's an average of over 220 km per hour. The ride was so smooth. It is interesting to note that a plane ride would have taken one hour but the train is actually faster since you can get there just a few minutes before departure and it drops you off in the middle of Paris instead of the "far away" Charles De Gaulle Airport. That's a reminder that we should always use the right tools for the right problem :-)
Second day of class in Strasbourg. It started well: I covered IDS extensibility and the students went through the lab without more serious problems than misspelling and keyboard issues.
Life was good until we got to Data Studio..
I went through the presentation quickly since all the students said they were familiar with eclipse. We then moved on to the lab. Fifteen minutes into the lab, most students were still waiting for Data Studio initialization to complete. For some, it took much longer. All that to say that the lab machines were under-configured in memory. I should have mentioned that the lab machines needed at least 1GB of memory and not only 512MB.
Despite the memory problems, most student were able to get through the lab. I suspect that it will be impossible to do the lab on web services since we also need to start WAS CE in addition to Data Studio.
We continued with a discussion on OO and databases followed by a review of web environments with more emphasis on application servers. These went quicker since, as expected, students covered the OO approach in detail during their years at the university. this was a good thing since we were behind in my original schedule.
the day ended with a presentation on WAS CE followed with the lab where they were able to see an application that runs in the application server and accesses Informix. Another decent day, now on to the last one.
I just finished the first day of class at the university of Strasbourg. Almost everything went well.
There are 20 students in the class in addition to Pierre Tellier. The class could accommodate 26 people so a total of 21 makes it look quite full. We started the day with an introduction presentation that includes, among other things, the class objectives, an agenda with tentative timing, some background on databases, and a description of the lab environment. Thiswas followed by a presentation discussion consideration on performance as it relates to hardware, operating systems, and databases. In the afternoon, we went through an introduction to IDS which led to our first lab.
After spending an hour telling them how great IDS is and how easy it is to use, the first thing I saw in the lab was that IDS would not come up! It turns out I hardcoded an IP address for the host address that worked fine on my laptop but when executed on the lab machines, the address was wrong. After struggling through finding the character positions on a French keyboard (think '|', '/', '$', etc.) I figured out the problem, provided the solution to the class and the lab continued without problems. I made sure to explain that the problem was a network address, not IDS!
We finished the day with a partial presentation on database extensibility that will continue Tuesday morning.
the students seemed to enjoy the content and the delivery of the presentations and Pierre was happy with it. I'd say that the first day was a success, two days to go. Let's see what happens...
My trip to the University of Strasbourg, in a region of France called Alsace, started the usual way with a trip to the airport and a plane ride of over 10 hours.
Then it got better. Pierre Tellier who coordinated the event at the University drove from Strasbourg to Frankfurt to pick me up. He did not want me to have to wait a few additional hours for a bus. When I got out of custom and he was there waiting for me. We drove around 2 hours back to Strasbourg driving at times at 160km per hour due to traffic. On his way to Strasbourg, Pierre was able to test the maximum speed of his car. It apparently clocked at 220km per hour. Of course, he had to make way for other drivers that really took advantage of the no speed limit of the German infobahn.
Picking me up in Frankfurt is extremely nice. Doing it on mother's day in France sounds like a death wish!
We got to Strasbourg and started doing some site seeing that made me feel like I was on vacation. To top it off, Pierre invited me to his house where I met his beautiful family and had dinner. What a way to start an Informix on campus event. Pierre and Strasbourg get an A-plus in hospitality. they are setting the bar very high for any other Universities in the world that will participate in the program.
the only thing I can say is vive l'Alsace, vive Strasbourg!
You may not know but the Informix lab is extending a helping hand to universities around the world. One example of that was the hosting of university professors at the last Informix conference.
As part of this, I am on my way to the university of Strasbourg (France) to teach a 3-day seminar on subjects related to IDS. I had all the latitude I wanted (and more) to decide on the content. I will be delivering this seminar starting next Monday (June 8). We'll see how it is received. Watch for my blog entries after each day, network access permitting.
I recently met with a consultant friend of mine. He told me that he would like to have a new feature in the database server: a blocking select.
This feature would block on a SELECT until a new row is available. The SELECT statement could also have a condition (WHERE clause). If no rows are returned, the SELECT blocks until it could return at least one row. This way, an application could be event-driven without having to constantly check if there was anything new in the database. You can imagine the overhead of checking if there is something new every few seconds. A blocking SELECT could possibly reduce the pooling overhead and be more responsive depending on what the pooling interval is.
IDS does not have a blocking SELECT and as far as I know, nobody else has it either. I can think of two ways to achieve this:
- Use triggers on the table.
This could be an INSERT and an update trigger. The trigger would test the condition and then execute a user-defined routine (UDR) to send a signal to a process that would tell the process to read the table.
This approach could work when the table does not receive too many INSERTs and UPDATEs. The UDR could potentially find information from a file or from a configuration parameter stored in a table. It could also keep that data in memory to avoid getting the parameter for each trigger execution.
- Use the Change Data Capture API (CDC)
CDC provides an SQL interface to the database logs. You can configure your "log snooping" for a specific table. Each time a new log record is created, the function would return. The application then needs to look at the type of record and decide what to do. This approach should be very efficient and add virtually no overhead to the database server.
Another approach would be to use the InfoSphere Change Data Capture product to replicated specific records on a message queue and have the application subscribe to these messages.
Maybe a blocking SELECT would still be a neat feature to have but with all the different ways we can provide event-driven processing, maybe it is not needed after all.
I was talking to an Informix partner last week. They are upgrading to IDS 11.50.
A partner going to IDS 11.50...What's the big deal? I think it is when the partner is looking at upgrading from Informix SE versions as far back as version 2! And we thought we had problems convincing customers to upgrade from IDS 7.x :-).
Their customers are happy with what they have. It took a long time to convince them that they should move to a more current version of Informix. The customers were just happy with their current systems. Talk about reliability!
Going to IDS 11.50, they are now looking at new approaches to providing reliability in a distributed environment. With shared disk secondaries (SDS), high-availability data replication (HDR), remote secondary servers (RSS), and enterprise replication (ER), they have all the flexibility they need to provide the right solution for their customers availability requirements.
Informix reliability is often taken for granted by Informix customers. Now if the people that are not using Informix could realize the benefit they could get out of IDS 11.50, we could have a database revolution on our hands: set it, forget it, and focus on running the business, not the database system.
I came back from the Informix conference Thursday night and woke up thinking about an analogy about why we use Informix Dynamic Server. More on that in a minute.
I've been using databases for a long time. I believe that the first formal database system I used was back in 1984. It was a hierarchical database. I developed an inventory system for the Canadian Coast Guard. Over the following years, I used and supported multiple databases systems some looking more like C-ISAM and others relational. I still remember the good old days where I had to debug Oracle installation scripts :-)
So, why Informix? Isn't a database a database?
I uses to use a car analogy: people buy cars and they are used to what happens to it: If they have to go to the shop to get it fixed or tunes every other month, that's just the way cars are. Who would believe that you could buy a car and only have to put gas in it for years after years without having to waste time in the shop? the car is used to get you from point A to point B day after day. It almost makes it invisible but not quite since you still have to drive it. It's not the same with a database system: it can really be invisible.
I woke up Friday with this thought: You can write just about any application in any computer language you want. Why don't we all use COBOL. Way back, I know a guy that could do EVERYTHING in COBOL. He was even doing system programming! An object oriented version of COBOL has been available for years buy why. Isn't the "vintage" version of COBOL good enough? If I'm not mistaken, the number of COBOL lines of code in production still surpass any other programming language. That should be enough of an argument to standardize on it.
It seems to me that many people apply this line of reasoning to database systems. The trend is to look at databases as commodity. Who cares that one barely requires any attention? Who cares that it provides easy continuous availability? Who cares that it has great storage optimization? The difference is only more overhead. that translates only into more costs. Those significant costs are easy to hide so why worry about them. Everybody does it so no need to be more efficient...
Well, me, I'm old school. I come from an era where memory was measured in kilobytes and disk drives in megabytes. Yes, memory is much bigger now and not that expensive. Disk drives are so much bigger and not very expensive. Computers are so fast now. It seems to me that we should stop the insanity and pay attention to efficiency. Isn't that what cloud computing, virtualization and being green is all about?
No matter how I try to slide it, to me, Informix is number 1.
This was the last day of the conference with a 35 sessions. I was surprised to see how many people attended the presentations until the end. I see this as a big endorsement of the value provided by these presentations.
On my part I delivered one presentation first thing in the morning and another one starting at 2:10 pm. Despite that, my session was well attended.
Overall a very successful conference that was well worth attending.
The day started with a keynote presentation by Dr. Anant Jhingran on "Cloud computing, databases and the role of IDS". He was assisted by our own Guy Bowerman. That was quite a good start to another great day of learning and networking.
There were 35 sessions covering subjects including Gillani Fourgen case tool, Genero report writer, IDS tasks and sensors, performance tuning, backups, trouble shooting, encryption expert, and index enhancements. Quite a range of subjects and that's not the half of it!
I had interesting conversations with some partners. One of them mentioned how the AGS Server Studio product transformed someone that knew nothing of databases into a database administrator in no time flat. Looks like the ease of use of IDS with the ease of use of AGS is an unbeatable combination. I also had a discussion about collecting and sharing the information about sensors worldwide to monitor the health of the planet. Talk about a stimulating conversation.
There is one more day of this! I don't know how much more I can take :-).
I mentioned the Informix warehouse in my previous entry. There is the chat with the lab coming up. Here's something more: a new tutorial on DeveloperWorks:
Get started with Informix Warehouse Feature, Part 1: Model your data warehouse using Design Studio
Then there are the informix Warehouse product pages:
There were 2 keynote addresses, 25 sessions, 5 bird-of-a-feather sessions, usability labs and demos in the exhibit hall. Then the day closed with an Hawaiian Luau sponsored by Gillani. A packed day to say the least.
All this was topped with the announcement of IDS 11.50.xC4 that includes a new storage optimization feature and the Informix warehouse bundle. Things are moving fast with Informix!
The storage optimization includes compression, repack the spaces that is saved and shrink the dbspace to free the space. This new feature could save 30% to 50% or even more in some cases.
The Informix warehouse bundle include a too that allows you to define your warehouse and define the process of extract, load and transform. There is a chat with the lab schedule for Wednesday, April 29th at 8:30 AM Pacific, 10:30 AM Central, 11:30 AM Eastern, 4:30 PM London, 5:30 PM Paris that will provide more information on the subject.
This week we have the International Informix Users' Group annual conference. It is being held at the Overland Park Marriott.
Here's a bit of trivia for you: In 2008, Overland Park was listed as the 9th best place to live in the US. No doubt having the Informix lab close helped its ranking :-). You can check it at:CNNMoney.com
Sunday was a day of tutorials with eight tracks running at once. I arrived in the early afternoon. It is amazing how quickly you can get into interesting conversations, discussing different projects and business solutions.
The evening reception was a success with lots of networking and good food. This is a great start to what should be a very interesting and useful conference. Monday starts with a keynote from Dr. Arvind Krishna and continues with five tracks on a variety of subjects.