Big data in motion
JacquesRoy 120000A2MS 1,524 Views
I remember seeing something like this title in some Informix marketing material many years ago. I think it was related to the fact that IDS has extensibility features that allow developers to adapt IDS to their business requirements as the technology and needs evolve.
The "future built-in" idea came back to me as I was reading a computerworld article titled: "the desktop traffic jam" (see: http://www.computerworld.com/s/article/342870/The_Desktop_Traffic_Jam). In there they were talking about a new feature in Windows 7 (User Mode Scheduling) that lets thread multiplexing take place in the application instead of in the kernel. They go on to say: "Handling this multiplexing in the application instead of in the operating system kernel makes thread scheduling more efficient.
I know it's not quite the same but it is similar to the idea that IDS decides its thread scheduling, making it more efficient since it is will not re-schedule a thread that is in a critical section of code. This way it avoids having threads that get scheduled to find out that they have to wait. All that making the threading model more efficient. I wonder how difficult it would be to take advantage of thread multiplexing onto cores. Could it be as simple as having one CPU VP per core wih some "core affinity" of the operating system supports that? Then IDS would already be there... with the future built into it.
JacquesRoy 120000A2MS 1,523 Views
Carlton Doe has distilled his many years of Informix expertise into a new book. It is titled "Administering Informix Dynamic Server". It covers IDS 11.5 and even talks about IDS on MAC OS X.
Of course, you can get this book on-line or, hopefully, from your local book store. I got my copy a week or so ago and I already used it to check on a few things about IDS 11.5. Great time savings! In addition to the easy to read great content, I actually like the size. Despite being over 400 pages, it is less than one inch thick and feels quite light. I think it will establish permanent residence in my briefcase when I travel.
Here's one more for you: The book is available at the bookstore at the IBM Information on Demand conference (Oct 26-31). For any of you who are going to the conference and have not already ordered this book, you can buy it there.
But wait! If you buy or bring your book to the conference, you could get Carlton to autograph it. He will be available on Monday between 3:00pm and 4:00pm in the main event center hall and again on Wednesday from 1:30pm to 2:30pm.
I'm bringing my copy.
Carlton is not done yet. He is working on a sequel to this book: "Administering Informix Dynamic Server, Advanced Topics". it will be more focused on areas like performance tuning, ER, HDR/MACH-11 and other topics. Something to look forward to next year hopefully.[Read More]
JacquesRoy 120000A2MS 1,520 Views
The IOD conference is less than a week away. I received an email about a blog entry that lists all the book signings that will happen at IOD. A total of 10. I happen to be one of them.
I wrote a short book titled: "Informix Dynamic Server Application Development - Getting Started". It is a free book that will be available at the conference. My book signing session is as follows:
Tuesday 12:00 pm - 1:00 pm
Location: Mandalay Bay Registration Desk South
Since I'm giving up my lunch for this, please stop by and say hi. For more information on all the book signings at IOD, please see the following blog entry:
(Short URL: http://bit.ly/KB8zy)
Another day at IOD.
I did not mention Jerry Keesee's roadmap presentation from yesterday. Praising his presentation seems too self serving :-)
Today, I attended a presentation from a company called Finish Line. They manage 700 stores. Their IDS installation includes 5 servers, 8 instances, 28 databases with a total of 2TB of data. They use 4GL for their backend processing and use SOA to provide a single view of the business environment. One of the benefits of using SOA is that they can keep track of all inventory in all stores and, if needed ship merchandise from one store to a customer. This effectively gives them the capability of 700 distribution center.
They keep looking for ways to make their environment better. IDS is the cornerstone of this strategy.
The presentation started with the top 10 things that their one DBA does:
Another party in the evening... nights are short...
JacquesRoy 120000A2MS 1,501 Views
Have you kept up with the IDS 11.50 fixpacks? Do you know what new?
Fixpack 2 came out 20-Aug-2008 and fixpack 3 became available 31-OCT-2008. Fixpacks usually fix some problems but often include improvements to existing functionality. Here are some of the things that appeared in the fixpacks:
Go take a look at the release notices to find out more about what changed. You can find the release notices and all the documentation at:
Stay tuned for Fixpack 4 in the next few months![Read More]
IOD 2008 is now over. as I mentioned in my blog during the conference, there was a lot of interesting content. And that was just in the Informix track. Many other sessions in other tracks are relevant to Informix since we can use Informix with the other Information management products. Now, if I only had the time to go over all the sessions to see what's interesting...
Overall, this was a great conference to attend for the sessions and for the interactions with customers, partners, and IBMers.
The next MUST ATTEND event for me is the 2009 IIUG Informix Conference, April 26-29 in overland Park, Kansas, USA. By then I am sure we'll have a lot of interesting news to discuss![Read More]
JacquesRoy 120000A2MS 1,498 Views
Sunday was the beginning of the conference. Even though there were tutorials during the day. The real beginning was the welcome reception. The exhibitors booths were ready and the attendees were cheerful. IT was a great networking opportunity with good food and plenty to drink.
I have to admit I missed a lot during the reception because I was in multiple intense discussions.
All in all, a great start to the conference.
JacquesRoy 120000A2MS 1,485 Views
Last week, on October 22, IBM announced a new version of InfoSphere Streams: version 3.2.
The new version includes some nice improvements such as remote development, Rest API for data access, and improved toolkits.
If you are interested in trying Streams, IBM provides the quick start edition that you can download as native product or
Of course, you may need more information on how to use Streams. You can start by browsing through the InfoSphere Strreams Playbook at:
If you have questions, don't hesitate to drop me a note or comment on my blog entries.
Until next time!
JacquesRoy 120000A2MS 1,482 Views
We had many good presentations on Monday and of course several impromptu meetings all over the place. Time is running short so I have to keep this entry to a minimum.
In One session I heard about Choice Hotel that has 6000 properties in 10 different brands. They strongly depend on IDS to run their business. In another one, I heard about peapod online grocer that also relies on Informix IDS to run their business. Finally, we also heard about the new IBM system x bundles for Informix where they provide tested configurations in "T-shirt" sizes (small, medium, large, X-large) to fit any businesses. They mentioned that such a configuration showed much better price performance than a Sun system running Solaris.
More to come on Tuesday.
It seemed so far away and now it is almost here: IOD 2008 in Las Vegas October 26 to October 31. There are even some meetings on October 25th.
I'll be there of course. I am delivering two presentations:
Thursday Oct 30, 8:30am - 9:30am, Mandalay Bay - Tradewinds C
Thursday Oct 30, 10:00am-11:00am. Mandalay Bay - Coral A
I also participate in the "Meet the experts" sessions. These sessions occur almost constantly throughout the conference.
There are a lot of good sessions happening that week. There is also the benefit of meeting so many knowledgeable people. That will be a tough week but it is worth it![Read More]
JacquesRoy 120000A2MS 1,461 Views
We're up and going.
The conference is still being setup but there are events happening this Saturday.
All sorts of other sessions are taking place in other areas of the Mandalay Bay convention center.
If you are already in Las Vegas for the Insight conference, this would be a good use of your time.
Finally, Sunday evening, the Insight conference officially starts with the Solution EXPO Grand Opening Reception
I'll post comments on the conference daily so, stay tuned!
JacquesRoy 120000A2MS 1,450 Views
The event went as planned at the Mandalay Bay convention center with presentation on:
Many people attended and were engaged in the presentations. Overall a success.
The Insight conference officially started with the opening reception.
JacquesRoy 120000A2MS 1,449 Views
The general session started with an example of context computing and an interview with Captain Phillips.
All that was pretty exciting but what stole the show is the announcement of the partnership
Then I went on my way to attend Streams sessions talking about use cases.
The first one i attended is about a partner, Voci, that has a appliance that converts audio to text.
The next session was a panel of expert on geospatial analytics.
In the afternoon, I attended a session on the features of the new Streams beta that was announced last Friday.
I followed with a session on context computing used to counter fraud. I finished my day
The conference is winding down with the last day tomorrow.
JacquesRoy 120000A2MS 1,445 Views
When we talk about processing data in real time, it is easy to just write a program and be done with it.
A program is easy to write when it can process records sequentially. Once you reach the limit of this sequential processing, you start adding complexity that may represent the bulk of your work: You start by using multi-threading and eventually you need to also go to multi-processing to take advantage of multiple machines. It is much easier to use a framework to reduce those issues.
Still, a framework may give you the ability to distribute your processing but how easy is it to do? Now you want proper tools to assemble the many operations that you want to link together. Then, you also need to have the tools to easily identify bottlenecks so you can parallelize you operations. What about all the standard operations you would expect to be able to do?
This is where a platform comes in. It gives you the foundation for distributed processing but also gives you pre-built capabilities to interact with the outside world (files, message queues, databases, and so on) and also analytics so you don't have to reinvent the wheel.
Cows can detect odors up to 5 miles away.
You can learn all sorts of interesting facts at IOD...
I went to a presentation titled: "Taking replication to the terabyte level at Fonterra". They are a dairy company that exports milk to more than 140 countries and territories worldwide. They recently completed a project with the help of the oninit IBM Informix partner.
The bottom line is that they have 20 IDS leaf nodes that feed 1.2GB of data daily to a central IDS node. Since they are planning to keep 7 year of data, that adds up really fast. They currently have only 5 years of data...
Other notable presentations: Frederick Ho's musing on data warehousing and IDS, IDS helping the American Forests non-profit organization in reforestation (our partner Advanced Data Tools has been working with them for years).
More Tomorrow.[Read More]