Big data in motion
JacquesRoy 120000A2MS 1,061 Visits
The IOD conference is less than a week away. I received an email about a blog entry that lists all the book signings that will happen at IOD. A total of 10. I happen to be one of them.
I wrote a short book titled: "Informix Dynamic Server Application Development - Getting Started". It is a free book that will be available at the conference. My book signing session is as follows:
Tuesday 12:00 pm - 1:00 pm
Location: Mandalay Bay Registration Desk South
Since I'm giving up my lunch for this, please stop by and say hi. For more information on all the book signings at IOD, please see the following blog entry:
(Short URL: http://bit.ly/KB8zy)
JacquesRoy 120000A2MS 1,055 Visits
Sunday was the beginning of the conference. Even though there were tutorials during the day. The real beginning was the welcome reception. The exhibitors booths were ready and the attendees were cheerful. IT was a great networking opportunity with good food and plenty to drink.
I have to admit I missed a lot during the reception because I was in multiple intense discussions.
All in all, a great start to the conference.
JacquesRoy 120000A2MS 1,037 Visits
Coming back to that PoC I talked about last week, one requirement was to display data in real-time. The data comes in as 22 different pieces of information every 40 millisecond. This translates into processing 25x22 (550) pieces of information every second.
That could be a problem. Imagine having to do a SELECT statement, display the data on the screen, and do that 25 times per second. How long does it take to get the data, format it and display it on the screen? Would I fall behind because of that? I could create a multi-threaded application where one thread reads the data and another displays it.
Here's another way to look at it: How does the data come into the database? Is it coming it every 1/25th of a second? How do I select that last piece of data? What if I execute my SELECT statement just before the data comes in? That means that I get nothing back and have to re-execute my SELECT. If the data comes in using the TimeSeries Real-Time Loader (RTL), I could have RTL send the data directly to the application in addition to putting it in the database. If I don't use RTL, I could consider using the Change Data Capture API to read the logs and process the data that way...
Looking at the implications of a real-time approach is all good but the right question was not yet asked: Why do you need to display data in real-time?
It comes back to requirements and this is one example why it is important to involve database experts in the early stages of application development. Database experts should be involved from day one to review the use cases and detect the database usage implications. Unfortunately, database experts such as DBAs are often involved only when it comes to deploying the application.
Coming back to my "real-time" problem...
In my case, the requirement turned out to be to replay data as if it was in real-time. Quite different from the original request. Just to show what a difference you can make if you get involved in a project early.Till next time...[Read More]
Back in around 1988, I decided it was time for me to learn about object-oriented programming, design, etc.
Learning C++ was not too bad but when it came to define problems in an object oriented way, I started to panic: Had I reached the limit of what I was able to learn? Had I been passed by technology? Was I now a dinosaur? (and you thought the title referred to something else...)
It turned out ok... I think :-)
Today, the rate of change in technology has been accelerating and does not seem to be slowing down. To make things worst for database people, we've been told for years that database are commodities and they are just a persistent storage. I may expand on that later but let's just say for now that I totally disagree.
This being said, why start a blog? For one, I want to communicate with the Informix community in a more continuous manner and on subjects that may not require a one-hour powerpoint presentation. I want to discuss any technologies that is remotely related to databases and I want to start a continuous dialog with the Informix community on any subject of interest.
This blog will be in part educational (I hope) and also a place to discuss business problems and potential approaches to solutions. I believe that DBAs are experts in optimizing database access. It is time to expand DBAs impact in the enterprise to improve data processing. There is no need for a DBA to become a programmer. It is an issue of getting involve in the analysis and design of new applications. For programmers using Informix, let's start talking about what you are trying to accomplish. We may be able to find a better approach specially considering the IDS new features and database extensibility.
Please comment on my blog entries and send me your questions and let's start talking![Read More]
JacquesRoy 120000A2MS 1,011 Visits
I was working on some existing PHP code that accessed a database. I ran into a problem: When using the PDO->fetchObject() method, IDS was returning the column names in uppercase. Since that translated into attribute names in the returned object that was a problem because the application expected these attribute names to be in lower case. Being new to PDO, I came up with the following clever code (sorry about the formatting, I can't figure out how to indent):
$row = $conn->fetchObject();
I was pretty proud of myself for finding about the stdClass and how to add attributes to the class dynamically.
It turns out that there is a much easier way to do it. I only had to set an attribute on the connection:
I'm sure one day I'll be glad I learned about stdClass. For now, I was reminded about reading the relevant section of the documentation...[Read More]
JacquesRoy 120000A2MS 1,010 Visits
Another year, another conference. It has been so busy that I have not had the time to write a short blog entry for each day. Here is my quick update.< /p>
It all started Saturday morning with the business partner council and the customer advisory council on Saturday. I attended the customer advisory council and I found it interesting and full of good discussions.
The conference was kicked off with an opening reception on Sunday night and we were off to the race. There were eight Informix sessions on Monday including presentations on how IBM helps Cisco, open source, hands on lab on high-availability, another one on the new features of Informix 11.70, bests practices for virtual environments, and performance enhancements. Of course, the most popular session was from Jerry Keesee titled: "Informix at IBM: The next decade".
The day ended with an Informix reception at the Mandalay Bay beach casino for an Informix 11.70 launch celebration and to start looking forward to the next decade of Informix at IBM.
Tuesday started early with an Informix "eat and meet" breakfast at 7:00am, followed by nine Informix sessions throughout the day. The sessions covered areas such as upgrade, new features, Informix warehouse, application development, 4GL, embeddability, flexible grid, and more. It was also interesting to hear about how Informix is used to run a steel plant.
The day ended with a beach party reception. Now it is on the Wednesday with another full agenda.
JacquesRoy 120000A2MS 1,005 Visits
Have you kept up with the IDS 11.50 fixpacks? Do you know what new?
Fixpack 2 came out 20-Aug-2008 and fixpack 3 became available 31-OCT-2008. Fixpacks usually fix some problems but often include improvements to existing functionality. Here are some of the things that appeared in the fixpacks:
Go take a look at the release notices to find out more about what changed. You can find the release notices and all the documentation at:
Stay tuned for Fixpack 4 in the next few months![Read More]
JacquesRoy 120000A2MS 1,002 Visits
Carlton Doe has distilled his many years of Informix expertise into a new book. It is titled "Administering Informix Dynamic Server". It covers IDS 11.5 and even talks about IDS on MAC OS X.
Of course, you can get this book on-line or, hopefully, from your local book store. I got my copy a week or so ago and I already used it to check on a few things about IDS 11.5. Great time savings! In addition to the easy to read great content, I actually like the size. Despite being over 400 pages, it is less than one inch thick and feels quite light. I think it will establish permanent residence in my briefcase when I travel.
Here's one more for you: The book is available at the bookstore at the IBM Information on Demand conference (Oct 26-31). For any of you who are going to the conference and have not already ordered this book, you can buy it there.
But wait! If you buy or bring your book to the conference, you could get Carlton to autograph it. He will be available on Monday between 3:00pm and 4:00pm in the main event center hall and again on Wednesday from 1:30pm to 2:30pm.
I'm bringing my copy.
Carlton is not done yet. He is working on a sequel to this book: "Administering Informix Dynamic Server, Advanced Topics". it will be more focused on areas like performance tuning, ER, HDR/MACH-11 and other topics. Something to look forward to next year hopefully.[Read More]
JacquesRoy 120000A2MS 996 Visits
I was joking around about security-related events in a previous blog. It looks like what goes around comes around...
I was in Montreal last week visiting partners and customers. To make it there I went through Los Angeles and had the joy of being selected for additional security screening. It is not your father's additional screening! They did not only go through my bag and looked at all my papers - luckily they did not take the time to read everything or I would have missed my flight - They went through my wallet, looked at all my credit cards and IDs. For the first I was starting to wonder about privacy-violation. They also took the time to go through my passport and look at the different countries I visited.
I'm all for good security measures but I fail to understand why going through my credit cards makes planes safer. In fact, they asked me if this procedure made me feel safer. My parents told me I should always tell the truth...
Over the last few years we regularly heard stories about information leaks. Information like credit card numbers, email accounts with password and so on. The problems are numerous, going from disgruntled employees to lost backups and different security access problems.
If you have needs for security and data privacy, make sure to review what IDS 11 provides. It includes communication encryption, backup encryption, column-level encryption, database roles including security officer, and label-based access control (LBAC). Of course we could add to that the resistance to denial-of-service attacks and other features that can be sued to assist in securing your database system.
Make sure you discuss your security and privacy needs with your IBM-Informix technical specialist.
JacquesRoy 120000A2MS 993 Visits
I was reading recently that scripting languages are becoming more and more popular. One of the benefits listed was that it can increase productivity significantly. That brings me back a long time when I used to argue that you could write a solution in APL so much faster than any of the traditional languages...
PHP is a popular scripting language that is used to create web solutions. Here are some information if you want or need to use it with IDS.
To use IDS with PHP, you need to add the PDO_INFORMIX interface to it. tis interface can be found in PECL:http://pecl.php.net/package/PDO_INFORMIX
The current version is 1.2.0 and came out in March 2008. You will to use the appropriate Informix CSDK to compile it for your platform.
Note that there is another PHP driver available for IDS 11: PDO_IBM. It is available from IBM in the IBM Data Server Client as a pre-built component. See:https://www14.software.ibm.com/webapp/iwm/web/reg/pick.do?lang=en_US&source=swg-idsc11
If you want to get going quickly, you can find Apache-PHP-PDO_INFORMIX ready to go with the Open Admin Tool for IDS (OAT). This product is available for IDS 11. You can find it at:https://www14.software.ibm.com/webapp/iwm/web/reg/download.do?lang=en_US&cp=UTF-8&S_PKG=dl&source=swg-informixfpd
You can also use the package from Zend: Zend Core for IBM at:http://www.zend.com/en/products/core/for-ibm
If you want to build the environment yourself, here are two articles that can help you figure out the process:
With this information, you should be able to get started quickly[Read More]
JacquesRoy 120000A2MS 992 Visits
Informix often adds features in fixpacks and xC2 and xC3 are no exception. I strongly suggest that you take a look at the list of new features that are listed in the release notice. You can find it at:
Release notice 11.70.xC3
In my next few blog entries I will not cover all the new features. I will limit myself to two main areas:
For anything else, see the release notice and the Informix documentation. The easier way to do this is to use the information center that can be found at:
This release added a few compatibility function that makes it easier to move application to informix. They include:
Take a look at the details of these functions. I'm sure you will find a good use for them.
JacquesRoy 120000A2MS 984 Visits
Continuing from the last entry...
Out of these 197,000 entries, here are some more that may be of interest:
I'm sure we can find a lot more of interesting sites. There is a lot about IDS on the web[Read More]
IOD 2008 is now over. as I mentioned in my blog during the conference, there was a lot of interesting content. And that was just in the Informix track. Many other sessions in other tracks are relevant to Informix since we can use Informix with the other Information management products. Now, if I only had the time to go over all the sessions to see what's interesting...
Overall, this was a great conference to attend for the sessions and for the interactions with customers, partners, and IBMers.
The next MUST ATTEND event for me is the 2009 IIUG Informix Conference, April 26-29 in overland Park, Kansas, USA. By then I am sure we'll have a lot of interesting news to discuss![Read More]
Another day at IOD.
I did not mention Jerry Keesee's roadmap presentation from yesterday. Praising his presentation seems too self serving :-)
Today, I attended a presentation from a company called Finish Line. They manage 700 stores. Their IDS installation includes 5 servers, 8 instances, 28 databases with a total of 2TB of data. They use 4GL for their backend processing and use SOA to provide a single view of the business environment. One of the benefits of using SOA is that they can keep track of all inventory in all stores and, if needed ship merchandise from one store to a customer. This effectively gives them the capability of 700 distribution center.
They keep looking for ways to make their environment better. IDS is the cornerstone of this strategy.
The presentation started with the top 10 things that their one DBA does:
Another party in the evening... nights are short...
Cows can detect odors up to 5 miles away.
You can learn all sorts of interesting facts at IOD...
I went to a presentation titled: "Taking replication to the terabyte level at Fonterra". They are a dairy company that exports milk to more than 140 countries and territories worldwide. They recently completed a project with the help of the oninit IBM Informix partner.
The bottom line is that they have 20 IDS leaf nodes that feed 1.2GB of data daily to a central IDS node. Since they are planning to keep 7 year of data, that adds up really fast. They currently have only 5 years of data...
Other notable presentations: Frederick Ho's musing on data warehousing and IDS, IDS helping the American Forests non-profit organization in reforestation (our partner Advanced Data Tools has been working with them for years).
More Tomorrow.[Read More]
JacquesRoy 120000A2MS 931 Visits
A lot of people will have some time off in the next few weeks. Maybe it's a good time to relax and read something that is not directly related to our day to day work. Here is a suggestion for easy and fun reading:
The Pleasure of Finding Things Out
Richard Feynman was a Nobel Prize winning physicist that has had an interesting live. If you have never heard of him, you should read this set of short stories. It is fun and motivating. Just what's needed to relax during a vacation.
Till next time[Read More]
JacquesRoy 120000A2MS 923 Visits
With all the excitement of IOD, I did not notice the release of 11.50.xC3 on November 5, 2008.
In this fixpack, you will find several new features:
You can find the details in the release notice at:http://publibfp.boulder.ibm.com/epubs/html/i1190812.html
JacquesRoy 120000A2MS 915 Visits
Talk about busy times. You probably saw Guy's blog entry on the redbook we are working on. It is still early in the redbook but it is very promising. One thing I'm working on is describing the tasks and sensors. If you have any ideas on how these could be used, let me know. I've already gathered great information on it and I am excited about the possibilities.
Talking about redbooks, have you seen the latest two redbooks:
SG24-7488-01, Redbooks, published 24 June 2008
This book provides an overview of the high availability and enterprise replication features (ER, HDR, RSS, SDS).
SG24-7522-00, Redbooks, published 11 June 2008
This book provides an overview of some of the capabilities of IDS version 11 that enable it to be easily customized for your particular environment. Although many capabilities are available, the focus of this book is on the areas of ease of administration and application development. It describes and demonstrates these capabilities with examples to show how it can be done and provide a model as you begin your customization.
The one before those came out in March and is about security. To see more, go to the following URL:
Also, the IOD conference is just around the corner. We are all busy preparing our material for it.
And that's just in addition to our normal work of talking to customers and getting IDS into more hands!
Until next time![Read More]
Thursday was a busy day for me. I was giving two presentations. One on Java best practice with IDS and another one about the Retail Integration Framework with WebSphere Remote Server. I also had one hour scheduled for "meet the expert".
I also attended a very interesting presentation given by Andreas Weininger from IBM Germany. He told us a project that implemented the MACH 11 features in a banking environment. As you can imagine, failure is not an option in that context. the implementation included a lot of redundancy with a primary, an HDR secondary and six shared disk secondaries. The SDS nodes were used to feed a number of Linux servers used for number crunching. The implementation was a success. It went from concept to production in 3 months!
The conference in winding down. It will be time to go home soon. It's been great but I'm ready.[Read More]
It seemed so far away and now it is almost here: IOD 2008 in Las Vegas October 26 to October 31. There are even some meetings on October 25th.
I'll be there of course. I am delivering two presentations:
Thursday Oct 30, 8:30am - 9:30am, Mandalay Bay - Tradewinds C
Thursday Oct 30, 10:00am-11:00am. Mandalay Bay - Coral A
I also participate in the "Meet the experts" sessions. These sessions occur almost constantly throughout the conference.
There are a lot of good sessions happening that week. There is also the benefit of meeting so many knowledgeable people. That will be a tough week but it is worth it![Read More]