Managing the data lifecycle
Is "agile data" just another buzzphrase? Does it even make sense to try to apply agile development principles to the database?
An expert in agile development, Scott Ambler, sees agile data as an essential component for application development that goes against a database. You can learn more about agile data here: http://www.agiledata.org/
I think one of the classic challenges that agile data faces is about dealing with a "brittle" database. What do I mean by brittle? Basically, I am talking about how difficult and time consuming it can be to refactor the database schema to improve software. Check out the results of this survey question: "How long does it take to safely rename a column in a production database?"
Source of this survey:
Source: Data Quality Techniques survey by Ambysoft, September 2006.
The database and/or your software development techniques around the database are "brittle" if it takes longer than one week to make a simple rename change. Almost half of these respondents fell into that category. I would venture to say that more interesting refactoring would therefore take most shops much longer than a week.
Another part of the agile data challenge is about being able to quickly tell what the impact of a change is going to be. If we want to rename a column, what are all the database objects (tables spaces, views, stored procedures, etc ...) that will be impacted, and is there a tool to help me automate a script to make these changes?
If this sounds interesting to you and you want to learn more about agile data and how Data Studio can help, come listen to a replay (until May 09) of a webcast I did last week on how Data Studio can help make data more agile.
If you listen to the replay or are exploring agile data I am very curious to get your feedback. Just call me an agile guy. What do you think of applying agile techniques to the database? Are you doing it? If so, what is your experience? What tools are you using? What tools do you need?
What do you think?
-- Rafael Coss
New Data Studio releases bring us one step closer to realizing the integrated data management vision
IBM_Optim 27000269HS Tags:  administrator cotner announcements purequery performance_expert 2 Comments 6,662 Views
It's been less than 5 months since we announced our 1.2 releases of Data Studio, which I blogged about back in July.
Since then, we have talked to thousands of people, provided demonstrations to hundreds, and visited dozens of customers. People are starting to understand Data Studio and the value of Integrated Data Management better.
With this latest release, announced today, we are really targeting the DBA with enhancements across the portfolio to help DBAs improve application performance, security, manageability, and TCO. In this release, the enhancements are particularly targeting Java applications that access DB2 data, but you'll see we're starting to branch into .NET as well.
The announcements today are for:
Data Studio Administrator 2.1, in which we've really focused on both usabilty and functionality. We've done lots of usability testing with DBAs and have provided a more natural approach for doing many tasks, including copy and paste of database changes, flatter traversal of the data source explorer, better sorting and filtering of objects, and new task assistants for utilities, commands and configuration parameters, so you won't have to leave your environment to go out to the command line or control center to perform those tasks.
Data Studio Developer and Data Studio pureQuery Runtime 2.1, which extends the power of pureQuery for developers and DBAs to collaborate together to:
If you extend DB2 Performance Expert with the Extended Insight feature (separate PID and separately priced but prereqs DB2 PE), you can enable new end-to-end database monitoring for Java applications for DB2 servers on Linux, UNIX, and Windows. This monitoring capability will really help improve availability of mission-critical database applications by making it much easier to detect performance issues and figure out whether the problem is one in the database or somewhere else in the software stack.
Also, you can set thresholds (your SLAs, so to speak) so you can easily see how the application is performing against those targets. If you haven't read it yet, I encourage you to see the article that the Germany team who develops this feature wrote. It's a great introduction to this new capability, and it's really just our first step. This whole concept of providing greater insight to DBAs and developers is planned to be rolled out across more databases and more data access environments.
Just a head up. We're not done. We have more announcements coming soon!
Ever since I bloggedmy experience with Shell Sharing, people (ok, that will be exactly one) startedthinking I were some sort of expert and asking me questions. Luckily, Ihave found a newly published Shell Sharing article (written by real experts mayI add) on IBM developerWorks that should have all the answers. To save you thetime for searching, you can get it from http://www.ibm.com/developerworks/db2/library/techarticle/dm-0811khatri/index.html.
This article uses DSD 1.2 as an example, which is just a bitdifferent from my previous experience with Data Studio 1.1.2. First of all, thedefault installed directories have been changed to:
1) Installation directory:C:\Program Files\IBM\DSDEV1.2
2) Shared ResourcesDirectory: C:\Program Files\IBM\DS12Shared
The default package name has also changed to “IBM DataStudio” instead of the more general “IBM Software Development Platform” in lastrelease.
I then followed the instructions to download a trial versionof Rational Data Architect (RDA,) and used Installation Manager to install itas Shell Shared with Data Studio. The only funny thing was “InstallationManager” displayed a message that a newer version of “Installation Manger” mustbe installed in order to continue. After I clicked OK, it just went ahead andinstalled a new version of itself. Don’t you wish every product would upgradeitself like this?
Here I’ll share a secret: developerWorks has a special spacefor “Data Studio” at http://www.ibm.com/developerworks/spaces/datastudio.Click on the “Trials and downloads” tab will not only bring you to the trialdownload page, but it will also tell you what will happen to you after thetrial period ends. You may also find thefollowing information useful:
The following products can shell-share with Data StudioDeveloper Version 1.2.
Data Studio Developer 2.1 has been announced and will beavailable soon. Since it’s an Eclipse 3.4 based product, it is safe to assumeit won’t shell share with the Eclipse 3.2 based products. My secret source toldme one nice improvement is there will be a common splash screen with a list ofthe products being shell-shared upon launch. I can’t wait to try it out.
-- Michael Hsing
What if there were no walls between the DBA and developers?
I'm not really talking about your cubicle walls but those implicit barriers we create when we are so focused on our own thing and work using our own tools. But to develop higher quality applications with more agility, we all need to work together and even cross-pollinate our skills more.
I'm the Team Lead for the Data Studio Developer tooling. Earlier this year, with Data Studio Developer 1.2, we provided tools that help teams break down barriers between developers and DBAs:
As always, if you use DB2, you can use static SQL to reduce CPU consumption in some cases. With this release, you'll find the experience from developing static SQL applications to deploying them to be significantly improved.
I've written more on developerWorks about What’s new and exciting in IBM Data Studio Developer 2.1.
BTW: I sincerely appreciate the feedback that some of you provided during my IOD sessions where I previewed Data Studio Developer 2.1. Your feedback is really important as we build our new releases.
Watch this space for the announcement of the Data Studio Developer 2.1 download shortly... Then let me know either here or on the Data Studio Forum what you think of the new release.
I wanted to bring your attention to an article that was recently published on developerWorks that describes the pureQuery / OpenJPA integration that I discussed in my earlier blog post, which is a new feature in WebSphere Application Server v7 that enables developers to generate SQL from their JPA application entities and named queries, which can then be bound into static DB2 packages, providing a fast path to the security and performance benefits of static access for DB2 data. In addition, when you use this capability, your JPA app can take advantage seamlessly of optimizations provided in pureQuery such as the ability to update multiple tables in a single network call.
This integration is our first step toward providing a strong, integrated stack among DB2 and WebSphere, using pureQuery as the "glue". (As an aside, the Performance Expert Extended Insight Feature also uses pureQuery to provide new insights into the interactions between Java applications and DB2, with the most capabilities being provided for Java applications in WebSphere.)
From a tooling perspective, this initial JPA/pureQuery integration is fairly light, but you will notice that the static binder utility is now invocable from the WebSphere Application Server console, so WebSphere admins won't have to switch to another tool to do the bind. In addition, you'll be able to use the new capability in Data Studio Developer 2.1 to visualizeelapsed time for SQL statements directly from the pureQuery outline.
Nevertheless, there is still much more we can do to make this process easier and less command-line driven. WebSphere plans to ship enabling technology that will "turn on" capability in Data Studio Developer 2.1 to invoke the wsdb2gen utility. In addition, you'll be able to use the output from wsdb2gen within Data Studio Developer and take advantage of other pureQuery outline capabilities, including the ability to correlate SQL statements with specific OpenJPA queries and the relations between the SQL and the associated tables and columns.
Check out the article if you get a chance.
-- Steve Brodsky
Same product, same features, same organization, different name
I often say the above phrase when I have to explain to everyone what the difference is between 'RDA' and 'IDA'. We announced the renaming of IBM Rational® Data Architect to IBM InfoSphere™ Data Architect today, December 16, 2008. See the announcement letter. It really is the same product, still part of the Data Studio family, and even built on top of the same Eclipse level.
So why the name change now?
The name change features the InfoSphere Data Architect role in IBM InfoSphere Foundation Tools, an open set of tools that help prepare an organization to adopt an information agenda. Read more about the Foundation Tools in this executive brief.
There has always been integration between our data architect product and the InfoSphere Family and its predecessors. The very first release featured function to assist in data integration design, but at that time the DB2 Information Integrator branding was too limiting for our offering. Given the broad database support, we opted to give it a Rational brand featuring its integration with the Rational Software Development Platform. InfoSphere Data Architect still is and will continue to be fully integrated with the Rational portfolio, and in particular with the architectural components including Rational Software Architect, WebSphere Business Modeler and other Rational products.
What exactly is InfoSphere?
I was talking with one of the InfoSphere reps to about how he explains InfoSphere to clients when I was at the IOD conference in Las Vegas this year. He started to explain to me about moving. When I say "moving" I mean like you've bought a new house and need to physically move your items from one location to another. So let's describe the process of what you do when you move.
Same product, same features, same organization, different name
-- Anson Kokkat
Howdy! In case you missed it, we just announced a new release of HPU (High Performance Unload) for DB2 for LUW... V4.1. In case you've never looked at our HPU products (for DB2 for z/OS and DB2 for LUW), they can be great productivity enhancers and possibly even save you some resources.
One of the great things about HPU is that it has both a utility-like interface and an SQL interface. The SQL interface is perfect for application developers since they aren't used to invoking utilities. Once invoked, HPU can access the underlying table space or backup / image copy directly, producing multiple data type conversions and unload file formats suitable for most any target data store. When extracting a high volume of data in this way, or by sampling the source, the elapsed time and CPU savings are humongous versus using SQL (or Export or DSNTIAUL).
HPU for DB2 for LUW is also partition-aware, allowing you to unload from multiple partitions with a single execution of HPU into a single output file/pipe or multiple files/pipes. It also provides a re-partitioning capability that unloads and re-partitions the output for new data distribution on the same or different system.
The hot new feature in the 4.1 release for DB2 for LUW adds the ability to migrate data directly (unloading, transferring, and loading) from one database to another without the need for intermediate disk storage. This capability delivers the fastest way to migrate your data. The new release also has other usability improvements and now supports Windows 64-bit platforms.
For more information on HPU, visit http://www.ibm.com/software/data/studio/high-performance-unload/
-- Bryan Smith
IBM_Optim 27000269HS Tags:  optimization_expert client_optimization purequery demo z/os bui 4,481 Views
Just want to let you know that we recently posted another demo for Data Studio on DemoZone. This is a two-part, scenario-based demo focused on a DB2 for z/OS environment. In part 1, we wanted to show some cool features available in DB2 Optimization Expert for z/OS for tuning and optimizing query performance. The second part is an extension to the previous demo set, with details of steps in setting up and using the client optimization feature in Data Studio Developer and pureQuery Runtime for performance stability. Let us know your comments on this video and what specific demos you’d like to see.
I found DB2 Optimization Expert for z/OS easy to use. Of course, when you’re trying to create a demo that shows off the features, it can be challenging. We wanted to come up with a query that would enable us to get tuning recommendations from all the advisors in the product to showcase its capabilities. That was a bit difficult but we managed to do it. You can read more about the various advisors in Ray’s blog entry.
DB2 Optimization Expert is currently available only for z/OS. As query tuning capabilities are a key skill for all database professionals, you might expect to see more database platforms supported in the future.
In addition to the demos on DemoZone, we are posting other less formal, more feature-oriented demos on Channel DB2. In particular, there is a series of demo for new features in Data Studio 2.1 including What’s New for Data Studio Developer and DB2 Performance Expert Extended Insight Feature.
We’re looking forward to hearing from you – just either add a comment to this blog or send an email to email@example.com.
Hi, I belong to the DB2 Performance Expert development team, and one of my roles is to support customers and help them to get the most out of the product.
DB2 Performance Expert for Linux, UNIX, and Windows 3.2 and the new DB2 Performance Expert Extended Insight Feature for end-to-end database monitoring of Java applications is now available. The Performance Expert Extended Insight Feature gives DBAs a new view into the performance of Java database applications. It helps to:
It’s sometimes difficult to explain the real value of what we call end to end database monitoring in words (although you can read about it in this article), so I wanted to use this as an opportunity to introduce you to a short demo that I think is worth more than a thousand words.
Thuan mentioned this video in Monday's posting. It's a live demo given by the lead architect of DB2 Performance Expert, Torsten Steinbach. In the demo you learn how easy it is to identify SQL statements that are responsible for a bad response time and to identify who issued the SQL statement and where the SQL statement spent it is time ( application, driver, network or data server ). Take a look at the video to get a first impression on this great new feature of DB2 Performance Expert.
Please let me know by adding a comment to this blog or sending an email to firstname.lastname@example.org if you have any specific questions about DB2 Performance Expert you would like me to address in future blog entries.
Hi all! If you have been reading this blog for a while, you must remember Steve Brodsky’s entry on pureQuery and pureXML. If you are a new reader of this blog or don’t remember Steve’s post, you can find it here.
In his entry, Steve described how both technologies were born and why both of them got the “pure” in the name. He finished the post by describing some of the integration points between pureQuery and pureXML. That’s where I jump in! Motivated by his post, I decided to create some code snippets to show you how you can plug pureQuery and pureXML together to create Java applications that persist data into a DB2 pureXML database. My initial plan was to put that in a blog post, but as I started writing it down, more ideas were flowing in my mind than I could actually fit a single blog entry, so I decided to work on a more complete article.
The article contains the code samples (available for download) that will get you started developing with pureQuery and pureXML, but its main focus is on the different approaches that one can use when developing such applications.
In a typical application development scenario with three layers – SQL, data access API and business logic - I suggest three different approaches to handle the XML data, each one focusing on a different layer. There are certainly more approaches you can use, and you can even mix and match some of them, but my main goal was to get you started with these two great technologies and to open your mind to different ways of thinking when it comes to integrating XML into your Java applications.
Without further ado, here are the approaches I suggest in the article:
I hope you find it a good read! We continuously get questions from you, our customers, regarding this topic, so I hope I have answered them. After you are done reading it, go play with pureQuery (you can get it by downloading the trial version of Data Studio Developer) and pureXML yourself and make sure you give us your feedback, either here on the blog or on the Data Studio forum.
-- Vitor Rodrigues