Table or Range partitioning is a powerful feature of DB2 that facilitates good database design principles that can help lead to easier maintenance operations, increased data availability and more optimized queries.
But why is table partitioning so good in a warehousing environment? Here are some reasons:
1. Range-specific maintenance operations
Where data partitions (ranges of data within a table) are placed in individual table spaces, maintenance operations can be targeted at active data only.
Many DB2 commands, for example REORG and BACKUP, can be specified to execute against specific table spaces or data partitions. This can significantly reduce maintenance times.
2. Data lifecycle management
Table partitioning large fact tables by date means that older data in a table can be detached as an online operation. This can help eliminate the need to perform costly delete statements.
In addition, as data ages, it can be moved as an online operation to less costly storage. In db2 v10 this multi-temperature data management is facilitated by the new storage groups feature.
3. Partition elimination
Range Partitioning benefits queries where the query spans one or a subset of the range partitioning key.
The DB2 optimizer can then eliminate entire data partitions from the query, and this reduction in rows/read (I/O) can help increase query performance.
4. Local indexes
Local indexes can help to significantly reduce index maintenance and increase query performance where significant sorting is not required.
In addition, local indexes can be placed in separate table spaces which provides more flexibility in building a backup schedule and a recovery strategy.
For example, in a restore scenario you have the choice between restore or rebuild of indexes.
5. Backup performance
Backup performance can be improved by backing up just those table spaces (table ranges) that are active.
By balancing the average size of your table spaces, parallelism within the BACKUP operations can also increase, helping to reduce the elapsed time of your backup operations.
These and other warehouse design issues are discussed in our many papers on warehousing. If you have any comments or experiences you would like to share with the authors, please leave a comment.
We have just published a new best practices paper for IBM Smart Analytics System and IBM PureData System for Operational Analytics customers: Performance monitoring in a data wartehouse.
This best practices paper covers real-time monitoring of the IBM Smart Analytics System and IBM PureData System for Operational Analytics. You can apply most of the content to other types of clusters of servers running a data warehouse system with DB2 software and database partitioning under AIX and Linux operating systems. The focus of this paper is finding the reasons for performance problems. These can be bottlenecks that are in the operating system, are in the DB2 database software, or are related to a single query. The focus is on data warehouse systems with long-running queries rather than transactional systems with mostly short queries.
A main goal of this paper is to provide a set of key performance indicators (KPIs) or metrics for the operating system and DB2 software, along with a methodology for analyzing performance problems in a distributed DB2 environment. This paper describes scenarios to help you gather the right information depending on the symptoms of the performance problem.
This paper first provides an overview of the approach and what to consider in general when monitoring the performance of a data warehouse. It then describes the most important operating system and DB2 metrics for multiserver data warehouse systems. The last section describes in detail several performance problem scenarios that are related to data warehouse or BI workloads and explains how to use the metrics for analyzing the problems.
Most of the information about KPIs that are described in the paper has sample commands that extract actual values. However, these examples are not intended to provide comprehensive tooling. You can use this best practices paper as a guideline for selecting the metrics to focus on when using monitoring tools such as IBM InfoSphere® Optim™ Performance Manager.
You can use DB2 silent installation and uninstallation to install or uninstall DB2 products and components without user interaction. Silent installation is useful for large-scale deployments of DB2 product editions. It is also useful when you need to embed the DB2 installation and uninstallation processes within the installation process of solutions that include DB2 products.
This paper covers the following tasks:
- silent DB2 installation and uninstallation
- silent DB2 fix pack updates
- silent DB2 upgrades to later product versions.
In addition to providing detailed recommendations for each task, the paper also includes practical scenarios to help you implement silent DB2 installation and uninstallation in your own environments.
Share your impressions and questions about this paper by adding a comment on the paper's web page: https://ibm.biz/Bdx8Hr
You will need to login to developerWorks with your IBM ID first.
Modified on by sboivin
IBM Knowledge Center Open Beta is available!
We are very happy to announce the availability of our open IBM Knowledge Center Beta, live on ibm.com.
You can access IBM Knowledge Center here:
The Beta will run until the end of February 2014.
Improving your technical content experience
IBM Knowledge Center is our new technology designed to bring IBM's technical publications together in a single location, and will replace our individual IBM Information Centers.
For this release, we simplified the user experience, improved search, and refined the overall experience with many other enhancements. As always, you can get help on IBM Knowledge Center from the information icon in the upper right corner of the pane (also linked here):
Send us your feedback!
After you've worked with IBM Knowledge Center, sign in with your IBM ID and take a few moments to complete the survey on IBM Knowledge Center located here:
Known Beta limitations
We are still:
Fine tuning IBM Knowledge Center, so you might experience some minor functional issues
Configuring and adding content to IBM Knowledge Center, so the content you see might not be exactly what you expect
Configuring and indexing content for search, so search results might not be exactly what you expect, or might not be in all the languages you expect.
We are happy to announce the publication of a new best practices paper to help you understand and get the most from the new BLU Acceleration technology introduced in DB2 for Linux, UNIX, and Windows V10.5: Optimizing analytic workoads using DB2 10.5 with BLU Acceleration (https://ibm.biz/BdDrnq)
BLU Acceleration is a new collection of technologies for analytic queries that are introduced in DB2 for Linux, UNIX, and Windows Version 10.5. At its heart, BLU Acceleration is about providing faster answers to more questions and analyzing more data at a lower cost. DB2 with BLU Acceleration is about providing order-of-magnitude benefits in performance, storage savings, and time to value.
This paper gives you an overview of these technologies, recommendations on hardware and software selection, guidelines for identifying the optimal workloads for BLU Acceleration, and information about capacity planning, memory, and I/O.
The examples in this paper are based on DB2 V10 fix pack 2 and GPFS 220.127.116.11 efix13 installed on AIX 6.1 TL6 SP5 but can be extended to more recent versions and other supported platforms. All versions of GPFS are supported with DB2 for Linux, UNIX, and Windows, however, the latest supported fix packs are recommended to ensure the best quality experience.
Technical paper summary:
In today’s highly competitive marketplace, it is important to deploy a data processing architecture that not only meets your immediate tactical needs, but that also provides the flexibility to grow and change to adapt to your future strategic requirements. To help reduce management costs, add flexibility, and simplify the storage management of your DB2® for Linux®, UNIX®, and Windows® installation, you need to choose a file system that is designed to provide a dynamic and scalable platform. The IBM® General Parallel File System™ (GPFS™) is a powerful platform on which to build this type of relational database architecture. This paper describes why GPFS is the right file system to use with DB2 databases by outlining the benefits and providing best practices for deploying GPFS with DB2 software. In addition, a section has been added to this paper to describe the DB2 pureScale feature, and how it configures and uses GPFS.
You are a busy professional and you don't always have the time and resources to travel to a technical conference, or call into a live web presentation, to listen to technical experts give great presentations about the products and technology you care about. Recorded webcasts offer you the benefits of listening to the same experts, at your own convenience, at the office or at home.
In this new DB2 best practices webcast, DB2 pureScale performance and monitoring
, watch and listen as Steve Rees, Senior Technical Staff Member and performance expert at the IBM lab, explains a wide array of configuration and tuning best practices to make your DB2 pureScale environments perform in an optimal fashion.
The tips and techniques presented in this webcast reflect information validated through the DB2 team's internal performance testing, as well as performance benchmark tests and customer engagements in real life DB2 pureScale environments.
In addition to the webcast video, you can download the presentation slides and transcript for easy offline viewing and reading.
If you have questions for the authors please add a comment to this blog or on the webcast's web page.
A new supplement to the popular DB2 best practices paper "Implementing DB2 Workload Management" has just been published. The supplement will help you set the DB2 client information fields for a variety of common middleware applications.
You can find it, along with other useful supplements, on the paper's information web page:https://ibm.biz/Bdx2n6
The DB2 client information fields are available on each connection to a database. These fields enable an external application that is using a connection to provide additional information to the DB2 database server that can be used to discriminate among connections based on end-user identification. The values in the client information fields are reported by DB2 for Linux®, UNIX®, and Windows® and other members of the DB2 family through various database monitoring and auditing interfaces. They are also leveraged by the DB2 workload definition in DB2 for Linux, UNIX, and Windows Version 9.5 and later as another way to aggregate connections to the database for purposes of monitoring and control.
Share your impressions and questions about the paper and supplements by adding comments to the web page (you need to join developerWorks and login first).
The new video from the DB2 team, "Getting up and running with HADR", provides a demonstration of how straightforward it is to set up HADR.
As we set up HADR in the video, we provide insight into some of the more important configuration decisions we are making, hopefully heading off some of the more common issues users face when setting up HADR.
The best practice paper Managing data growth
provides a wealth of recommendations to help you design and manage a database environment for efficient data growth, including tips on how to choose the right distribution key for a partitioned database.:
Database partitioning helps you to adapt to data growth by providing a way to expand the capacity of the system and scale for performance. A distribution key is a column (or group of columns) that is used to determine the database partition in which a particular row of data is stored. The following guidelines will help you to choose a distribution key.
- Choose the distribution key from those columns having the highest cardinality. Unique keys are good candidates. Columns with uneven data distribution or columns with a small number of distinct values might result in skew, where query processing involves more work on a subset of database partitions and less work on others.
- Choose the distribution key from columns with simple data types, such as integer or fixed length character; this will improve hashing performance.
- Choose the distribution key to be a subset of join columns to facilitate join collocation.
- Avoid choosing a distribution key with columns that are updated frequently.
- In an online transaction processing (OLTP) environment, ensure that all columns in the distribution key participate in transactions through equality predicates. This ensures that an OLTP transaction is processed within a single database partition and avoids the communication overhead inherent with multiple database partitions.
- Include columns that often participate in a GROUP BY clause in the distribution key.
- Unique index key columns must include all of the distribution key columns. The DB2 database manager creates unique indexes to support unique constraints and primary key constraints, and these system defined unique indexes have the same requirement that the constraint key columns must be a superset of the distribution key.
If you have any comments or questions for the authors of this best practice paper, feel free to log a comment on the paper's summary page and we will respond. You need to login with your IBM ID to be able to enter comments. Registering your Id is free and easy at developerWorks.
For this and more best practice guidelines for managing DB2 products, visit DB2 for Linux, UNIX, and Windows Best Practices
The table below, an excerpt from the best practices paper “Transforming IBM Industry Models into a production data warehouse
” describes guidelines for implementing an intelligent table space design strategy that gives you the flexibility you need to meet your service level objectives for all workloads; not just query, but backup, archive, maintenance, recovery, and ETL.
For this and more best practice guidelines for data warehousing visit DB2 LUW Best Practices on developerWorks
Welcome to the new DB2 Best Practices developerWorks community group.
You will find here technical papers, videos, and webcasts that offer expert advice on a variety of topics, to help you get the most out of your DB2 solutions, including DB2 for Linux, UNIX, and Windows, DB2 pureScale, and InfoSphere Warehouse products.
The best practices are written and tested by teams of technical experts who work in the IBM DB2 development and quality assurance teams, as well as with customers just like you, to determine and document the practical recommendations that can help you save time and resources on your information management projects.
If you are already familiar with our DB2 best practices on developerWorks, this new group is the next step and replaces the previous site. It includes all the great technical papers that we have published in the past, with the addition of new papers and videos on an ongoing basis.
Although you can already browse and download all the best practices in this group, we encourage you to join the group for added interaction with our teams. As a member, you are able to rate individual papers and leave comments about them, to help us make the best practices better. You can also subscribe to individual pages so that you will get notified when they get updated or when new comments are added.
If you like a particular best practice, you can also use developerWorks' sharing feature to easily share pages with friends and colleagues on a variety of social networks such as Facebook, Twitter, and LinkedIn.