Warning: This blog entry contains questions heard at Pulse, but not necessarily the answers. Rather than just jotting down a few fuzzy answers to these questions, I am going to go back to the lab to collect the most up to date information surrounding these questions, which I will roll out over the next couple of weeks (or months, if I get too busy with my day job).
Questions Asked About TPC at Pulse - Monday, Feb 28, 2011
What is the performance hit on a server when scanning its file and storage utilization data from a TPC storage resource agent?
If you have hundreds or thousands of servers, TPC can create collective reports about capacity, usage and utilization, files and directories, and volume correlation to storage systems. You must first deploy a TPC storage resource agent (SRA) to each server and create and run computer probes. A computer probe will gather detailed asset and high level file system and volume capacity data for the computer, which is sufficient for server volume (LUN) and file system correlation reports with the back end storage system volumes and for end to end data path monitoring in the TPC topology viewer. However, to produce detailed reports about files and the usage of those files, you must also create and schedule TPC file scans. Since file scans do gather a significantly larger amount of information from a server, you may wish to schedule these to run during off peak times. You can also optimize the scan using its profile to collect the specific information you require. Care must also be taken to not overload the TPC server by attempting to scan thousands of computers concurrently.
Now, back to the original question, which I will not answer (see warning above). What is the exact performance hit to a computer when performing a file scan? I will go back to the lab and gather the most recent data from our Customer Environment Test lab and answer in a later post.
A related question: How many SRAs can a single TPC server handle? I will gather the most recent best practices for this topic and answer in a later post. The answer, however, will depend on a number of factors: the size of the TPC server (8GB? 16GB? 24GB RAM?, Number/speed of CPUs), number of storage systems monitored for performance, number of volumes and files in the data center, and the complexity and scheduling of the file scans. If you are only running computer probes, which do not collect the individual file usage data, a given TPC instance can potentially handle more computers.
On a final note, starting with TPC 4.1 and now fully implemented in TPC 4.2.1 is this new storage resource agent. The deployment of the SRA is improved and much more reliable than in previous editions of TPC which required both a stand-alone Tivoli Agent Manager to handle security key management and a common agent process on each computer. This dependency, which added an additional layer of complexity, has been completely removed in TPC 4.2.1 FP2. Regardless of your license, I
invite you to try out the SRA and view your end to end storage data
paths in TPC's topology viewer. While you can probe your computers with TPC BE and TPC for Disk, performing file utilization scans requires the TPC for Data or TPC SE license.
If you currently have hundreds or thousands of legacy TPC agents deployed and want to migrate to the SRA or are planning to deploy hundreds or thousands of agents into a production environment, I recommend contacting your IBM representative to help ensure a seamless migration or deployment.
Also new in TPC 4.2.1 FP2 is the ability to monitor the VIO Server in a PowerVM system using the SRA, thus replacing the legacy data agent.
Does TPC for Databases support Informix
No. TPC for Databases does not support Informix databases, and this is currently not in the roadmap. Normally, Informix performs best when using raw partitions. The TPC SRA will still discover and report information about that raw partition, but it will not have insight into the utilization of that raw partition. If the Informix database is created in a file system, then a TPC SRA file scan will return utilization information about these files, just like any other file.
Informix does provide a simple, easy to use web based Informix Server Administrator that you could bookmark in your browser located on your TPC GUI client.
Can I launch EMC Control Center from TPC?
This should work, but I have not personally tried it. The SMI-S profile in the storage device CIM Agents should include a Remote Access Point which should include the URL to launch the device's element manager (i.e. the device's GUI). TPC manages EMC storage systems using the EMC CIMOM. When TPC discovers the EMC, it uses this Remote Access Point to populate the default Element Manager launch URL.
Why Do I Need TPC If I Have the SVC GUI, DS8000 GUI, or XIV GUI?
What is the business value of TPC over these individual element managers?
Here are 3 characteristics of TPC that help answer this question:
1. TPC has visibility into every entity in your storage environment
2. TPC collects performance data from all storage devices and switches
3. TPC has an enterprise database
TPC for Disk and TPC SE collect performance metrics from all your storage devices and switches. Currently, this performance monitoring and management functionality is not available in the SVC and DS8000 GUIs. Although the XIV GUI does display near real time performance data from a single XIV, TPC, given its support of multiple storage systems and switches and it's database of historical performance data, still adds significant performance management value by providing the following features:
1. You can monitor the performance along the entire data path
Since TPC collects switch port performance data in addition to storage system and storage port performance data, you can easily inspect the performance along a given data path, from host server to back-end volume. Performance data overlays are available in the Data Path Explorer in TPC's Topology Viewer.
2. You can create historical performance reports
Since TPC embeds DB2, historical performance data can be stored for many months or years to generate performance baselines, to look for performance problem patterns such as spikes, and to create historical performance reports.
3. You can generate performance trend reports
TPC allows you to inspect performance trends, which can be used for capacity and workload planning.
4. You can detect hot spots across your storage environment
TPC SE's Storage Optimizer uses collected performance data and disk magic models to detect and display hot spots in your storage environment. The Storage Optimizer then recommends migration actions to resolve performance bottlenecks.
5. Snapshots enable performance and configuration change tracking on changed and removed entities
If a volume or pool or system gets modified or deleted, TPC can still create historical performance reports on that volume or pool.
6. Performance Aggregations for Capacity Reporting across Multiple Storage Systems
TPC's History Aggregator enables you to view the sum of usage across multiple storage resources by file system. Trending enables you to see patterns of your historical aggregated data across your entire network.
Visualize Your SVC and Storwize Storage Topology
You can use TPC's Topology Viewer and Asset and Capacity Reporting capabilities to visualize and display back-end storage connectivity to your SVC or Storwize. Your SVCs might be backed by DS4800, DS6000, DS8000, EMC or HDS. You can monitor all of these storage systems from TPC. Not only can you visualize the pool (mdisk group) connections from mdisk to backend disk, you can view the performance of those connections and disks, including the switch port performance. If you do need to drill down into an individual element manager, you can do that from TPC, so TPC can serve as a convenient launching point for all of your element managers.
Two and Three Site Replication for Disaster Recovery
Since TPC can monitor your entire multi-site storage environment, TPC for Replication enables management of storage system copy services, such as Metro Mirror, Global Mirror and Global/Metro Mirror for single or multiple sites from a single console. Thus, from one place, you can monitor and control replication sessions, paths, individual copy pairs, and all SAN entities that play a role in disaster recovery.
TPC for Replication also supports HyperSwap, Open HyperSwap, System Z, and runs on System Z. (TPC SE does not run natively on System Z, just TPC for Replication).
If you have TPC Servers deployed into multiple sites across the globe, you can use Server Roll-up to consolidate essential storage management data onto a single TPC instance.
Troubleshooting the Entire Data Path End-to-End
Since TPC knows about
the switches and servers in addition to the just the storage systems known by each storage system's GUI, you can monitor the health of the SAN end to end.
As such, TPC knows the complete data path of each server side volume through the fabric to the back end storage volume. If a Database Administrator calls and asks why their application is performing slowing, you can open the server in TPC's Data Path Explorer and quickly detect the health and performance of the entire data path. You can then drill into more detailed performance reports for ports, volumes, or pools for the relevant time period and look for spikes or drop-offs in response times, IOPS, and throughput.
Monitor Configuration Changes in your Storage Environment
A nice value-add feature of TPC SE is the ability to easily view configuration changes in your storage environment. With TPC SE's Configuration History, you can compare the exact configuration of your SAN at any two points in time, including the current time. This can be useful for troubleshooting and impact analysis.
SAN Planner and Storage Provisioning
While many storage administrators effectively use command-line interfaces to provision volumes from their storage systems, some newer IT admins tend to like and expect graphical interfaces. While TPC in no way precludes experienced storage admins from using the DS CLI, SVC/Storwize CLI, or XIV CLI for provisioning operations, all versions of TPC enable on-the-fly provisioning from a single pane of glass. Within TPC's Disk Manager is the Storage System view which lists all Storage Systems in your environment. From there, you can quickly create a volume, assign it to a server, and, if needed, modify the zoning.
If you wish to provision a pool or storage system or set of storage systems, the SAN Planner, available in TPC SE, can use existing performance profiles and Disk Magic models to create and implement an optimized, efficient provisioning plan.
File Usage and Utilization Across All Your Servers
While you can always ssh to a box and run some Unix scripts to show you all files greater than 100 MB, what if you want to see all files in your entire data center that are greater than 100 MB? What if you want to see all files that have not been accessed in 1 year? 2 years? TPC has built in reports to answer these questions. Or, say you have allocated 100 GB to each of 400 Windows servers. How much of that 40 TB is actually storing files or how much is sitting there unused? TPC can answer that, and it can even correlate those Windows drives to the back-end storage systems, pools and volume. You can also create reports to look for specific files such as multiple redundant Windows Service Pack installation packages, music and movie collections, or other files that may be unnecessarily consuming your valuable space.
Integration with Tivoli Monitoring, Tivoli Application Dependency Discovery Manager, Tivoli Netcool/Omnibus and Tivoli Business Service Manager
If you currently use other Tivoli software products for data center and service management, then TPC serves as a convenient means to enable visibility and availability of your storage environment into those Tivoli products. IBM Tivoli has a variety of agents and sensors to allow TPC's storage data and events to be visualized and processed in ITM, TADDM, Omnibus and TBSM.
While this is not a complete list, hopefully this post has demonstrated the business value of TPC as an important addition to your storage management toolkit. To summarize, the key value
proposition of TPC derives from the fact that TPC sees the entire
storage environment, recording detailed historical and current data
about your storage environment for monitoring, troubleshooting, performance management,
analysis, and provisioning from one
The TPC Administrator
robhotch 110000SPT8 953 Visits
Storage Management Track
If you are attending IBM Pulse 2011, you can view the list of storage management presentations by going to http://pulsesmartsite.com (and creating an account using your Pulse confirmation number). From the SmartSite home page, click Sessions and Activities. In Select a Stream, select "Service Management for the Data Center." In Select a Track, select "Storage Management." All Storage Management presentations will then appear and you can add them to your personal agenda by clicking the plus '+' sign icon.
The Storage Management Track kick-off is session 1704 on Monday Morning:
1704 Storage Management Track Kickoff: The Butterfly Effect on Information and Storage
Mon, Feb 28, 2011
10:45 AM - 12:15 PM
Storage Management Solutions Expos
To see the Storage Management Solution Expos, click Solution Expo on the SmartSite home page. In the Solution Expo left pane, click "IBM Exhibits." In the Select IBM Exhibits drop down list, select Storage Management.
Booth 73 - Do you know how well your storage is performing?
I will spend much of my time in the TPC booth 73:
IBM - Do you know how well your storage is performing?
Booth:73 IBM Tivoli® Storage Productivity Center
IBM Exhibits: Storage Management