The need for end-to-end IT management

The traditional boundary between organizations and service providers has been along the line dividing telecom and IT services. Increasingly, however, organizations are looking for single-provider, unified telecom service and IT provision -- a corporate need driven by the requirement for fewer bills and fixed costs. Achieving this tight level of integration requires more than merely deploying and using software technology. As you'll see, software entities must be appropriately instrumented to achieve manageability. IBM is one of the few organizations developing the technology to work in this highly specialized and potentially profitable niche.

Figure 1 shows a fairly typical enterprise schematic in which a broad range of software and hardware is routinely used. Increasingly, enterprises are employing specialist providers, such as grid providers, to host applications. So elements of the network cloud on the left of Figure 1 are migrating in the direction of the clouds on the right.

IT management tools are critical elements in both the enterprise and service provider networks. The problem of interest in this article series is the lack of end-to-end IT management tools for applications like Derby. IBM is leading the way in solving this problem with the adoption of autonomic computing and Web Services Distributed Management (WSDM). Careful management requires that all IT infrastructure be instrumented. There are, therefore, two distinct views of Derby: an application deployment view and a management view. This article derives principles for expressing and resolving the needs of this dual view of all such IT resources.

Back to top

IT: A victim of its own success?

In my 2003 book, Network Management, MIBs and MPLS: Principles, Design and Implementation (see Resources for a link), I remarked that modern networks consist of unconnected islands of automation. This statement remains true today, where a growing number of networks and services operate in a nonintegrated fashion. The global economy seems to love technology. Just look at the many networks and services we take for granted: Internet (at home and at work), mobile telephones, Internet Protocol TV (IPTV), satellite television, online games, games with online content, and so on. Access to each of these technologies has become something of a commodity. As an example of the overlay nature of these networks, a UK-based television provider now allows the timing of television program recording over mobile telephones. This recording involves access to the mobile telephone network, the Internet, and the television provider. Rather than impose a flat rate, each provider in this supply chain imposes a charge for access.

Meanwhile, the IT industry has seen dramatic consolidation and impressive new (and some not-so-new) software standards, such as:

• Web services
• Business Process Execution Language (BPEL)
• Java™ technology (now a de facto standard)
• A rich mixture of open source products

The success of the open source model has been so complete that we can confidently state the major requirements for modern technology products:

• Products must be low cost.
• Products must be flexible, and users must be able to configure them.
• Products must work.
• Products must be priced at commodity levels.

It must be difficult these days for software vendors to make ends meet. Some vendors -- for example, antivirus software vendors -- use a subscription model to provide at least some revenue. You need look no further than Linux® and Apache Axis to see the power of open source tools.

One significant negative of the open source model for software is that it tends to encourage silos -- that is, stand-alone vertical applications, such as Linux and Axis. Such silo applications have some interesting characteristics. For example, silos are often difficult to install and automate. They aren't built for automated management, and silo software features are typically seen as more important than manageability.

The silo nature of modern software (both open and closed source) adds considerably to its cost of ownership. So while it might seem that the software problem has been solved, two significant problems loom:

1. Organizations need automated end-to-end management (the focus of this series).
2. Organizations increasingly need IT compliance, which is all about ensuring that IT and its use conform both to organizational policies and to regulatory rules. (See Resources for more information on IT compliance.)

Back to top

Service provision: A house divided

The main problem with the deployment and management of IT and telecom services is that the two areas have traditionally been treated as separate and distinct disciplines. More precisely:

• IT services are facilities provided to business software users -- networking, database engines, applications, patches, antivirus software updates, operating systems, and so on. IT services represent the computing infrastructure that organizational business processes use.
• Telecom services are network technologies either that enterprises use directly (for example, telephony) or that form the plumbing of other services -- for example, Virtual Private Networks (VPNs) and Multiprotocol Label Switching (MPLS). Telecom services are traditionally supplied by specialist providers.
• Deployment is the rolling out and application use of IT elements, such as database engines, networks, and middleware. The key requirement of deployment is the use (not the management) of the deployed element.
• Management is the set of processes concerned with maintaining the operational service of the underlying IT infrastructure.

A deployment-centric approach tends to be completely different than a management-centric approach. To help make this concept concrete, look at two examples of the difference between deployment and management: overprovisioning versus right-size provisioning and Quality of Service (QOS). The deployment-centric approach tends to overprovision on parameters such as bandwidth, number of servers, and processor speed. The management-centric approach tries to put infrastructure in place that matches size with the imposed and projected load.

The deployment-centric approach tends to hope for the best on QOS by overprovisioning. The management-centric approach tries to dynamically ensure QOS. IBM's autonomic computing initiative provides a management-centric framework that also forms the basis for effective deployment. Now let's take a brief look at the way enterprise applications use Derby.

Back to top

Application-level use of Derby

Applications (and users, such as database administrators, or DBAs) use Derby for storage, retrieval, and manipulation of relational data. Aside from a possible delay, it matters little to the application whether this data originates on the host computer or travels across multiple networks. The process occurs transparently as the data is pushed in the direction of the client program.

In this context, applications view Derby as illustrated in Figure 2 -- as a provider connected to a pipe (in the plumbing sense). Data is pushed in both directions through the pipe.

Listing 1 shows an example of a client accessing a Derby database instance.

C:\java\derby\db-derby-10.1.2.1-bin\demo\nserverdemo>java SimpleNetworkClientSample
Starting Sample client program
Got a client connection via the DriverManager.
connection from datasource; getDriverName = Apache Derby Network Client JDBC Driver
Got a client connection via a DataSource.
Testing the connection obtained via DriverManager by executing a sample query
number of rows in sys.systables = 16
Testing the connection obtained via a DataSource by executing a sample query
number of rows in sys.systables = 16
Goodbye!

Listing 1 is, in fact, the output from one of the example Java programs that come bundled with the Derby distribution. The important point is that the client simply uses the Derby instance and the underlying IT infrastructure, consuming whatever resources are necessary. But what about the resource consumption that occurs when this simple client program runs?

Back to top

Derby as a resource consumer

You want to be able to interrupt the Listing 1 program at a strategic point in its execution. So at this probe point, you perform a simple check just to see what type of resource consumption is occurring. Figure 3 provides an excerpt from the Microsoft® Windows® XP Task Manager. The highlighted totals form a crude baseline for examining Derby client resource consumption.

Now, when you rerun the Derby client, you see the modified program output shown in Listing 2.

C:\java\derby\db-derby-10.1.2.1-bin\demo\nserverdemo>java SimpleNetworkClientSample
Starting Sample client program
Got a client connection via the DriverManager.
connection from datasource; getDriverName = Apache Derby Network Client JDBC Driver
Got a client connection via a DataSource.
Testing the connection obtained via DriverManager by executing a sample query
Please press enter to test the Derby connection.

Figure 4 provides a resource consumption snapshot just before the Derby client executes its SQL statement -- namely, at the point Listing 2 describes.

Figure 3 and Figure 4 crudely illustrate resource consumption associated with this use of Derby. The highlighted counters in these two figures reflect host resource consumption. Handles refer to entities, such as files and registry entries. Threads are objects that allow for concurrent execution of program tasks. Processes relate to overall programs (like the Derby client). If the client operates across a network, you can add in network-related resource consumption, such as bandwidth, router/switch processing, and server access.

The resource elements consumed reflect the previous discussion concerning overprovisioning and QOS. In other words, when you want to deploy a system that involves clients, databases, networks, and so on, this is the type of resource consumption that affects the required infrastructure. Trying to estimate the required dimensions for a system is a difficult task. As you'll see in the next section, however, you can divide the management problem into several functional areas, which helps in designing an IT management solution.

Back to top

Management characteristics: FCAPS

The acronym FCAPS (or fault, configuration, accounting, performance, and security) has been used for many years in the area of network management. While not yet widely used in the IT management field, this acronym provides an excellent template for classifying and designing management solutions. FCAPS describes the major functional areas that a management system typically (or ideally) addresses. Applied to a Derby-related IT infrastructure, the FCAPS areas can be described as:

• Faults: Handles failures (for example, if a server fails, a backup can be switched into service)
• Configuration: Handles the manipulation of the system (for example, creating users and databases)
• Accounting: Handles all aspects related to billing for the use of Derby databases
• Performance: Handles the analysis of patterns of manipulation of Derby databases (for example, determining whether bottlenecks have occurred)
• Security: Handles the implementation of security (for example, password maintenance and checking for security incidents)

So, how might a hypothetical FCAPS-based IT management solution view Derby?

Back to top

IT management and its view of Derby

The following simple Derby application scenarios provide a brief description of how they might be classified in relation to FCAPS. Faults occur when a client tries to open a nonexistent database table; the IT management system can respond with a message to the client. The configuration capability is exercised when a client opens a database and executes a query that updates some columns of the table. Accounting handles the billing for use of disk space as the tables in a database grow. Billing need not necessarily be financial in nature -- it can merely reflect some type of usage tariff (for example, interdepartment settlement). Performance can consist of determining whether bottlenecks have occurred when large batches of SQL commands are executed. Security ensures that users access only the appropriate databases.

Derby includes an application programming interface (API) with which programmers can build complete database-centric applications. Conceptually, this API looks like Figure 5, in which a developer writes an application, compiles it, and then runs it against a Derby database (after creating the database).

The developer can write the semantics of the application in Figure 5 to control all elements of the database life cycle: database creation, table update, table deletion, table insertion, database deletion, and so on.

Another approach to using Derby databases is as managed objects. This approach is shown in Figure 6 where a management application handles the complexities of the Derby interactions.

The management application also implements FCAPS capabilities. So by treating Derby as a managed object, you get two elements of added value: simpler (that is, thinner) client programs and the basis for an IT management system.

Back to top

A brief description of instrumentation

Fortunately, technology exists for producing the management software shown in Figure 6. IBM and Sun Microsystems have both had a stab at this technology -- IBM with its autonomic computing touchpoints and Sun with Java Management Extension (JMX) technology. Both of these technologies can be generically termed instrumentation, and each has its advantages. For example, touchpoints are Web services-based and WSDM compliant, and they employ useful open source software, such as Axis. JMX is Java-based technology and leverages the Java platform (for example, Remote Method Invocation [RMI] and security).

Part 2 of this article series will use touchpoint technology to illustrate how you can use Derby as a managed object, thereby fulfilling the requirements of Figure 6. Here are some principles for employing Derby as a managed object:

• The need to remotely start and stop a Derby instance
• The need to use application server infrastructure for running Derby
• Ease in extending the source code of a Derby client program
• Ease of access to Derby infrastructure data
• Effective handling of Derby errors and exceptions

Autonomic computing touchpoints facilitate these and other needs, as you'll see in Part 2.

Back to top

Conclusion

IT requirements continue to grow at a rapid pace. To some extent, this growth is fueled by the increasing range of commercial network services, such as access to home-based technology through mobile telephones and the Internet. Consumers are increasingly technology savvy and demand similar services from enterprise IT.

The old model of simply deploying IT and overprovisioning capacity is reaching the end of its useful life. Systems must be equipped to grow with the dynamics of the imposed loads, and this can only be done with automation. While some elements of this problem can be moved in the direction of specialist providers (for example, grid service vendors), the bulk of the problem remains within the enterprise. In this context, IT may well be a victim of its own success. The upside is that help is at hand.

By using applications, such as, Derby as managed objects, you can begin to put the required automation elements in place in the network. You can then use these elements to manage the complexities of the managed objects, which in turn helps produce a much-needed simplification in the basic application software. You then get simpler clients operating in conjunction with FCAPS-compliant IT management systems. In the conclusion of this series, you'll find out how to achieve these requirements.

Resources

Learn

• Read Network Management, MIBs and MPLS: Principles, Design and Implementation by Stephen Morris (Prentice-Hall, 2003) for information about the software side of network management using the MPLS technology as a core topic. It discusses the details of how to start with technology and derive managed solutions.
  
  
• Visit NetIQ for more information on IT compliance and its complex relationship with IT management and to find products and services.
  
  
• Find documentation and downloads for JMX.
  
  
• Visit the Apache Derby site for Derby resources.
  
  
• Visit the developerWorks Open source zone for extensive how-to information, tools, and project updates to help you develop with open source technologies and use them with IBM's products.
  
  
• Browse all the Apache articles and free Apache tutorials available in the developerWorks Open source zone.
  
  
• Browse for books on these and other technical topics at the Safari bookstore.
  
  
• Check out the developerWorks Apache Derby project area for articles, tutorials, and other resources to help you get started with Derby today.
  
  
• Learn more about the IBM Cloudscape™ database, which is built using the Apache Derby code base.
  
  
• Visit the developerWorks Autonomic computing zone for more information about autonomic computing and IBM's efforts in this area.
  
  
• Stay current with developerWorks technical events and Webcasts.
  
  

Get products and technologies

• Build your next development project with IBM trial software, available for download directly from developerWorks.
  
  

Discuss

• Get involved in the developerWorks community by participating in developerWorks blogs.
  
  

About the author

Rate this article

Comments

Back to top

Table of contents

• The need for end-to-end IT management
• IT: A victim of its own success?
• Service provision: A house divided
• Application-level use of Derby
• Derby as a resource consumer
• Management characteristics: FCAPS
• IT management and its view of Derby
• A brief description of instrumentation
• Conclusion
• Resources
• About the author
• Comments

Next steps from IBM

WebSphere Application Server can leverage its built in Web Services support to expose services to remote consumers in an industry-standard way.

---

• Try: The no-charge WebSphere Application Server Community Edition is a pre-integrated, lightweight Java 5 application server built on Apache Tomcat and includes built in Web Services support.
• Article: Learn how to use of the JAX-WS programming model to create dispatch and dynamic proxy clients and how to create an asynchronous Web service client.
• Tutorial: This tutorial takes you through the steps to deploy Data Web Services to the WebSphere Application Server Community Edition.
• Demo: This demo shows how to extract and install the IBM Java Platform, Java EE 5 SDK with WebSphere Application Server.
• Buy: WebSphere Application Server - Express

Dig deeper into Open source on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers