The traditional boundary between organizations and service providers has been along the line dividing telecom and IT services. Increasingly, however, organizations are looking for single-provider, unified telecom service and IT provision -- a corporate need driven by the requirement for fewer bills and fixed costs. Achieving this tight level of integration requires more than merely deploying and using software technology. As you'll see, software entities must be appropriately instrumented to achieve manageability. IBM is one of the few organizations developing the technology to work in this highly specialized and potentially profitable niche.
Figure 1 shows a fairly typical enterprise schematic in which a broad range of software and hardware is routinely used. Increasingly, enterprises are employing specialist providers, such as grid providers, to host applications. So elements of the network cloud on the left of Figure 1 are migrating in the direction of the clouds on the right.
Figure 1. The IT and telecom ecosystem
IT management tools are critical elements in both the enterprise and service provider networks. The problem of interest in this article series is the lack of end-to-end IT management tools for applications like Derby. IBM is leading the way in solving this problem with the adoption of autonomic computing and Web Services Distributed Management (WSDM). Careful management requires that all IT infrastructure be instrumented. There are, therefore, two distinct views of Derby: an application deployment view and a management view. This article derives principles for expressing and resolving the needs of this dual view of all such IT resources.
In my 2003 book, Network Management, MIBs and MPLS: Principles, Design and Implementation (see Resources for a link), I remarked that modern networks consist of unconnected islands of automation. This statement remains true today, where a growing number of networks and services operate in a nonintegrated fashion. The global economy seems to love technology. Just look at the many networks and services we take for granted: Internet (at home and at work), mobile telephones, Internet Protocol TV (IPTV), satellite television, online games, games with online content, and so on. Access to each of these technologies has become something of a commodity. As an example of the overlay nature of these networks, a UK-based television provider now allows the timing of television program recording over mobile telephones. This recording involves access to the mobile telephone network, the Internet, and the television provider. Rather than impose a flat rate, each provider in this supply chain imposes a charge for access.
Meanwhile, the IT industry has seen dramatic consolidation and impressive new (and some not-so-new) software standards, such as:
- Web services
- Business Process Execution Language (BPEL)
- Java™ technology (now a de facto standard)
- A rich mixture of open source products
The success of the open source model has been so complete that we can confidently state the major requirements for modern technology products:
- Products must be low cost.
- Products must be flexible, and users must be able to configure them.
- Products must work.
- Products must be priced at commodity levels.
It must be difficult these days for software vendors to make ends meet. Some vendors -- for example, antivirus software vendors -- use a subscription model to provide at least some revenue. You need look no further than Linux® and Apache Axis to see the power of open source tools.
One significant negative of the open source model for software is that it tends to encourage silos -- that is, stand-alone vertical applications, such as Linux and Axis. Such silo applications have some interesting characteristics. For example, silos are often difficult to install and automate. They aren't built for automated management, and silo software features are typically seen as more important than manageability.
The silo nature of modern software (both open and closed source) adds considerably to its cost of ownership. So while it might seem that the software problem has been solved, two significant problems loom:
- Organizations need automated end-to-end management (the focus of this series).
- Organizations increasingly need IT compliance, which is all about ensuring that IT and its use conform both to organizational policies and to regulatory rules. (See Resources for more information on IT compliance.)
The main problem with the deployment and management of IT and telecom services is that the two areas have traditionally been treated as separate and distinct disciplines. More precisely:
- IT services are facilities provided to business software users -- networking, database engines, applications, patches, antivirus software updates, operating systems, and so on. IT services represent the computing infrastructure that organizational business processes use.
- Telecom services are network technologies either that enterprises use directly (for example, telephony) or that form the plumbing of other services -- for example, Virtual Private Networks (VPNs) and Multiprotocol Label Switching (MPLS). Telecom services are traditionally supplied by specialist providers.
- Deployment is the rolling out and application use of IT elements, such as database engines, networks, and middleware. The key requirement of deployment is the use (not the management) of the deployed element.
- Management is the set of processes concerned with maintaining the operational service of the underlying IT infrastructure.
A deployment-centric approach tends to be completely different than a management-centric approach. To help make this concept concrete, look at two examples of the difference between deployment and management: overprovisioning versus right-size provisioning and Quality of Service (QOS). The deployment-centric approach tends to overprovision on parameters such as bandwidth, number of servers, and processor speed. The management-centric approach tries to put infrastructure in place that matches size with the imposed and projected load.
The deployment-centric approach tends to hope for the best on QOS by overprovisioning. The management-centric approach tries to dynamically ensure QOS. IBM's autonomic computing initiative provides a management-centric framework that also forms the basis for effective deployment. Now let's take a brief look at the way enterprise applications use Derby.
Applications (and users, such as database administrators, or DBAs) use Derby for storage, retrieval, and manipulation of relational data. Aside from a possible delay, it matters little to the application whether this data originates on the host computer or travels across multiple networks. The process occurs transparently as the data is pushed in the direction of the client program.
In this context, applications view Derby as illustrated in Figure 2 -- as a provider connected to a pipe (in the plumbing sense). Data is pushed in both directions through the pipe.
Figure 2. Business process applications using Derby across one or more networks
Listing 1 shows an example of a client accessing a Derby database instance.
Listing 1. A client accessing a Derby database instance
C:\java\derby\db-derby-10.1.2.1-bin\demo\nserverdemo>java SimpleNetworkClientSample Starting Sample client program Got a client connection via the DriverManager. connection from datasource; getDriverName = Apache Derby Network Client JDBC Driver Got a client connection via a DataSource. Testing the connection obtained via DriverManager by executing a sample query number of rows in sys.systables = 16 Testing the connection obtained via a DataSource by executing a sample query number of rows in sys.systables = 16 Goodbye!
Listing 1 is, in fact, the output from one of the example Java programs that come bundled with the Derby distribution. The important point is that the client simply uses the Derby instance and the underlying IT infrastructure, consuming whatever resources are necessary. But what about the resource consumption that occurs when this simple client program runs?
You want to be able to interrupt the Listing 1 program at a strategic point in its execution. So at this probe point, you perform a simple check just to see what type of resource consumption is occurring. Figure 3 provides an excerpt from the Microsoft® Windows® XP Task Manager. The highlighted totals form a crude baseline for examining Derby client resource consumption.
Figure 3. Resource consumption baseline
Now, when you rerun the Derby client, you see the modified program output shown in Listing 2.
Listing 2. A client accessing a Derby database instance
C:\java\derby\db-derby-10.1.2.1-bin\demo\nserverdemo>java SimpleNetworkClientSample Starting Sample client program Got a client connection via the DriverManager. connection from datasource; getDriverName = Apache Derby Network Client JDBC Driver Got a client connection via a DataSource. Testing the connection obtained via DriverManager by executing a sample query Please press enter to test the Derby connection.
Figure 4. Resource consumption just before SQL command execution
Figure 3 and Figure 4 crudely illustrate resource consumption associated with this use of Derby. The highlighted counters in these two figures reflect host resource consumption. Handles refer to entities, such as files and registry entries. Threads are objects that allow for concurrent execution of program tasks. Processes relate to overall programs (like the Derby client). If the client operates across a network, you can add in network-related resource consumption, such as bandwidth, router/switch processing, and server access.
The resource elements consumed reflect the previous discussion concerning overprovisioning and QOS. In other words, when you want to deploy a system that involves clients, databases, networks, and so on, this is the type of resource consumption that affects the required infrastructure. Trying to estimate the required dimensions for a system is a difficult task. As you'll see in the next section, however, you can divide the management problem into several functional areas, which helps in designing an IT management solution.
The acronym FCAPS (or fault, configuration, accounting, performance, and security) has been used for many years in the area of network management. While not yet widely used in the IT management field, this acronym provides an excellent template for classifying and designing management solutions. FCAPS describes the major functional areas that a management system typically (or ideally) addresses. Applied to a Derby-related IT infrastructure, the FCAPS areas can be described as:
- Faults: Handles failures (for example, if a server fails, a backup can be switched into service)
- Configuration: Handles the manipulation of the system (for example, creating users and databases)
- Accounting: Handles all aspects related to billing for the use of Derby databases
- Performance: Handles the analysis of patterns of manipulation of Derby databases (for example, determining whether bottlenecks have occurred)
- Security: Handles the implementation of security (for example, password maintenance and checking for security incidents)
So, how might a hypothetical FCAPS-based IT management solution view Derby?
The following simple Derby application scenarios provide a brief description of how they might be classified in relation to FCAPS. Faults occur when a client tries to open a nonexistent database table; the IT management system can respond with a message to the client. The configuration capability is exercised when a client opens a database and executes a query that updates some columns of the table. Accounting handles the billing for use of disk space as the tables in a database grow. Billing need not necessarily be financial in nature -- it can merely reflect some type of usage tariff (for example, interdepartment settlement). Performance can consist of determining whether bottlenecks have occurred when large batches of SQL commands are executed. Security ensures that users access only the appropriate databases.
Derby includes an application programming interface (API) with which programmers can build complete database-centric applications. Conceptually, this API looks like Figure 5, in which a developer writes an application, compiles it, and then runs it against a Derby database (after creating the database).
Figure 5. A fat-client program interacts with Derby
The developer can write the semantics of the application in Figure 5 to control all elements of the database life cycle: database creation, table update, table deletion, table insertion, database deletion, and so on.
Another approach to using Derby databases is as managed objects. This approach is shown in Figure 6 where a management application handles the complexities of the Derby interactions.
Figure 6. A thin-client program interacts with a Derby management system
The management application also implements FCAPS capabilities. So by treating Derby as a managed object, you get two elements of added value: simpler (that is, thinner) client programs and the basis for an IT management system.
Fortunately, technology exists for producing the management software shown in Figure 6. IBM and Sun Microsystems have both had a stab at this technology -- IBM with its autonomic computing touchpoints and Sun with Java Management Extension (JMX) technology. Both of these technologies can be generically termed instrumentation, and each has its advantages. For example, touchpoints are Web services-based and WSDM compliant, and they employ useful open source software, such as Axis. JMX is Java-based technology and leverages the Java platform (for example, Remote Method Invocation [RMI] and security).
Part 2 of this article series will use touchpoint technology to illustrate how you can use Derby as a managed object, thereby fulfilling the requirements of Figure 6. Here are some principles for employing Derby as a managed object:
- The need to remotely start and stop a Derby instance
- The need to use application server infrastructure for running Derby
- Ease in extending the source code of a Derby client program
- Ease of access to Derby infrastructure data
- Effective handling of Derby errors and exceptions
Autonomic computing touchpoints facilitate these and other needs, as you'll see in Part 2.
IT requirements continue to grow at a rapid pace. To some extent, this growth is fueled by the increasing range of commercial network services, such as access to home-based technology through mobile telephones and the Internet. Consumers are increasingly technology savvy and demand similar services from enterprise IT.
The old model of simply deploying IT and overprovisioning capacity is reaching the end of its useful life. Systems must be equipped to grow with the dynamics of the imposed loads, and this can only be done with automation. While some elements of this problem can be moved in the direction of specialist providers (for example, grid service vendors), the bulk of the problem remains within the enterprise. In this context, IT may well be a victim of its own success. The upside is that help is at hand.
By using applications, such as, Derby as managed objects, you can begin to put the required automation elements in place in the network. You can then use these elements to manage the complexities of the managed objects, which in turn helps produce a much-needed simplification in the basic application software. You then get simpler clients operating in conjunction with FCAPS-compliant IT management systems. In the conclusion of this series, you'll find out how to achieve these requirements.
- Read Network
Management, MIBs and MPLS: Principles, Design and Implementation by Stephen
Morris (Prentice-Hall, 2003) for information about the software side of network management using
the MPLS technology as a core topic. It discusses the details of how to start with technology and derive managed solutions.
- Visit NetIQ for more information on IT compliance and its complex relationship with IT management and to find products and services.
- Find documentation and downloads for JMX.
- Visit the Apache Derby site for Derby resources.
- Visit the developerWorks Open source zone for extensive how-to information, tools, and project updates to help you develop with open source technologies and use them with IBM's products.
- Browse all the Apache articles and free Apache tutorials available in the developerWorks Open source zone.
- Browse for books on these and other technical topics at the Safari bookstore.
- Check out the developerWorks Apache Derby project area for articles, tutorials, and other resources to help you get started with Derby today.
- Learn more about the IBM Cloudscape™ database, which is built using the Apache Derby code base.
- Visit the
computing zone for more information about autonomic computing and IBM's efforts in this area.
Stay current with
technical events and Webcasts.
Get products and technologies
Build your next development project with
trial software, available for download directly from developerWorks.
- Get involved in the developerWorks community by participating in developerWorks blogs.
Stephen is the CTO of Omey Communications in Ireland. For the past 20 years, Stephen has worked for some of the world's largest networking companies on a wide range of software projects, including J2EE/J2SE-based network management systems, billing applications, porting and developing SNMP entities, network device technologies, and GSM mobile networking applications. He is the author of Network Management, MIBs and MPLS: Principles, Design and Implementation (Prentice Hall PTR, 2003) as well as several articles on network management and other topics for InformIT and OnJava.com. You can reach Stephen at email@example.com.