 | Level: Intermediate Tyler Anderson (tyleranderson5@yahoo.com), Freelance Writer, Backstop Media
28 Aug 2007 In this five-part "Building
a grid system using WS-Resource Transfer" series, we look at the use
of WS-Resource Transfer (WS-RT) in different areas of the grid environment —
from using it as a method for storing and recovering general information about
grid-to-grid monitoring to management and security. We also examine how WS-RT can
be used for the distribution and division of work. In any grid, there is a huge
amount of metadata about the grid that needs
to be stored and distributed. Using WS-RT makes sharing the information, especially
the precise information required by different systems in the grid, significantly
easier. Part 1 examines the WS-RT
standard and looks at how to develop a WS-RT solution using Java™ technology and
Apache Muse.
About this series
The WS-RT standard provides a new method for accessing and exchanging information on
resources between components. It is designed to enhance the WS-Resource
Framework (WSRF) and build on the WS-Transfer standards. The WS-RT system extends previous
resource solutions for Web services and makes it easy not only to access resource
information by name but also to access individual elements of a larger data set through
the same mechanisms by exposing elements of an XML data set through the Web services interfaces.
In this series, we will look at the use of WS-RT in different areas of the grid environment, from
using it as a method for storing and recovering general information about the grid to
grid monitoring and management, and security. We'll also examine how WS-RT can be used
for the distribution and division of work. Series details:
- Part 1 examines the WS-RT standard and looks at how to develop a WS-RT solution using
Java technology and Apache Muse.
-
Part
2 begins using Apache Muse and the WS-RT specification to develop a WS-RT system
to manage and access grid information.
-
Part 3 looks at how to collect monitor data, how to expose the monitor data through
WS-RT, and how to use WS-RT to extract trend information to help make predictions.
-
Part 4 looks at both sides of the security session, in terms of using WS-RT as an aid
to the authorization process and at combining WS-Security with WS-RT for secure resource exchange.
-
Part 5 looks at using WS-RT for work distribution.
Introduction and the big picture
Grid research and application development has been transitioning more and more to Web
services, starting with Open Grid Services Infrastructure (OGSI) in 2001. The plan was to start with OGSI, then
develop new specifications as time went on. WSRF and WS-ResourceProperties (WSRP) were
eventually released and were meant to comprise the framework needed for
grid services, but competing standards like WS-Transfer made interoperability difficult,
and consolidation in the open standards was necessary. The problem with WS-Transfer was
its inability to operate on fragments of resources the WSRF and WSRP
specifications already had.
Enter WS-RT. It was designed with WS-Transfer and WSRF and WSRP in mind. Thus emerged the
WS-Transfer specification with greater capabilities for operating on resources: WS-RT.
This article introduces the WS-RT specification and how we'll build a grid service
using it and Apache Muse for the implementation of WS-RT
throughout this series. First, you'll take a look at Apache Muse and what WS-RT work
has already been done with it. Then you'll look at how we can use WS-RT for managing
and accessing grid information, grid monitoring, security, and work distribution.
Apache Muse
Apache Muse has already advanced considerably since graduating from the incubator in
2005, and currently has a 2.2.0 release. A solid framework, such as that provided by
Apache Muse, is important for those deciding to use a certain application framework to
develop Web services applications.
In fact, about a year ago, IBM® contributed code and increased its support to the Apache
Muse project. This led to other companies increasing their support, as well. Thus,
there are already considerable advances in a real WS-RT implementation. And the great
thing about using open Web services standards in your grid application development is
the knowledge that what you're developing today will somehow fit in what gets developed
tomorrow; interoperability is key in any technological environment.
In this series, you'll leverage the current work that's already been done in the Apache
Muse WS-RT realm in developing the grid components of the overall grid service. Apache
Muse also has full WS-Notification 1.3/Web Services Distributed Management (WSDM)
1.1, which you'll also take advantage of in piecing together the overall grid service.
IBM alphaWorks offers a demo highlighting the interoperability of the
previous WSRP specification with the new WS-RT specification (see Resources) using a single XSLT stylesheet to map WS-RT messages
to WS-RP messages. Interoperability is important when it comes to achieving wide
support for new specifications, so legacy Web services leveraging the WSRF and WSRP
specifications can still be supported in the new model.
Be sure to browse, download, and experiment with the alphaWorks demonstration. It will
give you a good feel for WS-RT and how you can better implement it in your grid application.
Next, you'll start learning more about the grid service's specifics and how the
advantages of the new WS-RT specification are brought to light.
Grid information
In any grid, there is a huge amount of metadata about the grid that needs to be stored
and distributed. Using WS-RT makes sharing the information, especially the precise
information required by different systems in the grid, significantly easier.
Resources for the grid will be stored in various nodes, including a manager or managers
and several nodes. The interface used to communicate with the grid is a client. You'll
use the client to query and control the grid. One possible configuration for the grid
manager, the client, and the respective nodes is shown below.
Figure 1. Node configuration in the grid
Because each grid node has its own pieces of information, you'll use the WS-RT Create operation to create resources with initial data. This operation will be invoked by the
grid manager on the startup of each of the nodes to initialize each of the grid node's resources.
Most of the node's information will be updated by the nodes themselves as they perform
required tasks. Disk space or compute resources available on the machine the node runs
on are examples. However, commands and other task-specific information will be set
using the WS-RT Put operation by the grid manager, or perhaps via other nodes making
requests to other nodes in the grid.
When information needs to be read from another node, the WS-RT Get operation will be
used. The grid monitor, for example, will use WS-RT Get and
Put operations to monitor
nodes in the grid, among other things, which you'll learn next.
And when nodes start up, they need to be configured based on system settings, so you'll
need a runtime configuration file. An example configuration file is shown below.
Listing 1. Example configuration file
# Certificate storage location
certs: C:\apps\webapps\grid6\manager\rc\certs\
# URL of THIS manager service
self: http://localhost:8080/axis2/services/MovieGridManager
# URLs of node services
node: http://localhost:8080/axis2/services/MovieGridNode1
node: http://localhost:8080/axis2/services/MovieGridNode2
|
You then need to load the configuration file and set the node specific parameters as
they are read.
Listing 2. Reading in a configuration file
private static void loadConfigFile(String file) throws Exception{
File f = new File(file);
if(!f.exists()){
System.err.println("Manager config file not found!!!");
return;
}
FileReader fr = new FileReader(file);
BufferedReader in = new BufferedReader(fr);
String line;
while((line = in.readLine()) != null){
if(line == "" || line.charAt(0) == '#') continue;
// if line starts with "certs:" then set the location of
// certificates in the filesystem
// else if line starts with "self:" then set the URL of
// this service
// else if line starts with "node:" then add a new node's
// stub object to the list of node stub objects
}
certFilenames = (new File(pathToCerts)).list();
}
|
You'll build on this code in Part 2 and throughout
this series as you build the code for the nodes in your grid.
Grid monitoring
The support in WS-RT for grabbing entire blocks of data rendered in XML, and for
picking out specific elements makes it a good solution for sharing grid monitor data.
For example, we can use WS-RT to extract information about all the disks in a system,
about a specific system, or for grabbing trend data by asking our grid node to provide
historical information through the WS-RT interface.
More specifically, you'll monitor the operations and interactions in the grid using
WS-RT Get and Put operations, as
well as the WS-Notification specification. As events happen, such as direct
interactions between the grid client and a grid node, it's good to log such activities
by having the grid node fire off an event notification (one way) back to the grid
manager. You'll code this functionality using WS-Notification by having the grid
monitor subscribe to certain or all events available on grid nodes. Then as
notifications come into the grid monitor, you'll be able to log or record them
appropriately. As nodes come offline, you have to be sure to unsubscribe the grid
monitor from subscribed events in such nodes.
Using these Web services operations, you'll be able to perform a host of monitoring
tasks, including logging and collecting information, extracting individual node
statistics, and you'll pick up on trends across the grid.
Security
Considerations for security in the WS-RT specification employ the
security mechanisms as specified in the WS-Security specification:
- Authentication — To maintain system integrity, all nodes in the grid
(manager- and monitor-inclusive) need to be able to authenticate everyone else in the
grid. Essentially, everyone is untrustable until proven trustable using WS-Security methods.
- Authorization — After a node verifies itself to another, the two nodes
can begin communicating.
- Third-party snooping — A surefire way to make further communications
secure from third parties is to encrypt data transmitted to and from nodes in the grid.
Fortunately, WS-RT supports a simplified method for exchanging XML data, and you can
exploit that for exchanging security information. In an authorization environment,
where access to individual resources may be controlled through access-control lists,
you can take advantage of the structure request system of WS-RT to support the
validation of access-control lists.
You'll look at both sides of the security session, in terms of using WS-RT as an
aid to the authorization process and at combining WS-Security with WS-RT for secure
resource exchange.
Work distribution
Work is distributed among nodes from a central point — namely, the grid manager.
The flexibility of the WS-RT system in terms of transferring material and selections of
material means we can use the system as a method for distribution. In this scenario,
you can use WS-RT to request and submit resources that relate to individual work units.
This is particularly handy within the distribution of work within a computational grid
in that you can use it to pick and choose the work units we want, and even pick groups of work units.
Summary
You've learned a considerable amount regarding WS-RT and how to leverage it to
build a grid service. You learned about the advances in Apache Muse, and how there is
substantive and usable code there to implement Web services using the WS-RT
specification. You also learned about the grid systems/components that you'll be
touching upon in this series: grid information, grid monitoring, security, and work distribution.
In Part
2, use using Apache Muse and the WS-RT specification to develop a WS-RT system
to manage and access grid information.
Resources Learn
-
Read the entire "Building a
grid system using WS-Resource Transfer" series.
-
See what Ian Foster has to say about the release of the WS-RT specification in his blog
entry titled "WS-ResourceTransfer Specification Released."
-
A researcher in grid computing shares his experiences with WSRF in his Bruno's Blog entry
titled "Experiences with
WSRF."
-
Ian Robinson, the editor of the published WS-RT specification, gives an update on the
WS-RT specification in his blog entry titled "WS-ResourceTransfer update."
-
For a good introduction to the WS-Resource Transfer specification, read "Meet the specs:
Intro to WS-ResourceTransfer 1.0," "Meet the specs: WS-RT 1.0 operations, Part One" and "WS-RT 1.0
operations, Part One," Part Two, and Part Three.
-
See this grid series on building a grid more focused on WSRF: "Building a grid
using Web services standards."
-
The developerWorks SOA and Web
services zone provides guides and tutorials on Web services and Web services standards.
-
To listen to interesting interviews and discussions for software developers, check out check out developerWorks podcasts.
-
Stay current with developerWorks' Technical events and webcasts.
-
Check out upcoming conferences, trade shows, webcasts, and other Events around the world that are of interest to IBM open source developers.
Get products and technologies
Discuss
About the author | | | Tyler Anderson received his bachelor's degree in computer science in 2004 and his master's degree in electrical and computer engineering in 2005 from Brigham Young University. He worked with Stexar Corp. as a design engineer from May 2005 to August 2006. He was discovered by Backstop Media LLC in early 2005, and has written and coded numerous articles and tutorials for IBM developerWorks. |
Rate this page
| |
