![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
Corporations are looking for improvements in the way they use and control technology. Business process management (BPM) and Service-Oriented Architecture (SOA) are key enablers for business agility, enhanced business control, and risk management, and are creating new opportunities in the way that businesses perform essential functions.
End-to-end or straight-through-processing (STP) architecture accepts work items at its boundary and controls processing to completion without yielding control. Such architecture can span a single department, multiple lines of business (LOB), or members of a trading group. Compelling business and technical benefits are partially offset by new challenges.
A key challenge is effective reference and configuration data distribution in large and highly available SOA implementations. You can highlight this by investigating the nature of the workload. For example, a SOA needs to accommodate multiple business processes. The SOA controls each process instance (work item) for a different period (seconds, days, weeks, or years) and has varying resource requirements to complete. Item streams are created by grouping work items that must be processed using common rules and reference data.
This article discusses a solution for the distribution of configuration, reference, and business rules. Policy management is the act of ensuring that services are correct, timely, and accurately informed of policy. It is a system overhead and must be simple to operate, high performing, and cost-effective.
The term policy data refers to all types of data to make business or technical decisions. Inaccurate, late, or incomplete data is acceptable in some instances and not in others. Visibility of the status of a policy setting is critical. For example, a timestamp with no time zone is acceptable for a single locale but useless in a multi-locale environment.
For the purposes of this article, a policy is a set of named data items (settings) that are managed and retrieved as a single unit. Setting names are unique within a policy but might exist in more than one policy.
A policy is a definite course or method of action selected from alternatives, and in light of given conditions, to guide and determine decisions. For example, a business policy states that payments under 5,000 USD are not subject to fraud check. The rule and value (5,000 USD) are policy data. For example, if USD(<Item.Value>) is greater than <Policy.Value>, then Do-Fraud-Check.
This policy management solution (PMS) vision provides a centrally managed policy data service for a SOA. The solution is responsible for all metadata within the SOA. It provides:
- Single point of control
- Consistent interfaces
- Detection and protection from data corruption
- Accurate, complete, and synchronized (timely) distribution of policy data to any service that needs it
Key principles of the solution are:
- Services retain responsibility and control of their policies but delegate responsibility for management and retention to a dedicated PMS. This ensures that services can be added and removed from the SOA without complex configuration changes; that services might reside on non-fault-tolerant hardware because they can always recover their policies from the policy service; and that component failure does not incur the need to recover configuration information, and individual service components can safely focus on business and current work rather than policy.
- Work items "request" a version of policy be used rather than the service owning a policy. This ensures that services can be sure they are using the correct policy without needing to perform a remote version check. Old and new versions of a policy can be active at any given time and assured selection of the correct policy for an individual work item by any service. This also supports services concurrently processing work items in different work streams with different policy requirements (work-stream isolation).
- Immutable policy versions and the ability to check policy data completeness supports optimization for distributed and responsive environments. It also allows policy versions to be consistently applied by all instances of a service, regardless of location or proximity.
- By implementing a clear separation of responsibilities and preservation of clear, unambiguous ownership, the solution exhibits consistent policy syntax and semantics, the ability to view policy across multiple services, reduced implementation, and housekeeping costs.
- PMS is responsible for managing and distributing policy information but does not own all policy data, which remains the property of individual (client) services. Centralization of management allows policy to be tested, change-controlled, and moved to production as a signed unit. IT reduces configuration steps to start a new service instance and ensures that services use a consistent name and type management scheme. This leads to a single, simple management interface.
- PMS should minimize code or operational linkage between itself and client applications and must not compromise independence or encapsulation (for example, Java™ object versions). It also shouldn?t create operational dependencies, such as contention or single points of failure.
- The PMS provides a callable management interface. This creates a layered solution that can leverage the SOA user interaction solution (browser, portal, and Eclipse). This approach lets the SOA provide a consistent user interface for all services. To facilitate initial deployment and simple deployments, a simple console might be provided that uses the service through the published management application program interface (API). This approach adds no constraints to the service.
Solution development starts with the data structure and creates software to manipulate, care for, and distribute the policy data. This solution evolves from a simple solution and refined to address new problems and add functionality as required. The result is a solution that addresses the requirements without compromising the SOA or incurring excessive operational costs.
Figure 1. Policy management begins with everything coded into the software
This gives optimal performance at the expense of structure and flexibility. As environments become more complex, the effort of identifying and changing policy becomes expensive and risky. Policy cannot be changed without changing software and is not visible without the source code. Embedded policy cannot be shared and does not support reference data and is not suitable for packaged solutions.
Figure 2. Policy settings in a configuration repository
Placing policy settings in a configuration repository local to the software makes them more visible and easier to change and share. The local store is likely to give reasonable performance and not create complex accessibility and ownership problems. This solves many of the constraints of embedded policy. The local repository approach breaks down when applications wish to share policy across a network and depend on each other to provide timely and accurate services. Local configurations are not visible to people without access to the run-time environment or who do not have detailed knowledge of how the policy is stored and managed on a particular host. Common problems are that configurations are expressed using esoteric syntax and semantics, management requires bespoke consoles or simple text editors, or that local configuration files or database might not easily support enterprise-wide reference data.
Figure 3. Multi-instance services and distributed systems with mutual dependencies
Multi-instance services and distributed systems with mutual dependencies creates intolerable overheads to synchronize all local configurations. Shared repositories are generally databases with published schemas. Reference and configuration data is placed in the database using replication technology. Applications use SQL queries to access the data.
This solution has strong dependencies between the policy repository and the applications using it. The schema cannot change and the database cannot move without changes to all applications. User names and passwords are often stored with each application, but cannot change on a regular basis and are not secure.
Data ownership, performance problems, and being a single point of failure rank among the more common problems with this solution. Performance and availability concerns can be addressed by creating multiple, synchronized images of the repository. Services then need to be load balanced to avoid contention.
Replication copies data operations (insert, update, delete) from one repository to another. The operation is sent to replicate using store-and-forward techniques triggered from a database log file or database triggers. The standard replication model is scalable and provides "some time" synchronization of repositories using asynchronous transportation. It does not guarantee "point-in-time" synchronization.
This PMS vision requires that all services use the correct policy at all times and discards replication on its own as a solution.
Synchronous replication copies data operations using distributed transaction management that applies operations to all copies (almost) simultaneously as a single unit of work. Failure of one causes all to fail. Unfortunately, this technique is operationally expensive and does not scale. Fault fixing, adding, removing, and initially synchronizing replicates are all expensive and involved operations that create tight coupling between instances.
Lightweight Directory Access Protocol (LDAP) is a formalized directory access protocol. Implementations commonly provide a built-in distribution system that can be implemented across the enterprise and networks. LDAP provides a standards-based, hierarchical data structure that allows applications access without security credentials, or by supplying credentials stored in their own target records. The basic model is useful to the PMS distribution vision. It does not have the user name and password dependencies of the simple database approach and provides an extensible and documented schema. LDAP uses replication to push data to its replicates and does not support services selecting specific revisions of records.
Event-driven distribution is a solution that relies on components registering for updates to a set of records. This solution is not suitable for work items that require an older policy and places responsibility on the applications. The push technique itself might not provide the accuracy that the vision demands.
Replication that pushes data with no meaningful feedback from the final recipient cannot assure that every service is applying the correct version of a policy to a given work item. Services are dependent on the update speed of their local repository, and distributed repositories and high-performance SOA amplifies synchronization delays and the level of incorrect policy application. The question remains: How does the solution guarantee that every service uses the correct revision of a policy every time? How does the solution guarantee that it includes all new and back versions of a policy, and can it operate in distributed and responsive environments?
The PMS needs the work item to request a particular version to allow the service to check that it has the correct version. This approach works even for old and very new policies.
Now the service can explicitly request the correct version. While replication can be used to prepopulate distributed caches, the policy-revision request assures the service that it is using the correct version.
The first architectural principle is: Tag every work item with the correct policy revisions.
Figure 4. Tagging work items
The principle doesn?t state how to achieve individually tagging work item with the correct policy revisions. The options are to tag using a generic tag that requires an algorithm to determine the correct revision, or to tag items with explicit policy-revision information.
Generic tags are created from data intrinsic to the message or from dedicated tags added for the sole purpose of policy management.
Intrinsic tags use existing data (for example, date and time) and add meaning to the attribute. The added meaning is not immediately obvious to maintenance engineers and operations staff. Using this method, the flexibility of the additional content is limited and manipulation (for example, to change the policy version applied) is compromised.
Dedicated (extrinsic) tags are often generated from information readily available to the generator. This makes tag generation simple but requires a search algorithm to find the correct revision or version. The search might have to determine the highest revision closest, before, between two dates, or after a date. These algorithms are complex to implement and fail when the actual revision needed is not in the repository or does not conform to the algorithm applied. The algorithm dislocates the explicit request from any specific revision; they destroy the feedback required and are not suitable.
Explicit tags depict a direct connection to a single explicit policy revision. They are added for the sole purpose of policy management.
SOA contains many services, each with one or more policies defined and revised at different levels. If a solution lists all policy-revision pairs that the work item requires, the solution becomes cumbersome and prone to error and confusion. This approach forces this style of tagging to presume that every work item required every service and policy.
By creating and storing a named list of policy revisions that work together, you can create a cohesive set of specific revisions of all the policies in a SOA. It is then possible to use the name of this set of policy revisions as an explicit tag rather than the list itself. The first implementation principle is: Individually tag every work item, on entry to the architecture, with a policy set..
Figure 5. Policy set mapping to policies
Placing one version of each policy in a named set creates a policy set. Work items tagged with the policy set have a direct relationship with those policy revisions. There is no need for each revision to be listed in the work item, and the number of tags required is reduced to one. Groups of work items (work stream) tagged with the same policy set use the same policies and process in a consistent and predictable manner.
Services can use the policy set and policy name to explicitly request the version of the policy they require to process this work item.
A version of a policy might be in many policy sets but cannot be changed once it is deployed in one. "Updates" to the policy cause a new version of the policy to be created and updated.
By using simple checksums or digital signatures, facilitating differentiation between key-not-found (cache incomplete), and invalid-key (don?t make remote request, they don?t know either), you can check for changes and policy completeness. For example, use a sort code and bank account number to access a customer?s profile. If the profile is not found, use the sort code to fetch defaults for this customer.
Successfully deployment of policy and policy sets assures that the retrieved policy, using policy name and policy set, is always correct and no excessive overheads or cache validity problems occur when caching takes place.
Policies are the exclusive responsibility and property of a distinct service but might be used by all instances of that service. This constraint preserves implementation independence and simplifies change management. To preserve this clear demarcation, the PMS needs to provide controls to prevent other teams from updating a policy. A policy belonging to reimplemented or deprecated services can be scrapped to preserve the long-term viability of the solution.
Occasions do exist when one service needs to register the policy with other services (for example, to register reports with a report execution service). Beneficially, the ownership constraint clarifies the state of affairs. First, the report clearly remains the property and responsibility of the first service and is placed in that service?s policy. The PMS needs a mechanism for the report execution service to access that policy setting.
A domain is a class or type and indicates the rules, syntax, and semantics for a piece of data (for example, it must be an integer, float, conform to a picture, and so on). A domain is a name assigned to the rules and format.
Adding a domain to a policy setting lets the policy manager enforce those domain rules and format against the value and identifies that value as belonging to a known, named domain. The policy manager can use domains to allow services to define new domains. The name and validation rules are policy settings and the domain is, for example, PMS$DOMAIN.
The normal way to fetch policy settings is to request a policy from a named policy set and yield all the settings from a specific revision of the policy. The PMS can implement the ability for one service to inform another of the policy by allowing client services to fetch policy settings by domain name across multiple policies in a policy set. For example, Name = My Report, Value = "Report file + instructions.", Domain= REPORT.
Fetching by domain preserves proper ownership rules and prevents cross contamination between policies owned by different services. It provides a simple mechanism for services to register artifacts with other services while retaining ownership.
Changes to a deployed policy cause a new policy-revision to be created in a new policy set. The new policy set needs to be explicitly deployed to be used by production services. Explicit deployment allows changes to be tested, controlled, and authorized. Deployments checksum all policies and the policy set, marking them immutable.
Services joining the SOA fetch their technical policy before accepting work but get the policy set from the work items. New services do not require explicit policy synchronization.
Work items are tagged independently with the correct policy set. A service can be written to support multiple concurrent policies and to accurately apply the correct policy for each work item.
A policy set is assigned to each work item as it enters the architecture. The policy set is the latest set, a named set, or the policy set assigned to a particular work stream. Different gateways processing the same work stream do not need to mutually synchronize if the policy manager supports policy set requests that allow PolicySet = "" or PolicySet = "WS$<Name>" or PolicySet = "PolicySetName", for example.
Work items with a blank policy set are assigned a named policy set by the first service to require that information. The name of the latest policy set is used and the work item is tagged from that point forward.
Work items tagged with a work stream identifier are resolved using the work stream ID in the same way as a named policy set. If the work stream ID alias is already allocated to a policy set, the request might achieve a local cache hit. Otherwise, the request is sent to the policy management service. The service assigns the alias to a policy set and responds with the set and mapping.
If the immutable constraint is not appropriate, the policy must be allowed to change at any time. If refresh is managed by data expiration, as is the case for Web pages, it can be implemented using keywords inside policies or policy sets to inform any caches how long they can retain the data. For example, PolicyCacheDirty is set to "REALTIME" to indicate the policy is checked for changes every time it is used or DateTime / DeltaTime if the policy expires after the period or time specified.
The PMS can be told when expiration is the mechanism using another keyword; for example, PolicyVersionChange set to "On Change" to indicate that changes or addition of new values lead to a new version of the policy.
Policies are organized into policy sets for deployment. A policy set consists of a number of policies owned by individual services. Each policy is named and subject to version control. Deployed policies cannot change and are digitally signed to allow any service to check its authenticity and completeness. Work items dictate the active policy by specifying a policy set. Three important aspects of this include:
- Multiple items processed by different threads might be processed using different policies.
- Policy data can be replicated or cached without risk of using stale data.
- Work items request the correct policy for themselves.
This ensures that work items get exactly the policy they require, while removing the need for complex schemes to synchronize service policy caches.
Policy is constructed, maintained, and deployed through a single, dedicated service, enforcing a single, consistent syntax and domain-controlled semantics for policy. Predictability of policy applied to a work item provides an organization with a powerful diagnostic and management tool.
The solution provides businesses with the ability to make changes to policy in a controlled, traceable way and fulfill their corporate governance needs mandated by industry or federal laws (Sarbanes-Oxley Act 2002).
- The article "Dynamic Cache Policy Editor for WebSphere® Application Server" is on the alphaWorks® site and is a great resource.
- Check out "Understand WS-Policy processing" on the SOA and Web services zone of developerWorks.
- Browse for books on these and other technical topics.
- Want more? The developerWorks SOA and Web services zone hosts hundreds of informative articles and introductory, intermediate, and advanced tutorials on how to develop Web services applications.
- The IBM® developerWorks team hosts hundreds of technical briefings around the world which you can attend at no charge.
Richard works as a WebSphere Business Integration Architect at IBM. He works with many blue-chip companies to define their global integration strategies and implementations. Prior to joining IBM, Richard worked for a variety of companies, including five years as a freelance database consultant and eight years in investment banking.
Table of contents
Next steps from IBM
Rational Tester for SOA Quality is an automated web services testing and SOA testing tool that supports Web services SOAP, HTTP(S), JMS, WS-Security, UDDI, and RESTful standards.
- Try: The no-charge Rational Service Tester for SOA Quality helps quality assurance professionals validate web service based SOA application..
- Article: See the whats new with Rational Service Tester for SOA Quality, and its Generic Service Client, which provides a single client to interact with any SOA service type.
- Tutorial: Get hands-on experience with this Transport for SOAP feature tutorial using Rational Tester for SOA Quality.
- Buy: Rational Service Tester for SOA Quality
Tags
Use the slider bar to see more or fewer tags.
For articles in technology zones (such as Java technology, Linux, Open source, XML), Popular tags shows the top tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), Popular tags shows the top tags for just that product zone.
For articles in technology zones (such as Java technology, Linux, Open source, XML), My tags shows your tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), My tags shows your tags for just that product zone.
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
