One of the biggest challenges in the chemical and petroleum industry is timely access to information that informs business decisions. Oil and gas wells generate a torrent of technical and financial data throughout the exploration and production lifecycle, and that information is invaluable to decision-making. On-time business decisions about operations and production performance impact the effectiveness and efficiency of a plant, a field, and the equipment and assets that support them. However, for most chemical and oil production operations, the information required to make vital business decisions is either not available or comes too late. These challenges can become a barrier to business agility, which we define as an organization's ability to sense environmental change and respond quickly and appropriately to that change.
Cloud computing can improve business agility. It promotes a dialogue between business and IT decision makers and focuses its strength on business service priorities to gain better IT resource utilization, reduced cost of operations, faster business service provisioning, and better alignment of IT dollars spent on business service-level requirements. On one hand, cloud computing is exemplified as an ultimately virtualized system and a natural evolution for data centers that employ automated systems management, workload balancing, and virtualization technologies. On the other hand, cloud computing accommodates a new range of services and applications delivered to customers through various business models, most prominently Software as a Service (SaaS) and Platform as a Service (PaaS).
The IBM® Chemical and Petroleum Integrated Information Framework (IIF) is a unique and powerful real-time integration framework that takes advantage of the cloud to enable companies to improve their decision making across chemical and petroleum business operations. The IIF uses industry standards, web-based visualization, and Service Oriented Architecture (SOA) to provide a set of APIs in the form of web services, RESTful services, and Java™ APIs for the development of process-centric and event-driven applications that can meet the requirements of various enterprises along the value chain of the chemical and petroleum industry.
Figure 1 shows the new delivery model of cloud-enabled IIF. Cloud computing infrastructures help enterprises make more efficient use of their IT hardware and software investments, by breaking down the physical barriers inherent in isolated systems and automating the management of the group of systems as a single entity. PaaS lets you create web applications quickly without the cost and complexity of buying and managing the underlying software and hardware. PaaS helps business users minimize operational costs and increase their productivity.
Figure 1. New model of cloud-enabled IIF
Solution lifecycle management
By leveraging virtualization technologies, cloud-based IIF solution lifecycle management (as the first layer of a cloud-enabled IIF solution), provides the elastic infrastructure for automatic deployment, vertical scaling (scaling out and shrinking), horizontal scaling (scaling up and scaling down), and software upgrades for the whole solution lifecycle with the following phases:
- Creation: Service offerings bearing industrial best practices are predefined as assets for rapid reuse. Users can also customize specialized service offerings according to the individual environments. For instance, in addition to full-solution topologies, customized partial topologies can complement existing solution environments.
- Deployment: The required deployment parameters, including hardware resource configuration, operating system, and application software-level configurations, provide input for the deployed service instance. The service offering defined in the preceding phase is deployed automatically to create the service instances.
- Management: Dynamic scaling and automatic software upgrade capabilities are performed to the elastic infrastructure according to the real monitored performance. The highly scalable and reliable management is vital to successful and economical operation of computing resources in a cloud environment.
The service offering, which depicts the solution topology and configuration, is defined in the creation phase and then instantiated in the deployment phase. The service offering describes the service nodes within the solution topology and the dependency among these service nodes. For each service node, there are configuration parameters, including the specific software configuration parameters, operating system-level configuration, and the underlying hardware resource properties. The startup and shutdown order of the software within the solution topology is also defined in the service offering.
In the deployment phase, the prebuilt or custom service offering can be provisioned into the real physical environments. Naturally, the required hardware and other resources (such as IP address) should be verified as available for the deployed environment. There are three types of parameter values to be consumed by the deployment process. The first type is the user-specified values, such as CPU number, memory size, and some software configuration parameters. The second one is the system-allocated values from the resource pool, such as the IP address. End users cannot manually specify this type of value in order to make the deployed environment controllable. The third type is for other dependent service nodes that denote the constraint and dependency. The deployed service instance is the instantiation of the selected service offering with the values of the required parameters specified.
In the management phase, dynamic scaling and automatic software upgrade capabilities are enabled in the elastic infrastructure according to the real monitored performance. As Figure 2 shows, the automatic deployment capabilities provide the flexible mechanism to build the business agility, while the horizontal and vertical scaling mechanisms are provided as two-dimensional methods to resolve different performance tuning problems. The horizontal scaling denotes the scaling-up and scaling-down capabilities, while the vertical scaling specifies the scaling-out and shrinking capabilities.
Figure 2. Cloud-based lifestyle management
Scaling up and scaling down denote that the resources of the running service nodes can be adjusted based on the real performance monitoring situation. Scaling up increases the allocation of CPU or memory into the deployed service nodes, while scaling down decreases the allocation of CPU or memory resources from the deployed service nodes.
Scaling out and shrinking are other ways to adjust the performance capability of the deployed solution instances. Scaling out can be activated when the workload of the real running environment increases, and the deployed software cluster encounters more pressure. Then the new nodes are provisioned to join the cluster to enhance the computing ability and maintain the solution instance performance. Shrinking works in the opposite way. When the running environment is in idle status, some of the member nodes can be released from the cluster to save resources for another use.
Architecture and components
The architecture and components of cloud-based IIF solution lifecycle management include:
- Management console
- Service offering catalog
- Resource manager
- Provision and automation engine used to instantiate services on the cloud infrastructure.
Figure 3 shows the primary architecture and components of cloud-based IIF solution lifecycle management.
Figure 3. Architecture and components of cloud-based IIF solution lifecycle management
The management console provides a Web 2.0-based user interface to aid the administrator with an at-a-glance view of various services' sub-views, including request for the deployment, monitoring of the deployed environment, and management of the running environment.
Service offering catalog
The service offering catalog contains existing prebuilt and self-customized solution topologies that are stored in XML format. Listing 1 shows the solution topology metadata.
Listing 1. Solution topology metadata
<?xml version="1.0" encoding="UTF-8"?> <solutiontopologies> <solutiontopology> <name>WPS Deployment Manager, WSRR, RAM + 1 WPS Custom Node + DB2 + SISCO UIB Core + SISCO UIB Adapters (Win2003+VMWare)</name> <description>WPS Deployment Manager, WSRR, RAM + 1 WPS Custom Node + DB2 + SISCO UIB Core + SISCO UIB Adapters (Win2003+VMWare)</description> <vmtemplates> <vmtemplate id="54361" name="WPS Deployment Manager, WSRR, RAM"> <properties> <property key="Deployment Environment Name" value="IIF-IVT" /> <property key="Cell Name" value="IIF-IVTCell01" type="hidden"/> <property key="Type" value="dmgr_wsrr_ram" type="hidden"/> <property key="Memory" value="3072" /> <property key="Number of CPUs" value="2" /> </properties> </vmtemplate> <vmtemplate id="54361" name="WPS Custom Node"> <properties> <property key="Node Name" value="aygsoa32Node01" /> <property key="Type" value="node_app_was_msg" type="hidden"/> <property key="Memory" value="3072" /> <property key="Number of CPUs" value="2" /> </properties> </vmtemplate> <vmtemplate id="54437" name="DB2 for WPS, WSRR, RAM, IIF Application, etc"> <properties> <property key="Password for db2admin user" value="password" /> <property key="Type" value="db2" type="hidden"/> <property key="Memory" value="2048" /> <property key="Number of CPUs" value="1" /> </properties> </vmtemplate> <vmtemplate id="54438" name="SISCO UIB Core"> <properties> <property key="Type" value="uibcore_oracle" type="hidden"/> <property key="Memory" value="2048" /> <property key="Number of CPUs" value="2" /> </properties> </vmtemplate> <vmtemplate id="54439" name="SISCO UIB Adapters"> <properties> <property key="Type" value="uibadpaters" type="hidden"/> <property key="Memory" value="2048" /> <property key="Number of CPUs" value="1" /> </properties> </vmtemplate> </vmtemplates> <scale> <out from="template_node_id@type=node_app_was_msg" to="vm_dmgr_id"> </out> </scale> </solutiontopology> </solutiontopologies>
The resource manager is used to manage the resources for deployment. The management console component invokes the resource manager to obtain the resource and component requirements to satisfy the provisioning of the requested service based on the selected service offering. Once these requirements are decided, the resource manager and management console logic component interact with the provisioning engine and configuration metadata to determine the currently available resources and use mapping algorithms to handle the deployment process.
Provision and automation engine
In the implementation, you can leverage an underlying IT resource provisioning engine to execute the decisions the management console makes. This process allows the IT resource virtualization management to be developed and maintained separately from the other components. In our initial implementation, we adopted the IBM Tivoli® Provisioning Manager product, along with some available and extended provisioning workflows. These workflows are largely reusable since they offer basic capabilities such as deployment of a virtual machine image on a particular type of hypervisor platform, or installation of a particular software package in a specified operating system environment. We use the web services API in Tivoli Provisioning Manager to drive the provisioning operations from the management console and resource manager components.
Business process environment in PaaS model
Cloud-based IIF solution lifecycle management serves as a powerful cloud infrastructure layer for the IIF as a platform. However, chemical and petroleum industry application developers still face the complexity of the application logic based on such software portfolios. The IIF itself is a framework, so naturally it supports a platform in a cloud not only for an infrastructure's automatic deployment and resizing, but also, and more importantly, for application development.
Conventionally, the development of process-centric and event-driven applications might include process modeling and coding and testing a set of complex on-premise tools, which might increase the time it takes for even a small application to go into a production environment. The steep learning curve for IIF-specific APIs in the form of web services, RESTful services, and Java APIs can become a major hindrance to time to market. The chemical and petroleum industry requires rapid integration of new business systems and external services to support the rapid changes of supply chain, acquisition, or changes in organizational structures. In some cases, applications only involve the integration of IIF services with simple logic, which does not require complex Java programming. But a different approach is needed to rapidly develop applications by integrating IIF services, user interface components, business processes, databases, and business events for decreased workload, high efficiency, and shorter time to market.
By delivering computing platform over the web through PaaS, you can quickly create applications. The IIF business process environment in the PaaS model eases the workload of the IIF developer, enabling both a lower cost and quicker time to market, as well as an easier integration to the IIF infrastructure. This environment is formed by a set of interrelated web-based rapid online development tools. In this next section, we elaborate on two key features of these tools:
- A visualized programming model with integrated middleware capabilities such as business process, business event, Web 2.0 user interface, and database
- Cloud-based application lifecycle management
Visualized programming model for application development
Applications in the chemical and petroleum industry usually contain logic to monitor and integrate performance data from multiple-site plants with the aid of real-time events generated from devices and instruments. In order to have that data reflected in a business perspective, a dashboard-enabled user interface is needed to present on-time or historical data.
Additionally, if you want to diagnose problems, you usually need to activate processes to calculate key performance indicators (KPIs) in response to the above events, which are also shown in user interface pages for analysis. To meet such business requirements, you need software bundled to support processes, events, user interfaces, and databases, and that software inevitably challenges IIF developers' skills and development costs.
We present a visualized programming model with integrated middleware that includes all the above mentioned capabilities to help mitigate the obstacles. As Figure 4 shows, a widget named Service Assembler acts as a visualized business process editor. In the palette on the left, the basic building blocks for a general process are listed in the drawer named Integration Elements, and reusable web services and RESTful services already hosted in the IIF solution runtime are listed in the drawers named RSM Web Services, KPI Web Services, and KPI RESTful Services.
These services help IIF developers to draft any business processes meant to manipulate with RSM (Reference Semantic Model) models and in-monitor KPIs of an IIF solution. The main canvas on the right serves as the business process modeling view in which developers simply drag and drop composing elements from the palette and use arrows to link them together. Under the modeling view, a property view is presented to allow developers to conveniently specify configuration parameters for each element of the process. Developers without any programming language skills can easily fulfill the task, but only if they have general knowledge of XML, Web Services Description Language (WSDL), XML Schema Definition (XSD), and web service invocation.
Figure 4. Service Assembler, a visualized business process editor
The Mashup Callout shown in Figure 4 is a specialized element that helps developers to swiftly link certain steps within a process with a user interface implementation. Application developers using the IIF can create user interface logic in the unit of widgets and then register them into any widget mashup software. The Mashup Callout element together with a paired element, Mashup Callin, helps to specify which widget the process will be linked to and what content of the XSD formatted input/output parameter values should be exchanged between the process and the target widget.
Database is another specialized element used to include off-premise database capability in the process. In the property view of this element, developers can insert basic database CIUD (create, insert, update, delete) operations visually without regard to the backend database connection configuration and writing SQL statements.
The visualized programming model also can be used to build up the link between business events and processes to further coincide with the IIF application requirement. We allow web-based online business event template creation by uploading predefined event syntax in XSD format. We also allow association of the created business event template with business processes by matching event entries with process input parameters to form an event-driven process paradigm. For example, an equipment monitoring event that carries real-time equipment diagnosed data is linked with a business process authored from the Service Assembler that validates the working status of this machine and checks KPI violations.
With such an integrated IIF business process environment in the PaaS model, developers can dramatically shorten the application development cycle, and the resulting business process is ready for further transformation to runnable format.
Cloud-based application lifecycle management
In addition to the visualized programming model, cloud-based application lifecycle management strengthens the consumability of the entire IIF business process environment in the PaaS model.
Figure 5. Cloud-based application lifecycle management: Service provider and cloud administrator
As Figure 5 shows, the application lifecycle management capability serves two roles: The service provider who acts as the IIF developer to do development-related affairs, and the cloud administrator who performs cloud resource allocation and authorizes service provider requests.
Application lifecycle management is incorporated into the PaaS model and also involves several widgets as web-based user interfaces. Both the developer and the administrator share the same login gate page, and after signing in, are routed to different role-specific user interface paths. The service provider is first required to register any external web services that will be further leveraged by process development. Then that service provider will use all web-based tools mentioned in the Visualized programming model for application development section to develop an IIF application online.
Finally, after choosing a suitable topology maintained by cloud-based IIF solution lifecycle management as the target deployment environment, the service provider submits the application deployment request. A cloud administrator first sees a list to review all pending requests from the service provider. Then after resource justification, the cloud administrator can approve or deny each of them:
- If approved, the application is automatically installed onto the selected IIF solution and then be functional after a reasonable time duration.
- Otherwise, the corresponding service provider gets a rejection notification for request modification.
The role-based application lifecycle management mode supports publish-subscribe and the business model for dynamic application composition and instantiation in order to be more compliant with cloud management concepts.
Business agility is vitally important to the chemical and petroleum industry, especially in rapid response to diagnosis and exchange of real-time information and other relevant data sources. Cloud computing solutions in the forms of elastic infrastructure, PaaS, and SaaS, and certain cloud-enabled investments will have implications for version migration and integration.
This second part of a series on cloud-enabled industry solutions demonstrated how cloud computing technologies and architecture patterns significantly enhance the capability and agility of the chemical and petroleum industry and potentially other industries with similar requirements, business models, and IT architectures. This article presented the capabilities and implementation mechanism of cloud-based IIF solution lifecycle management and showed how a business process environment using the PaaS business model can lead to rapid development of business agility.
- Cloud Computing Use Cases
- IBM Chemical and Petroleum Integrated Information Framework
- "Cloud, SaaS, Hosting and Other Off-Premises Computing Models" (registration required), (Gartner, July 2008)
- IBM WebSphere CloudBurst Appliance information center
- IBM Tivoli Service Automation Manager information center
- In the developerWorks cloud developer resources, discover and share knowledge and experience of application and services developers building their projects for cloud deployment.
- Find technical articles, tutorials, communities, wikis, and business resources that should be interesting to you if you work in a specific industry domain on IBM developerWorks Industry zone.
Get products and technologies
- Try this single, fully configured image for development and deployment of industry vertical applications using WebSphere® Application Server and DB2® Express-C.
- Join a cloud computing group in the developerWorks community.
- Read all the great cloud blogs in the developerWorks community.
- Join the developerWorks community, a professional network and unified set of community tools for connecting, sharing, and collaborating.
Dig deeper into Cloud computing on developerWorks
Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.
Complete cloud software, infrastructure, and platform knowledge.
Software development in the cloud. Register today to create a project.
Deploy public cloud instances in as few as 5 minutes. Try the SoftLayer public cloud instance for one month.