Plug-in development steps

Plug-ins contribute components, links, and policies that you use in the Pattern Builder pane to create virtual application patterns, or extend existing virtual application patterns.

Before you begin

You can develop your own custom plug-ins. Custom plug-ins add behavior and function that users can use to enhance and customize the operation of their virtual applications and create virtual application types.

Download the Plug-in Development Kit (PDK). You can also download the PDK from the Cloud Pak System user interface Welcome page.

About this task

Consider the following design principles as you plan and develop your plug-ins:

Plug-ins are designed for application-centric usage. Components, links, and policies represent application artifacts and quality of service levels, not explicit infrastructure or middleware artifacts. For example, web application artifacts might include a WAR file and a database instance. A link is required to associate WAR files with the database instances. Possible extensions that you might make available to a deployer include support for load balancing of the web application. Scaling is a quality of service, which would be modeled as a policy object on the WAR file component. Adding the scaling policy might result in more processes in the deployment such as proxy or a cache for session replication, but the plug-in handles the details that are based on the intent of the deployer.
The design must consider application management. For example, deployers might want to replace the WAR file on the running deployment.
There are different approaches to plug-in design and customization. You might choose to create a pattern type to isolate the user experience. Or you can add a plug-in to an existing pattern type so that deployers have access to capabilities in the new and the existing plug-ins.
Plug-ins can extend capability in various places, as follows:
- At design time by through the components, links, policies, and the configuration options that you display
- At deployment time by using binary files and automation scripts
- Transformation logic that is used to convert the logical model of a virtual application pattern to the deployed topology
- Post-deployment management options that you make available

Use the following steps to develop your own custom plug-ins. The plug-in can be developed in the Eclipse tool or the integrated development environment (IDE) of your choice.

Procedure

Define and package plug-in artifacts.
1. Define the config.json file.
  The config.json file is the only required file in a plug-in. The name, version, and patterntypes elements are all required. The name element specifies the name of the plug-in, and the version element defines the version number of the plug-in. The patterntypes element specifies the pattern types with which the plug-in is associated, and you must specify one primary or one secondary pattern type at minimum. You can specify only one primary pattern type, but you can associate the plug-in with one or more secondary pattern types.
  The following example is a WebSphere® Application Server Community Edition plug-in that extends the IBM® Web Application Pattern type:
```
{
   "name"    : "wasce",
   "version" : "1.0.0.1",
   "patterntypes":{
   "secondary":[{ "*":"*"}]
   },
   "packages" : {
      "WASCE" : [ {
            "requires"  : {
               "arch"   : "x86_64",
               "memory" :  512,
               "disk"   :  300
            },
            "parts":[ {
                  "part"  : "parts/wasce.tgz",
                  "parms" : {
                     "installDir" : "/opt/wasce"
                  }
            } ], 
      "WASCE_SCRIPTS":[ {
            "parts":[ {
                  "part":"parts/wasce.scripts.tgz"
            } ]
      } ]
      } ]
   }
         "update":{
         "type":"rolling",
         "constraint":{
           "was":"2.0.0.5"
          }
                    }
}
```
  This example defines most of the following common elements:
  - patterntypes element:
    The value is specified as webapp.
    
    This means that the capabilities contributed by this plug-in are available when you create patterns from the Web Application Pattern. No primary element and a secondary element of *:* means that it shows up in the Pattern Builder for all pattern types.
  - requires element:
    This element contains other elements that specify resource requirements of the plug-in.
    
    os
    Specifies the operating system that the plug-in requires.
    
    arch
    Specifies the virtual machine architecture that the plug-in requires. In the previous config.json sample code, the specified architecture is 64 bit, X86.
    
    cpu
    Specifies the minimum processing capacity that is required for each package that is defined by your plug-in. The requires element specifies the required attributes of the package, all parts, and node-parts in it. For cpu, it represents the total required resources of each type for all parts and node-parts in the package.
    
    memory
    Specifies the minimum memory requirement, in megabytes, for each package that is defined by your plug-in. The requires element specifies the required attributes of the package, all parts and node-parts in it. For memory, it represents the total required resources of each type for all parts and node-parts in the package.
    
    disk
    Specifies the minimum disk requirement, in gigabytes, for each package that is defined by your plug-in. The requires element specifies the required attributes of the package, all parts and node-parts in it. For disk, it represents the total required resources of each type for all parts and node-parts in the package.
    
    Note: During the provisioning process, Cloud Pak System adds up the minimum processor, memory, and disk values for each package, and provisions a virtual machine that meets the specified requirements. The disk value is converted to megabytes when it is stored in the topology document, so if the required sizes exceed the size of the available images the value might be shown as megabytes in an error message.
  - packages element:
    Defines the file packages with both the part and node part elements. The example plug-in provides two packages: WASCE and WASCE_SCRIPTS. The WASCE package contains the parts/wasce.tgz part file. This archive contains the wasce image - all the files that compose wasce. The binary files are required to install WebSphere Application Server Community Edition, and package it directly in the plug-in.
    
    There are other options for specifying the required binary files. You can define a file attribute and have administrators upload the required binary files after the plug-in is loaded in Cloud Pak System. You can also link to a remote server that stores the required artifacts. The WASCE_SCRIPTS package provides the lifecycle scripts to install the WASCE image to the wanted location to install the enterprise archive (EAR) or web archive (WAR) file, and to start the server.
    
    If you want to configure some parts for a specific configuration and have other parts apply to all configurations, you can use multiple requires elements.
    
    You can use the requires element with an arch element to specify that a specific architecture is required for the parts. Only the parts for the current architecture and parts with no specified architecture requirements are loaded when the plug-in is deployed.
    
    If the plug-in is supported on more than one architecture and the parts are different for each architecture, you can use multiple requires elements. For example, if your config.json file contains the following code, then only the parts that include "binaryFile":was/was-8005-linux32-20121115-product.tgz and "binaryFile":was/was-8005-linux32-20121115-patch.tgz are loaded when the plug-in is deployed on Cloud Pak System 8283. If the same plug-in is deployed on Cloud Pak System 8278, then only the parts that include "binaryFile":was/was-8005-aix-ppc32-20121115-product.tgz and "binaryFile":was/was-8005-aix-ppc32-20121115-patch.tgz are loaded.
    { "name": "was", "version": "2.0.0.5", "files": [ "was/was-8005-linux32-20121115-product.tgz", "was/was-8005-linux32-20121115-patch.tgz", "was/was-8005-linux64-20121115-product.tgz", "was/was-8005-linux64-20121115-patch.tgz", "was/was-8005-aix-ppc32-20121115-product.tgz", "was/was-8005-aix-ppc32-20121115-patch.tgz", "was/was-8005-aix-ppc64-20121115-product.tgz", "was/was-8005-aix-ppc64-20121115-patch.tgz" ], ... "packages": { "WAS32": [ { "requires": { "arch": "x86_64", "memory": 640, "disk": 726 }, "provides": { "middleware": [ { "product": "IBM WebSphere Application Server", "version": "8.0.0.3" } ] }, "parts": [ { "part": "parts/was.tgz", "parms": { "installDir": "/opt", "binaryFile": "was/was-8005-linux32-20121115-product.tgz" } }, { "part": "parts/was-ifixmgr.tgz", "parms": { "installDir": "/", "binaryFile": "was/was-8005-linux32-20121115-patch.tgz" } } ] }, { "requires": { "arch": "ppc_64", "os": { "AIX": "*" }, "memory": 640, "disk": 726 }, "provides": { "middleware": [ { "product": "IBM WebSphere Application Server", "version": "8.0.0.3" } ] }, "parts": [ { "part": "parts/was.tgz", "parms": { "installDir": "/opt", "binaryFile": "was/was-8005-aix-ppc32-20121115-product.tgz" } }, { "part": "parts/was-ifixmgr.tgz", "parms": { "installDir": "/", "binaryFile": "was/was-8005-aix-ppc32-20121115-patch.tgz" } } ] } ], ...
    For more information about using a files element, see the topic "Pattern type packaging reference".
  - update element.
    This element is a JSONObject with the following attributes:
    
    type: String that indicates the update type. Valid values are rolling or batch. The default value is rolling. If a virtual application deployment update type is calculated as rolling, the virtual machines in the virtual application instance are divided into two groups by default. While one group is being updated, the virtual machines in the other group are running to prevent service loss. For more information about how the update type is calculated, see step 3.
    
    constraint: A JSONObject that contains a list of key:value pairs that represent a plug-in scope, such as "was":"2.0.0.5". This attribute is only supported for the rolling update type, and is used to indicate that if the topology contains the plug-ins that are defined in this attribute, the rolling update does not work. In this scenario, the update is done in batch mode.
    
    singlevmgroup: Added in Cloud Pak System version 1.1.0.2. Boolean value. Valid values are true or false. This attribute is only supported for the rolling update type. If you set singlevmgroup to true, you indicate that this plug-in requires the virtual machines in the deployment to be updated one at a time.
    
    Note:
    
    When a virtual application pattern is deployed, the deployment fails during the transformation phase if the update element is used in the transformer code but supported values or types are not defined. In this scenario, an error is displayed on the console
Define configurable application model components.
The web and enterprise application archive components are displayed in the Pattern Builder. Each component is specified in the metadata.json file that is in the plugin/appmodel directory of the plug-in archive and plug-in development project. The following example illustrates the JSON to define the web archive component. Thumbnail images must be 48 x 48 pixels.
```
[{
     "id"          : "WARCE",
     "label"       : "Web Application (WebSphere Application Server Community Edition)",
     "description" : "A web application cloud component represents an execution service for Java EE Web applications (WAR files).",
     "type"        : "component",
     "thumbnail"   : "appmodel/images/WASCE.png",
     "image"       : "appmodel/images/WASCE.png",
     "category"    : "application",
     "attributes"  : [
         {
             "id"          : "archive",
             "label"       : "WAR File",
             "description" : "Specifies the web application (*.war) to be uploaded.",
             "type"        : "file",
             "required"    : true,
             "extensions"  : [ "war" ] 
         }
     ] 
}] 
```
There is a similar stanza for the enterprise archive component for its downloadable archive.
The first type field of the listing is important. The value options for this field are component, link or policy, and this field defines the type in the application model. The ID of the component is WARCE. The ID can be any value if it is unique.

The category refers to the tab under which this component is shown on the pane in the Pattern Builder. The attributes array defines properties for the component that you are defining. You can see and are able to specify values for these properties when you use this component in the Pattern Builder. Attribute types include file, string (shown here), number, boolean, array, and range.
Define a template to convert the visual model into a physical model.
Plug-ins must provide the knowledge and logic for how to implement, or realize, the deployment of the defined components. In the next example, the meaning of how to deploy an enterprise or web application component must be specified. A single transform is provided which translates the application model that is derived from what users build in the Pattern Builder into a concrete topology.
The following example displays a Velocity template that represents a transformation of the component into a JSON object that represents a fragment of the overall topology document. Each component and link must have a transform. In this plug-in, the WARCE and EARCE components share the transform template.
```
{
   "vm-templates":[
      {
         "name"     : "${prefix}-wasce",
         "packages" : [ "WASCE", "WASCE_SCRIPTS" ],
         "roles"    : [
            {
               "plugin"       : "$provider.PluginScope",
               "name"         : "WASCE",
               "type"         : "WASCE",
               "quorum"       : 1,
               "external-uri" : [{"ENDPOINT":"http://{SERVER}:8080"}],
               "parms":{
                  "ARCHIVE"   : "$provider.generateArtifactPath( $applicationUrl, ${attributes.archive} )"
               },
               "requires"     : { "memory":512, "disk":300 }
            }
         ],

        "scaling" : { "min":1, "max":1 }
      }
   ]
}
```
The topology fragment is a JSON object that contains a vm-templates element, which is an array of vm-templates. A vm-template is a virtual machine template, and defines the parts, node parts, and attributes of a virtual machine to be deployed. For this example, only a single vm-template that contains four important elements is needed:
- name: Specifies a unique name for a deployed virtual machine.
- packages: Specifies a list of parts and node parts that are installed on each deployed virtual machine. The WASCE entry indicates the use of the WASCE virtual image. The WASCE_SCRIPTS entry specifies the WASCE lifecycle scripts.
- roles: Specifies parts in a plug-in that start lifecycle scripts for roles. You can have one or more roles in your plug-in, but in the sample plug-in there is a single WASCE role. When all roles on a node go to the RUNNING state, the node changes to the green RUNNING state.
- requires: Specifies the minimum memory requirements, in megabytes, disk requirements, in gigabytes, and architecture that are needed for the deployed virtual machine.
Note: When a virtual application pattern is deployed, the overall update type for the vm-template is calculated based on the update type that is set for all of the plug-ins in their config.json files, and is added to the topology document, topology.json. The update mode for a deployment is set to rolling in the vm-template only if all the plug-ins in the vm-template support the rolling update mode. If a topology has more than one vm-template, each vm-template must support rolling for the update mode for the deployment to be set to rolling. For example, if the topology contains two vm-templates, one for WebSphere Application Server, which is configured for rolling and one for DB2®, which is configured for batch, the deployment uses batch mode.
Starting in Cloud Pak System 1.1.0.2, if the vm-template update type is rolling, and one plug-in requires that the rolling update is done one virtual machine at a time, by setting "singlevmgroup" : true, then that option is set to true for that vm-template. If all of the vm-templates for a deployment support a rolling update, and one vm-template has "singlevmgroup" : true set, then that option is recognized. In this scenario, each group for the rolling update contains only one virtual machine that is updated, and the virtual machines are updated one at a time. For example, if five virtual machines must be updated in the deployment, there are five rolling groups. Each group contains one virtual machine.
Define lifecycle scripts to install, configure, and start software.
In this step, you define the lifecycle scripts for the plug-in. This process includes writing scripts to install, configure, and start the plug-in components. You can view the complete scripts in the downloadable archives. The following information includes the key artifacts:
- install.py script
  The install.py script copies the WASCE image from the download location to the wanted installDir folder. It also sets the installDir value in the environment for subsequent scripts. All parts and node parts that are installed by the Cloud Pak System agent are run as root. The chown –R virtuser:virtuser command changes file ownership of the installed contents to the wanted user and group. Finally, the install.py script makes the scripts in the WebSphere Application Server Community Edition bin directory executable files. The following sample code is the contents of the install.py script:
  installDir = maestro.parms['installDir'] maestro.trace_call(logger, ['mkdir', installDir]) if not 'WASCE' in maestro.node['parts']: maestro.node['parts']['WASCE'] = {} maestro.node['parts']['WASCE']['installDir'] = installDir # copy files to installDir to install WASCE this_file = inspect.currentframe().f_code.co_filename this_dir = os.path.dirname(this_file) rc = maestro.trace_call(logger, 'cp -r %s/files/* %s' % (this_dir, installDir), shell=True) maestro.check_status(rc, 'wasce cp install error') rc = maestro.trace_call(logger, ['chown', '-R', 'virtuser:virtuser', installDir]) maestro.check_status(rc, 'wasce chown install error') # make shell scripts executable rc = maestro.trace_call(logger, 'chmod +x %s/bin/*.sh' % installDir, shell=True) maestro.check_status(rc, 'wasce chmod install error')
  This example shows how the script uses the maestro module that is provided within the plug-in framework. The module provides several helper methods that are useful during installation and elsewhere.
- wasce.scripts part and install.py script
  The wasce.scripts part also contains an install.py script. This script installs the WebSphere Application Server Community Edition lifecycle scripts. The following example illustrates the contents of the install.py script in wasce.scripts:
  # Prepare (chmod +x, dos2unix) and copy scripts to the agent scriptdir maestro.install_scripts('scripts')
- configure.py script
  The configure.py script in the wasce.scripts part installs the user-provided application to WebSphere Application Server Community Edition. The script takes advantage of the hot deployment capability of WebSphere Application Server Community Edition and copies the application binary files to a monitored directory. The following example includes the contents of the configure.py script:
  installDir = maestro.node['parts']['WASCE']['installDir'] ARCHIVE = maestro.parms['ARCHIVE'] archiveBaseName = ARCHIVE.rsplit('/')[-1] # Use hot deploy deployDir = os.path.join(installDir, 'deploy') if os.path.exists(deployDir) == False: # Make directories os.makedirs(deployDir) deployFile = os.path.join(deployDir, archiveBaseName) # Download WASCE archive file maestro.download(ARCHIVE, deployFile)
- start.py
  The start.py script in the wasce.scripts part is responsible for starting the WebSphere Application Server Community Edition process. After the process is started, the script updates the state of the role to RUNNING. When the deployment is in the RUNNING state, you can access the deployed application environment. The following example shows the use of the geronimo.sh start command to start WebSphere Application Server Community Edition, and the gsh.sh command to wait on startup:
  wait_file = os.path.join(maestro.node['scriptdir'], 'WASCE', 'wait-for-server.txt') installDir = maestro.node['parts']['WASCE']['installDir'] rc = maestro.trace_call(logger, ['su', '-l', 'virtuser', installDir + '/bin/geronimo.sh', 'start']) maestro.check_status(rc, 'WASCE start error') logger.info('wait for WASCE server to start') rc = maestro.trace_call(logger, ['su', '-l', 'virtuser', installDir + '/bin/gsh.sh', 'source', wait_file]) maestro.check_status(rc, 'wait for WASCE server to start error') maestro.role_status = 'RUNNING' logger.info('set WASCE role status to RUNNING') logger.debug('Setup and start iptables') maestro.firewall.open_tcpin(dport=1099) maestro.firewall.open_tcpin(dport=8080) maestro.firewall.open_tcpin(dport=8443)
Draft comment:
The following illustrates the view of all the parts in the wasce plug-in as seen from the Package Explorer view in Eclipse: NEED to ADD directory graphic here

Other scripts and artifacts make up the plug-in, but the preceding example provides an explanation of the most significant scripts.

What to do next

Add your custom plug-in to Cloud Pak System where the plug-in can be used to create or extend a virtual application.