Script package examples

• Add IBM HTTP Server node
• Disable Root Login
• WebSphere Application Server Samples
• Vertical CPU Scaling
• Vertical Memory Scaling

cbscript.json

This file contains the following parameter information:

[
   {
      "command": "\/tmp\/disableRootLogin\/disableRootLogin.sh",
      "description": "Disables Root Login for AIX and Linux",
      "execmode": 0,
      "filename": "disableRootLogin.zip",
      "location": "\/tmp\/disableRootLogin\/",
      "log": "\/tmp\/disableRootLogin\/logs",
      "name": "Disable Root Login",
      "ostype": "linux\/unix",
      "timeout": 0,
      "type": "APPLICATION"
   }
]

To modify the script package to run manually, change the value of execmode to 2.

extendedattributes.json

This file contains the following parameter information:

[
   {
      "savevars": "0"
   }
]

The savevars parameter controls whether environment variables are saved after post-deployment executions. A value of 0 indicates that variables are not saved.

disableRootLogin.sh

This is the script that is called when the script package is run.

README.TXT

Some basic information is included in this file.

su $WAS_USERNAME -c "$WAS_INSTALL_ROOT/samples/bin/install.sh"

cbscript.json

This file contains the following parameter information:

[
   {
      "command": "sh \/tmp\/installSamples.sh",
      "execmode": 0,
      "filename": "samples.zip",
      "location": "\/tmp",
      "log": "${WAS_INSTALL_ROOT}\/logs ${WAS_PROFILE_ROOT}\/logs",
      "name": "WebSphere Application Server Samples",
      "ostype": "linux\/unix",
      "timeout": 0,
      "type": "APPLICATION"
   }
]

Note that this script supports only stand-alone WebSphere Application Server. It cannot be run in a cluster.

extendedattributes.json

This file contains the following parameter information:

[
   {
      "savevars": "0"
   }
]

The savevars parameter controls whether environment variables are saved after post-deployment executions. A value of 0 indicates that variables are not saved.

installSamples.sh

This is the script that is called when the script package is run.

cbscript.json

This file contains the following parameter information:

[
   {
      "command": "\/tmp\/VerticalScaleGenericCPU\/enable_as.sh",
      "description": "A generic scaling policy that uses operating system metrics to scale the number of vCPUs allocated to a virtual machine.",
      "execmode": 0,
      "filename": "VerticalScaleGenericCPU.zip",
      "keys": [
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "1",
            "scriptkey": "MIN_CPU_COUNT"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "8",
            "scriptkey": "MAX_CPU_COUNT"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "80",
            "scriptkey": "SCALE_UP_CPU_THRESHOLD"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "0",
            "scriptkey": "SCALE_DOWN_CPU_THRESHOLD"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "300",
            "scriptkey": "TRIGGER_TIME"
         }
      ],
      "location": "\/tmp\/VerticalScaleGenericCPU",
      "log": "\/tmp\/VerticalScaleGenericCPU",
      "name": "VerticalScaleGenericCPU",
      "ostype": "linux\/unix",
      "timeout": 0,
      "type": "APPLICATION"
   }
]

The following environment variables are defined:

MIN_CPU_COUNT

This is a required variable that indicates the minimum number of vCPUs that the virtual machine must have. The default value is 1.When the number of vCPUs reaches this minimum number, the scale down operation cannot be performed.

MAX_CPU_COUNT

This is a required variable that indicates the maximum number of vCPUs to which the virtual machine can be scaled. The default value is 8.

SCALE_UP_CPU_THRESHOLD

This is a required variable that indicates the percent threshold of CPU usage that, when exceeded, triggers a scale up operation to be automatically performed on the virtual machine. This threshold must be exceeded for the duration of time indicated by the TRIGGER_TIME variable before scaling is performed. The default value is 80 percent.

SCALE_DOWN_CPU_THRESHOLD

This is a required variable that indicates the percent threshold of CPU usage that triggers a scale down operation to be automatically performed on the virtual machine. The percentage of CPU usage must be below this threshold for the duration of time indicated by the TRIGGER_TIME variable before scaling is performed. The default value is 0 percent, indicating no scale down is performed. You should set this to an appropriate threshold level for your environment.

TRIGGER_TIME

This is a required variable that indicates the time, in seconds, that a threshold must be exceeded before a scaling operation is performed. The default value is 300 seconds. For example, if the CPU usage on a virtual machine exceeds the SCALE_UP_CPU_THRESHOLD value (80%) for 300 seconds (5 minutes), then a scale up operation is triggered to increase the vCPU count.

extendedattributes.json

This file contains the following parameter information:

[
   {
      "savevars": "0"
   }
]

The savevars parameter controls whether environment variables are saved after post-deployment executions. A value of 0 indicates that variables are not saved.

enable_as.sh

This is the script that is called when the script package is run.

CPU changed for virtual machine auslpas165-OS Node-scalingCPUTest-766 from 2 cores to 3 cores

cbscript.json

This file contains the following parameter information:

[
   {
      "command": "\/tmp\/VerticalScaleGenericMemory\/enable_as.sh",
      "description": "A generic scaling policy that uses operating system metrics to scale the amount of memory allocated to a virtual machine.",
      "execmode": 0,
      "filename": "VerticalScaleGenericMemory.zip",
      "keys": [
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "4096",
            "scriptkey": "MIN_MEMORY"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "8192",
            "scriptkey": "MAX_MEMORY"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "80",
            "scriptkey": "SCALE_UP_MEMORY_THRESHOLD"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "0",
            "scriptkey": "SCALE_DOWN_MEMORY_THRESHOLD"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "300",
            "scriptkey": "TRIGGER_TIME"
         }
      ],
      "location": "\/tmp\/VerticalScaleGenericMemory",
      "log": "\/tmp\/VerticalScaleGenericMemory",
      "name": "VerticalScaleGenericMemory",
      "ostype": "linux\/unix",
      "timeout": 0,
      "type": "APPLICATION"
   }
]

The following environment variables are defined:

MIN_MEMORY

This is a required variable that indicates the minimum amount of memory that the virtual machine must have. The default value is 4096.When the amount of memory reaches this minimum number, the scale down operation cannot be performed.

MAX_MEMORY

This is a required variable that indicates the maximum amount of memory to which the virtual machine can be scaled. The default value is 8192.

SCALE_UP_MEMORY_THRESHOLD

This is a required variable that indicates the percent threshold of memory usage that, when exceeded, triggers a scale up operation to be automatically performed on the virtual machine. This threshold must be exceeded for the duration of time indicated by the TRIGGER_TIME variable before scaling is performed. The default value is 80 percent.

SCALE_DOWN_MEMORY_THRESHOLD

This is a required variable that indicates the percent threshold of memory usage that triggers a scale down operation to be automatically performed on the virtual machine. The percentage of memory usage must be below this threshold for the duration of time indicated by the TRIGGER_TIME variable before scaling is performed. The default value is 0 percent, indicating no scale down is performed. You should set this to an appropriate threshold level for your environment.

TRIGGER_TIME

This is a required variable that indicates the time, in seconds, that a threshold must be exceeded before a scaling operation is performed. The default value is 300 seconds. For example, if the memory usage on a virtual machine exceeds the SCALE_UP_MEMORY_THRESHOLD value (80%) for 300 seconds (5 minutes), then a scale up operation is triggered to increase the amount of memory allocated to the virtual machine.

extendedattributes.json

This file contains the following parameter information:

[
   {
      "savevars": "0"
   }
]

The savevars parameter controls whether environment variables are saved after post-deployment executions. A value of 0 indicates that variables are not saved.

enable_as.sh

This is the script that is called when the script package is run.

Memory changed for virtual machine auslpas158-Standalone from 2048 to 3072

cbscript.json

This file contains the following parameter information:

[
   {
      "command": "\/bin\/sh \/tmp\/WASHV_CPUBased\/enable_as.sh",
      "description": "A WASHV script example for scaling up based CPU",
      "execmode": 0,
      "filename": "WASHV_CPUBased.zip",
      "keys": [
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "1",
            "scriptkey": "MIN_CPU_COUNT",
            "scriptvalue": "1"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "8",
            "scriptkey": "MAX_CPU_COUNT",
            "scriptvalue": "8"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "80",
            "scriptkey": "SCALE_UP_CPU_THRESHOLD",
            "scriptvalue": "80"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "0",
            "scriptkey": "SCALE_DOWN_CPU_THRESHOLD",
            "scriptvalue": "0"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "120",
            "scriptkey": "TRIGGER_TIME",
            "scriptvalue": "120"
         }
      ],
      "location": "\/tmp\/WASHV_CPUBased",
      "log": "\/tmp\/WASHV_CPUBased",
      "name": "WAS_VerticalScalingCPUScriptPkg",
      "ostype": "linux\/unix",
      "timeout": 0,
      "type": "APPLICATION"
   }
]

The following environment variables are defined:

MIN_CPU_COUNT

This is a required variable that indicates the minimum number of vCPUs that the virtual machine must have. The default value is 1.When the number of vCPUs reaches this minimum number, the scale down operation cannot be performed.

MAX_CPU_COUNT

This is a required variable that indicates the maximum number of vCPUs to which the virtual machine can be scaled. The default value is 8.

SCALE_UP_CPU_THRESHOLD

This is a required variable that indicates the percent threshold of CPU usage that, when exceeded, triggers a scale up operation to be automatically performed on the virtual machine. This threshold must be exceeded for the duration of time indicated by the TRIGGER_TIME variable before scaling is performed. The default value is 80 percent.

SCALE_DOWN_CPU_THRESHOLD

This is a required variable that indicates the percent threshold of CPU usage that triggers a scale down operation to be automatically performed on the virtual machine. The percentage of CPU usage must be below this threshold for the duration of time indicated by the TRIGGER_TIME variable before scaling is performed. The default value is 0 percent, indicating no scale down is performed. You should set this to an appropriate threshold level for your environment.

TRIGGER_TIME

This is a required variable that indicates the time, in seconds, that a threshold must be exceeded before a scaling operation is performed. The default value is 120 seconds. For example, if the CPU usage on a virtual machine exceeds the SCALE_UP_CPU_THRESHOLD value (80%) for 120 seconds (2 minutes), then a scale up operation is triggered to increase the vCPU count.

extendedattributes.json

This file contains the following parameter information:

[
   {
      "savevars": "0"
   }
]

The savevars parameter controls whether environment variables are saved after post-deployment executions. A value of 0 indicates that variables are not saved.

enable_as.sh

This is the script that is called when the script package is run.

resource.py

This is the script that is run automatically to perform the middleware tuning to take advantage of the change in CPU resource. In this example, the scripts/was_web_thread.py script is called, which adjusts the thread pool size for WebSphere Application Server.

The script contains the following code:

import sys
import subprocess
import json

parms = json.loads(sys.argv[1])

if int(parms['newCpuCount']) > int(parms['oldCpuCount']):
   cpu = parms['newCpuCount']
   retcode = subprocess.call(['sh', '/opt/IBM/WebSphere/AppServer/bin/wsadmin.sh', '-lang', 'jython', '-f', '/tmp/WASHV_CPUBased/scripts/was_web_thread.py', str(cpu)])
   print 'wsadmin.sh return code:', retcode

was_webresource.py

This script is located in the scripts folder, and is called by resource.py to perform the middleware tuning to adjust the thread pool size for WebSphere Application Server.

The script contains the following code:

import AdminUtilities

def getName(objectId):
    endIndex = (objectId.find("(c") - 1)
    stIndex = 0
    if (objectId.find("\"") == 0):
        stIndex = 1
    return objectId[stIndex:endIndex+1]

assert len(sys.argv) == 1
target_cpu = int(sys.argv[0])

setMaxInt = 20 * target_cpu

setMaxStr = str(setMaxInt)
print "Set WebContainer's thread pool max size: %s, min size: %s" % (setMaxStr, setMaxStr)

try:
    theList = AdminControl.completeObjectName('WebSphere:*,type=ThreadPool,name=WebContainer')
    theList = theList.splitlines()
    
    for tp in theList:
        if tp.find('WebContainer') != -1:
            currMinSize = AdminControl.invoke(tp, 'getMinimumPoolSize')
            currMinSize = int(currMinSize)
            currMaxSize = AdminControl.invoke(tp, 'getMaximumPoolSize')
            currMaxSize = int(currMaxSize)

            if currMaxSize < setMaxInt:
                print "currMaxSize is %s" % currMaxSize
                print "new MaxSize is %s" % setMaxStr
                AdminControl.setAttributes(tp, [['maximumSize', setMaxStr], ['minimumSize', setMaxStr]])
except:
    print "WAS process is not running, only update the config file"
    
server1 = AdminConfig.getid('/Cell:CloudBurstCell_1/Node:CloudBurstNode_1/Server:server1/')
tpList = AdminConfig.list('ThreadPool', server1)
for pool in tpList.split("\n"):
    pool = pool.rstrip()
    if (getName(pool) == "WebContainer"):
        attrs = [["maximumSize", setMaxInt], ["minimumSize", setMaxInt]]
        AdminConfig.modify(pool, attrs)
        AdminConfig.save()
        break

cbscript.json

This file contains the following parameter information:

[
   {
      "command": "\/bin\/sh \/tmp\/WASHV_MemoryBased\/enable_as.sh",
      "description": "A WASHV script example for scaling up based Memory",
      "execmode": 0,
      "filename": "WASHV_MemoryBased.zip",
      "keys": [
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "1024",
            "scriptkey": "MIN_MEMORY",
            "scriptvalue": "1024"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "4096",
            "scriptkey": "MAX_MEMORY",
            "scriptvalue": "4096"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "80",
            "scriptkey": "SCALE_UP_MEMORY_THRESHOLD",
            "scriptvalue": "80"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "0",
            "scriptkey": "SCALE_DOWN_MEMORY_THRESHOLD",
            "scriptvalue": "0"
         },
         {
            "locked": false,
            "required": true,
            "scriptdefaultvalue": "120",
            "scriptkey": "TRIGGER_TIME",
            "scriptvalue": "120"
         }
      ],
      "location": "\/tmp\/WASHV_MemoryBased",
      "log": "\/tmp\/WASHV_MemoryBased",
      "name": "WAS_VerticalScalingMemoryScriptPkg",
      "ostype": "linux\/unix",
      "timeout": 0,
      "type": "APPLICATION"
   }
]

The following environment variables are defined:

MIN_MEMORY

This is a required variable that indicates the minimum amount of memory that the virtual machine must have. The default value is 1024.When the amount of memory reaches this minimum number, the scale down operation cannot be performed.

MAX_MEMORY

This is a required variable that indicates the maximum amount of memory to which the virtual machine can be scaled. The default value is 4096.

SCALE_UP_MEMORY_THRESHOLD

This is a required variable that indicates the percent threshold of memory usage that, when exceeded, triggers a scale up operation to be automatically performed on the virtual machine. This threshold must be exceeded for the duration of time indicated by the TRIGGER_TIME variable before scaling is performed. The default value is 80 percent.

SCALE_DOWN_MEMORY_THRESHOLD

This is a required variable that indicates the percent threshold of memory usage that triggers a scale down operation to be automatically performed on the virtual machine. The percentage of memory usage must be below this threshold for the duration of time indicated by the TRIGGER_TIME variable before scaling is performed. The default value is 0 percent, indicating no scale down is performed. You should set this to an appropriate threshold level for your environment.

TRIGGER_TIME

This is a required variable that indicates the time, in seconds, that a threshold must be exceeded before a scaling operation is performed. The default value is 120 seconds. For example, if the memory usage on a virtual machine exceeds the SCALE_UP_MEMORY_THRESHOLD value (80%) for 120 seconds (2 minutes), then a scale up operation is triggered to increase the amount of memory allocated to the virtual machine.

extendedattributes.json

This file contains the following parameter information:

[
   {
      "savevars": "0"
   }
]

The savevars parameter controls whether environment variables are saved after post-deployment executions. A value of 0 indicates that variables are not saved.

enable_as.sh

This is the script that is called when the script package is run.

resource.py

This is the script that is run automatically to perform the middleware tuning to take advantage of the change in memory resource. In this example, the script calculates the adjustment needed to the heap size, then calls the scripts/stopServer.py script to stop the WebSphere Application Server, then issues the command to start the server again. The WebSphere process must be restarted for the new JVM heap configuration to take effect.

The script contains the following code:

import os
import sys
import json
import re
import shutil
import commands
import logging
import subprocess

def calculate_64bit_jvmmemory(matchobj):
    newHeapSize = int(matchobj.group(1)) + int(gap/1.5/128)*128
    if newHeapSize > 6144:
        return '6144'
    else :
        return str(newHeapSize)
        
def calculate_32bit_jvmmemory(matchobj):
    newHeapSize = int(matchobj.group(1)) + int(gap/1.5/128)*128
    if newHeapSize > 2048:
        return '2048'
    else :
        return str(newHeapSize)

def jvm_memory_str(matchobj):
    newHeapSize = int(int(parms['newMemory'])/1.5/128)*128
    if newHeapSize > 2048:
        newHeapSize = 2048
    strval = str(matchobj.group(1)) + ' initialHeapSize="128" maximumHeapSize="'+str(newHeapSize)+'"'
    return str(strval)
    
def startServer():
    #os.system('/opt/IBM/WebSphere/AppServer/bin/startServer.sh server1')
    subprocess.call(['sh', '/opt/IBM/WebSphere/AppServer/bin/startServer.sh', 'server1'])
    logger.debug("start Server...")
    
def stopServer():
    #os.system('/opt/IBM/WebSphere/AppServer/bin/stopServer.sh server1')
    command = '/opt/IBM/WebSphere/AppServer/bin/wsadmin.sh -conntype SOAP -lang jython -f /tmp/WASHV_MemoryBased/scripts/stopServer.py'
    subprocess.call(command, shell=True)
    logger.debug("stop Server...")


logger = logging.getLogger("resource.py")
parms = json.loads(sys.argv[1])

if int(parms['newMemory']) > int(parms['oldMemory']):
    gap = int(parms['newMemory']) - int(parms['oldMemory'])
    #originalFile = '/opt/IBM/WebSphere/AppServer/profiles/AppSrv01/config/cells/localhostNode01Cell/nodes/localhostNode01/servers/server1/server.xml'
    originalFile = '/opt/IBM/WebSphere/Profiles/DefaultAppSrv01/config/cells/CloudBurstCell_1/nodes/CloudBurstNode_1/servers/server1/server.xml'
    backupFile = originalFile + '.bk'
    output = commands.getoutput('/opt/IBM/WebSphere/AppServer/bin/versionInfo.sh')
    #print output
    if os.path.exists(originalFile):
        if not os.path.exists(backupFile):
            shutil.copy(originalFile, backupFile)
        
        with open(originalFile) as cfgFile :
            data = cfgFile.read()  
              
        m = re.search('32 bit',output)
        n = re.search('64 bit',output)
        if m :
            print 'IBM 32-bit SDK for Java'
            t = re.search('maximumHeapSize=',data)
            if t :
                if re.search('maximumHeapSize="2048"',data):
                    print 'maximumHeapSize="2048" There is no extra memory size to scaling up'
                    logger.debug("There is no extra memory size to scaling up")
                    new_data = data
                else :
                    new_data=re.sub(r'(?<=maximumHeapSize=")(\d+)', calculate_32bit_jvmmemory, data)
            else :
                new_data=re.sub(r'(?<=verboseModeJNI=")(\S+)', jvm_memory_str, data)
        elif n :
            print 'IBM 64-bit SDK for Java'
            new_data=re.sub(r'(?<=maximumHeapSize=")(\d+)', calculate_64bit_jvmmemory, data)
        else :
            print 'Nothing changed...'
            logger.debug("There is no related information ")
        if cmp(data,new_data) != 0 :
            with open(originalFile, "w") as cfgFile :
                cfgFile.write(new_data)
            Info = commands.getoutput('ps -ef | grep java')
            res = re.search('server1',Info)
            if res :
                print 'stopServer ....'
                stopServer()
                print 'startServer ....'
                startServer()
            else :
                print 'startServer ....'
                startServer()
                
    else :
        logger.debug("There is no configuration file to update ")

stopServer.py

This script is located in the scripts folder, and is called by resource.py to stop the WebSphere Application Server instance.

The script contains the following code:

AdminControl.stopServer("server1", "CloudBurstNode_1")