Details and configuration for application servers as proxy resources

Common changes for all SAP application server types

The *SAPSRV add-on policy and the description in this topic contain sample resources and groups with names containing a placeholder of <SID>. You need to replace it in all resources and group names with your own SAP system ID of the system for which you are creating the new policy.

Make sure you also replace all occurrences of the placeholder <SID> inside the resource definitions (for example, in the policies APPLICATION INFO, STARTUP, SHUTDOWN, and USS CONTROL).

For example, SAP<SID>RAS_X should be changed to SAPHA1RAS_X if HA1 is your SAP system ID.

If you want to automate more than the two remote application servers that come with the *SAPSRV add-on policy, then you need to generate new sources by copying from existing resources and groups observing the naming conventions. The SAP HA wizard documentation describes a procedure that uses the INGEBIMP sample job of SA z/OS to add remote application server resources to your policy. See the file SAPHAwizard.pdf, which is shipped inside the ING_sap.tar file in the SA z/OS z/OS UNIX directory /usr/lpp/ing/SAP. If you want to automate less than the two remote application servers that come with the *SAPSRV add-on policy, then you need to remove the ones that are not needed from your policy.

ABAP application servers

The application SAP<SID>RA0 is a z/OS UNIX proxy resource, which is used to automate a primary ABAP application server running on a remote system (Linux®, AIX®, and Windows).

Additional ABAP application servers are modeled as SA z/OS resources SAP<SID>RA1, SAP<SID>RA2, and so on. The customization steps for SAP<SID>RA0 described in this section need to be executed for all ABAP application server resources.

Because the application server runs on a remote system, it cannot be directly started, stopped, or monitored by SA z/OS using native z/OS commands. The start, stop, and check scripts (see Scripts for the remote application server policy), issue the actual SAP instance start or stop commands via SSH remote command execution. An application server status indication is provided by the monitor routine of the check script.

Java application servers

The application SAP<SID>RJ0 is a z/OS UNIX proxy resource, which models a primary Java application server running on a remote system (Linux, AIX, and Windows).

Additional Java application servers are modeled as SA z/OS resources SAP<SID>RJ1, SAP<SID>RJ2, and so on. The customization steps that are described in this section need to be executed for all Java application server resources.

Similar to ABAP application servers, a Java application server instance cannot be directly started, stopped, or monitored by SA z/OS using native z/OS commands. The start, stop, and check scripts (see Scripts for the remote application server policy) issue the actual SAP instance start or stop commands via SSH remote command execution. An application server status indication is provided by the monitor routine of the check script.

In RELATIONSHIPS, change the HasParent relation to the ABAP resource SAP<SID>RA<n> to OMPROUTE as the supporting resource. For example, OMPROUTE/APL/=.

Policy configuration for ABAP and Java application servers

The STARTUP, SHUTDOWN, and POSTSTART sections and the z/OS UNIX Control specification in each remote application server resource definition need to be adapted to the specific SAP application server instance of your SAP system. The SAP HA wizard makes these adaptions, if it has access to the SAP profiles for these SAP instances.

If you need to adapt the resource definition manually, you should observe that most of the parameters in the command lines of STARTUP, SHUTDOWN, and POSTSTART are supplied via System Automation SYMBOLs - so you need to modify the SYMBOL values. The sample screen capture from Figure 1 shows the SYMBOL definitions for the SAP<SID>RA0 resource as it is delivered with the *SAPSRV add-on policy.

General parameters

General parameters apply to all remote application server resources of a given SAP system. The SYMBOL definitions (SYMBOL1 and SYMBOL2) for these parameters are contained in the class C_SAP_<SID>RAS for ABAP application server instances and C_SAP_<SID>RJS for Java application server instances. The SYMBOL definitions for these two symbols are inherited by each application server resource. Figure 1 shows these in bold font. When adapting these SYMBOLS, you should do this in the CLASS definition. In the definition for SYMBOL1 you need to replace <SID> with the 3-character SAP system ID of your system.

In most cases you can leave SYMBOL2 as it is. If you make changes in SYMBOL2, then consider the following:

• The string SSH must not be changed, since it is the only supported communication mechanism.
• For ABAP application server resources, the value 12 in SYMBOL2 stands for the number of retries (with a 10-second wait time between retries), that the startup script start_as waits for successful startup of the application server. If it takes longer than 12x10 = 120 seconds to start the ABAP application server in your environment, you should increase this counter. For a Java application server resource, the default number of retries in SYMBOL2 is 24.

Application server-specific parameters

The symbols SYMBOLS3 through SYMBOL6 that are shown in italic font in Figure 1 are specific to an application server instance.

• SYMBOL3 must contain the name of the SAP instance, for example, DVEBMGS12
• SYMBOL4 must contain the hostname of the SAP application server that is used to start the instance.
• In the two job names - SYMBOL5 and SYMBOL6 - you only need to substitute the substring <SID> with the 3-character SAP system ID of your system.

                             Application Symbols

Entry Type : Application           PolicyDB Name   : SAP35TMP
Entry Name : SAPSIDRA0             Enterprise Name : SAP35TMP

SYMBOL1  . . . . . SID
 Description 1 . . 3-character SAP System id for remote SAP app servers
SYMBOL2  . . . . . 12 SSH
 Description 2 . . number of retries and method for starting remote SAP ABAP AS
SYMBOL3  . . . . . RAS_A00_INST
 Description 3 . . Instance name for SAP remote app server instance
SYMBOL4  . . . . . RAS_A00_HOST
 Description 4 . . Host name of SAP remote app server
SYMBOL5  . . . . . SIDA0STA
 Description 5 . . Job name for commands during app server startup
SYMBOL6  . . . . . SIDCAR0C
 Description 6 . . Job name to be used for the prestart copy commands
SYMBOL7  . . . . .
 Description 7 . .
SYMBOL8  . . . . .
 Description 8 . .
SYMBOL9  . . . . .
 Description 9 . .    

For both ABAP and Java remote application server resources, two STOP commands are defined in the policy:

• The SHUTNORM command, which kills only the monitor routine. When the monitor routine has shut down, the remote application server appears to be down for SA z/OS. After a move of the resource to a different LPAR, the new monitor routine reconnects to the application server, which is still running. If you want to stop an LPAR and move all applications to another one, the SHUTNORM command is sufficient.
• The SHUTFORCE command, which sends a stop command to the SAP application server.

An option is available starting with SA z/OS 3.5 APAR level OA48922, which eliminates for SA z/OS operators the explicit usage of the SHUTFORCE command in order to really stop the application server itself. If the proxy resource definition uses the new REXX script SAPRASTP, then this script decides automatically when to use SHUTNORM or SHUTFORCE. The decision is based on the stop votes existing in SA z/OS for the proxy resource:

• If a stop vote with desired state UNAVAILABLE for the move group exists, then the remote Application Server is stopped.
• Otherwise, only the proxy resource is stopped and automatically restarted on a different LPAR. This suggests that the stop was for an LPAR maintenance scenario.

In order to exploit this option, the SHUTNORM policy must be changed. Replace as follows:

Cmd Ps   AutoFn/*   Command Text
Pass 1:  INGUSS     /bin/kill -2 &SUBSPID ===> with SAPRASTP 2
Pass 4:  INGUSS     /bin/kill -9 &SUBSPID ===> with SAPRASTP 9

The REXX program SAPRASTP is delivered starting with SA z/OS 3.5 APAR level OA48922 and higher as part of the ING_sap.tar file in the z/OS UNIX directory /usr/lpp/ing/SAP. Add the SAPRASTP REXX program to a user-defined data set within the NetView command list, for example USER.SINGNREX. These data sets are specified by the DSICLD DD statement of NetView.

Apply the changes as described hereafter, if these conditions are met:

• The SA z/OS policy includes automated remote application servers.
• The SA z/OS policy had been created before APAR level OA48922 was applied.
• You plan to automate additional remote application servers after applying and activating APAR level OA48922.
• You use the SAP HA wizard to automate another remote application server.
• You are familiar with the terms that are used to explain what to do. See the SAP HA wizard documentation for details.

The template policy that you create for the HA wizard execution includes a modified application class C_SAP_<SID>RAS where the previously mentioned changes to the SHUTNORM and SHUTFORCE commands are applied. The output policy of the HA wizard execution includes resources for the new application server and a modified application class C_SAP_<SID>RAS. Check and apply:

• If you did not apply any local adaptations to class C_SAP_<SID>RAS: On the import policy step, select to import both the class and the remote application server resources. This overwrites the local class definition with the new SHUTNORM and SHUTFORCE commands.
• If you applied local adaptations to class C_SAP_<SID>RAS: To preserve your adaptations, you must not import class C_SAP_<SID>RAS. In this case, you manually add the new SHUTNORM and SHUTFORCE commands in class C_SAP_<SID>RAS in the active policy.

In both cases you should remove the old SHUTNORM and SHUTFORCE commands from the APL resources of the old automated application servers. Also, let the APL resource takeover the new commands from the class C_SAP_<SID>RAS. Do not forget to build and activate the new policy.

Modeling Dual Usage Type SAP systems with the *SAPSRV policy and recommended adaptions

Since SAP NetWeaver 7.5 there is no SAP system supported anymore, which includes both, an ABAP stack and a Java stack within the same SAP System ID (SID). Such an SAP system was called a dual-stack SAP system. Its functionality is still needed for SAP Solution Manager, SAP Process Integration and SAP Business Warehouse Solutions. Customers must either install two new SAP Systems in dual usage type mode or perform a dual-stack split operation for an existing SID. In both cases, the new dual usage type setup consists of one SAP system (SID) for the ABAP stack and another SAP system for the Java stack, like HPA and HPJ (Process Integration ABAP and Process Integration Java) for example.

Both SAP systems of a dual usage type setup have a close relationship, for example, the User Management Engine in the Java system may use the ABAP system for persistence. Therefore, we recommend modeling this dependency by adding a MakeAvailable/WhenAvailable relationship between the topmost Java Application Server group and the topmost ABAP Application Server group. A sample is shown below.

We installed a dual usage type PI as reference system. The ABAP system has the SID "HPA" and the Java system "HPJ". The following refers to these two SIDs modeled via the *SAPSRV policy as ABAP-only and Java-only systems.

The Db2 Data Sharing group for the ABAP system HPA is modeled with resource HPA_X/APG. It has two members, HPA1_X and HPA2_X. The Db2 Data Sharing group for the Java system HPJ is modeled with resource HPJ_X/APG. It also has two members, HPJ1_X and HPJ2_X. The System Automation policy for both SIDs was created by using the HA Wizard.

We recommend setting the Satisfactory Target (SATTGT below) to one less (-1) than the number of all members. This allows to start the ABAP Application Servers as soon as one HPA Data Sharing member is up. On the negative side, this might result in having all Application Servers connect to this one member, if they are started together with the Data Sharing Group.

Resource      : HPA_X/APG              
Description   : HPA Data Sharing Group with 2 members, HPA1_X and HPA2_X

Group Details...                                                 
   Nature                                : SERVER                
   Model                                 : 1                     
   Prepare Move                          : YES                   
   Force prepare move on rolling recycle : NO                    
   Status Determination                  : CSonly                
   Members...                                                    
      HPA1_X/APG                HPA Subsystem - HPA1 HA solution 
         PREF = 700                                              
         MPOS = 0                                                
         MPOSOVR = 0                                             
         MSEL = 0                                                
      HPA2_X/APG                HPA Subsystem - HPA2 HA solution 
         PREF = 700                                              
         MPOS = 0                                               
         MPOSOVR = 0                                             
         MSEL = 0                                                
   Policy                                :                       
      PASSIVE = NO                                               
      AVTGT  = All                                               
      SATTGT = -1             

As already mentioned, the Satisfactory Target (SATTGT) is set to -1, one less than the number of all members. In a 2-way data sharing setup this means that the group is available if one of the two members is available. In that case System Automation triggers the start of all ABAP Application Servers.

If the ABAP Application Servers are available, System Automation starts the Application Server of the Java System, modeled by SAPHPJRAS_X. The reason is that this group has a MakeAvailable/WhenAvailable relationship to the SAPHPARAS_X, the ABAP Application Server group. Additionally, note that the Java Data Sharing group must be available to allow the Java Application Servers start.

You might think about using a HasParent relationship to also model the stop sequence between SAPHPJRAS_X and SAPHPARAS_X. Then, the Java Application Servers would be stopped before the ABAP Application Servers. We do not recommend this because the Java Application Servers are still working without the ABAP Application Servers, only new logins would not be possible during the downtime of the ABAP Application Servers. Finally, it is your decision what to define and prefer.

Below is the example from our reference setup. You see all relationships of the top-level ABAP Application Servers group SAPHPARAS_X. As explained, it shows a backwards relationship MakeAvailable/WhenAvailable from the top-level Java Application Server group SAPHPJRAS_X.

Resource      : SAPHPARAS_X/APG                        
 Description   : SAP HPA remote application server group
                                                                 
 Cmd Name        Type System   Dir Relationship                  
 --- ----------- ---- -------- --- ------------------------------
     HPA_X       APG            F  MakeAvailable/WhenAvailable   
     NFS_SERV_X  APG            F  HasParent                     
     SAPHPA_X    APG            B  HasMember                     
     SAPHPARM0_X APG            F  HasMember                     
     SAPHPARM1_X APG            F  HasMember                     
     SAPHPJRAS_X APG            B  MakeAvailable/WhenAvailable   

SAP application server groups

This section explains the resource group definitions. For additional information, also refer to Figure 1.

The remote application server resources from the *SAPSRV add-on policy include the following SA z/OS application groups for a single SAP system. These groups allow you to easily monitor and manage the remote application server resources via SA z/OS.

SAP<SID>RAS_X

This SERVER group contains all MOVE groups SAP<SID>n_X as members.

With the help of a SERVER group, it is possible to bring down SAP application servers for maintenance without SA z/OS triggering any action. The server group will only show a degraded group status.

You might consider setting the satisfactory target (STGT) of the SERVER group to N-1 for your N remote application servers. In this case the group status will be OK, even if one application server is not available.

The default for both the availability target and the satisfactory target is *ALL.

The *SAPSRV add-on policy contains an explicit MakeAvailable/WhenAvailable relationship between the remote SAP application server and Db2®. This means, an SAP application server can start as soon as one Db2 member is up. However, there is no check whether the definitions for Db2 connection failover (in db2dsdriver.cfg and config.xml configuration files) are consistent with these SA z/OS definitions. This must be ensured manually.

We do not recommend changing the MakeAvailable/WhenAvailable relationship to a HasParent, as this would take away the possibility to keep the application servers up and running during a Db2 maintenance.

The SAP<SID>RAS_X group has an HasParent relationship to the NFS server sysplex group NFS_SERV_X/APG because the application server executable files reside on the NFS. If the NFS server is moved, the application servers do not need not be stopped since such a move is transparent to the NFS clients. Before the NFS server is stopped completely on any system, the application servers must be stopped. Before the application servers can be started, the NFS server on z/OS must be running.

SAP<SID>RMn_X

For an ABAP or a Java application server, each of these MOVE groups (n=0,1,2) is defined to include resources for one remote SAP application server. The recommendation is to define SAP<SID>RA0 for the primary SAP ABAP application server resource and SAP<SID>RA1, SAP<SID>RA2, …, and so on, and, for additional SAP ABAP application servers of the SAP system <SID>. For SAP Java application servers it is SAP<SID>RJ0, SAP<SID>RJ1, … and so on.

With the definition of a MOVE group, each remote application server can be monitored via SA z/OS from one LPAR. If this LPAR is stopped, the monitoring of the remote application server is moved to another LPAR in the sysplex via the MOVE group. You need a resource definition for each LPAR that should host the monitoring task. See Figure 1 where active SA applications are represented as White boxes. When an LPAR is stopped, the application servers themselves are not stopped. Refer to Operating an SAP system under System Automation control for maintenance scenarios.