IBM Support

Configuration Instructions for Electronic Customer Support (ECS) and Electronic Service Agent (ESA) for IBM i

Troubleshooting


Problem

Provides a list of instructions for activating ECS & ESA in one document.

Diagnosing The Problem

The following document outlines creating the service configuration needed for all service and support applications for IBM i OS releases.


Electronic Customer Support (ECS) is used for such things as ordering fixes via the Send PTF Order (SNDPTFORD) command, manually reporting Hardware problems in the system problem log (WRKPRB) or by using the Send Service Request (SNDSRVRQS) command.

Electronic Service Agent (ESA) provides an automatic problem reporting function, commonly referred to as Call Home. ESA also collects and sends supplemental system service information, also known as Inventory data, to IBM. The inventory data is made available to IBM support centers to aid in problem resolution. It is also responsible for manage Update Access Key (UAK) for OS Firmware manage.

Effective 30 June 2020, IBM has withdrawn Performance Management for Power® Systems (PM Agent) services from marketing. Support has been withdrawn 30 September 2020.
Additional information and steps to stop sending data are included in Disabling PM Agent document.

NOTE: The ability to report a Software problem entry was disbanded due to changes introduced in the Support Platform. More information in Software related problem log entries will not send to IBM document

This change DOES NOT affect the ability to report Hardware problem entry (Call Home)
If a Software problem entry is generated, need to manually open a ticket through the Support Portal, with the information related to the WRKPRB entry.
 
  • Ways to connect to IBM


    HTTP/HTTPS is the only approved method at all OS release levels.

    ECS and ESA support a multi-hop environment, however this configuration uses VPN, which is disbanded. Reference the section titled Creating a Multihop HTTP/HTTPS Connection for the Local System in the CRTSRVCFG Command document for instructions on using the HTTP/HTTPS connection in a multi-hop environment to bypass the restrictions.

    At V7R1 and earlier releases, in the background, ECS is hardcoded to initially attempt an HTTP/HTTPS connection. If that fails, a VPN connection is tried. If that fails, a modem connection is tried, whether a modem physically exists on the system or not.
    At V7R2 and beyond, only an HTTP/HTTPS connection is attempted.

    Note: Modem and VPN connection are no longer supported.

Resolving The Problem

Click over each title tab to display the information:

Following is a summary of the firewall requirements for ECS and ESA:

For HTTP/HTTPS communication, ports 80 and 443 need to be open for both outbound and inbound (return) TCP traffic. 

If the ports must be restricted to communication with specific IP addresses, reference to the Electronic Service Agent (ESA) and Electronic Customer Support (ECS) VPN and HTTP Firewall Settings document for a listing of unique IP addresses used in the communication with IBM. Whenever possible, use host names instead IP address.

There are several IBM IP addresses involved for an HTTP/HTTPS connection. It is necessary that all IP addresses be allowed through the firewall otherwise unpredictable results will occur.

Starting V7R3, ECS/ESA use the new EDGE server which requires less and different IP address to be open on Firewall for ports 80/443, than before.
By default, V7R2 uses LEGACY servers to connect for ECS/ESA. By modifying a configuration file, and specific PTFs, it's possible to use the new EDGE server. Be aware that Firewall rules changes are required. Information on the Firewall document.

Considerations:

  • Typically, if a firewall port is opened for outbound communication, return traffic is automatically allowed on that port.
  • For 7.3, having PTF SI68172 on system would require ONLY port 443 to be open. Port 80 will not longer be needed. For 7.2, having PTF SI76398 on system do the same. ONLY VALID WHEN USING EDGE CONFIGURATION.
  • ESA has its own internal certificate to exchange with the IBM backend server, so any 'addition' by Proxy/Firewall during the communication will make it fail. If the environment has a Proxy/Firewall that is terminating the SSL connection and returning its own self-signed certificate, it is not supported.
  • Implement infrastructure improvements to electronic fix distribution on June 4, 2022. IP and hostnames will change for servers that support fix delivery. This affects specifically OS versions using LEGACY IP addresses, such 7.1 and 7.2 (when not using EDGE) Additional information on the Firewall document.

Ensure the following prerequisites are satisfied before creating the service configuration and testing:

1. Product options:

The following product options must be installed on the system. In GO LICPGM menu, select option 10. Display installed licensed programs and press F11 twice to see Product Option:

  • 5770SS1 Option 30 Qshell
  • 5770SS1 Option 33 Portable Application Solutions Environment (PASE)
  • 5770SS1 Option 34 Digital Certificate Manager
  • 5770JV1 Option 16 Java SE 8 32 bit
  • 5770JV1 Option 17 Java SE 8 64 bit
  • 5733SC1 *BASE - IBM Portable Utilities for i 
  • 5733SC1 Option 1 - OpenSSH, OpenSSL, zlib 
  • 5770UME IBM Universal Manageability Enablement for i 

Review the Standard and Keyed Media Set - Master List for information on where to find any missing product.

Note: ECS/ESA need JDK8 (JV1 options 16 & 17) to be on system, plus recommended PTFs to use that Java version. Information could be found in the Migrate Electronic Service Agent (ESA) from using Java 6 to Java 8 document.


Important information about 5770UME and the *CIMOM Server Associated with the Product:

Product 5770UME (IBM Universal Manageability Enablement for i), IS required for Electronic Service Agent. Specially, starting at 7.2 to start Electronic Service Agent jobs at IPL for Call Home and to collect Inventory. There are several hardware inventory types that are not collected when this product is not installed, and in some cases, will not activate. The CIMOM server associated with this product (QUMECIMOM) does need to be started for ECS/ESA functions to be successful.

It also requires a certificate assigned to CIMOM server. Additional information about CIMOM server certificate and creation steps could be found on the *CIMOM server certificate document.

It is recommended that the latest PTFs for the 5770UME product are applied on the system.

2. The Retain server security data (QRETSVRSEC) system value must be set to 1:
 
  • CHGSYSVAL SYSVAL(QRETSVRSEC) VALUE(1)
     

3. TCP configuration:
TCP needs to be configured correctly in order to avoid communication problems. In CFGTCP menu:

  • Option 1. Work with TCP/IP interfaces.
    IP address of the system has to be ACTIVE. Same for LOOPBACK interface

Use PING LOOPBACK and PING LOCALHOST to ensure it is successful. Both should return 127.0.0.1.
 

  • Option 2. Work with TCP/IP routes.
    There has to be a *DFTROUTE. The Next Hop has to be valid as well.
     
  • Option 12. Change TCP/IP domain information.

Must have a host name and domain name.
The Host Name Search Priority should be set to *LOCAL
There should be at least one IP address listed under the Domain Name Server. If there are multiple addresses, each must be unique.
 

  • Option 10. Work with TCP/IP host table entries.

IP address of the system (from option 1) has to be associated with the long name of the system (HostName.DomainName from option 12), and the short name for the system (Host Name from option 12)
Long name should be listed first, followed by short name. There should not be any other Host Table Entries that have the same short and/or long name for the system. 


4. Host Servers:
Host Servers needs to be running

Review NETSTAT Option 3 to verify Host Servers are started or restart them by doing:
ENDHOSTSVR *ALL

STRHOSTSVR *ALL
 

Are the host servers started? If started, then try the following commands:

STRQSH

export -s CLASSPATH=/QIBM/ProdData/HTTP/Public/jt400/lib/jt400.jar

java utilities.JPing localhost

This checks if toolbox whether able to connect to the local Host Servers. Something like this display if all is correct:

> java utilities.JPing localhost                              

  Verifying connections to system localhost...                
                                                              
  Successfully connected to server application:  as-file      
  Successfully connected to server application:  as-netprt    
  Successfully connected to server application:  as-rmtcmd    
  Successfully connected to server application:  as-dtaq      
  Successfully connected to server application:  as-database  
  Successfully connected to server application:  as-ddm       
  Successfully connected to server application:  as-central   
  Successfully connected to server application:  as-signon    
  Connection verified                                         
  $   

To verify or change the system contact information:

WRKCNTINF

Select option 2. Work with local service information

Select option 1. Display service contact information

If changes are needed, select option 2. Change service contact information


Verify the following:

The Company and Contact Name are correct

The Customer number: Customer identifier and Customer description and the Contract number: Contract identifier and Contract description are all be set to *NONE

The Primary contact telephone number and/or Help desk or pager number are correct

The Fax telephone number is correct, if applicable

The correct address is listed for the Mailing address, and that the City or locality, State or province, Country or region and Postal code are on the designated lines

Note: The Country or Region should be the two character abbreviation for the country or region, such as US for United States, CA for Canada, SE for Sweden.

      Two-digit country code can be found here: Country or Region Codes

A Primary email address is specified

The Media for PTFs is set to *DVDROM 

Note: If Call central site support is set to *YES, the Help desk or pager number will be used as the primary contact telephone number.

Due to the changes introduced in the New Support Platform, in order to receive notification when doing CALL HOME TEST, the Contact Information should match an existing IBMid.
Additional information is found in Call Home Contacts: Guideline and Best Practices Documentation for Clients

The instructions below are intended to delete any previous service configuration, Point-To-Point profiles, line descriptions and configuration files that may have been restored from another system and re-create it. The instructions are for a direct internet connection. Reference to the CRTSRVCFG Command document for other configuration options and additional information.

Do the following while signed on with a user profile that has *SECOFR authority:

  • DLTSRVCFG DLTCMNCFG(*YES)
  • WRKTCPPTP and delete (4=Remove) any remaining profiles that begin with the letter 'Q'
  • WRKLIND LIND(Q*) and delete (4=Delete) any remaining lines that have the type of *PPP and begin with the letter 'Q'. Also delete QESLINE and QTILINE which are type *SDLC
  • RMVLNK OBJLNK('/qibm/userdata/os400/universalconnection/*')
    Note: An error like 'Requested operation not allowed. Access problem.' might appear. Disregard.
  • CRTSRVCFG ROLE(*PRIMARY) CNNTYPE(*DIRECT) CNTRYID(*SELECT) STATE(*SELECT)

Select the appropriate country/region

Select the appropriate state/province (this is the state/province that the system is physically located)

If there is a proxy server involved, specify the appropriate proxy information

Note: If msgCPD0A35 is received after run CRTSRVCFG, check the RC related and refer to the CRTSRVCFG/CHGSRVCFG fails with msgCPD0A35: Java exception not handled for user-defined document for usual stuff that could cause the error.

The instructions are intended to clean up any old ESA scheduled job entries that may still exist or have been restored.

Do the following while signed on with a user profile that has *SECOFR authority:

  • GO SERVICE

Select option 1. Change Service Agent attributes

Specify Enable as *NO and press <ENTER>

Press F12=Cancel
 

  • WRKJOBSCDE JOB(QS*)

Select option 4=Remove next to the following entries, if they exist:

QSJERRRPT QSJHEARTBT QS9AUTOPTF QS9AUTOTST or QS9SACOL
 

  • GO SERVICE

Select option 1. Change Service Agent attributes

Specify Enable as *YES and press <ENTER>

Page down to the second page

In the Service Information section, change the Collect Time to a time when the system is least busy and press <ENTER>

  • GO SERVICE

Select option 10. Work with jobs

The following jobs should be active: QS9HDWMON, QS9PALMON, QS9PRBMON, QS9PRBSND 

Press F12=Cancel

Note: The job QS9SFWMON won't be listed if the PTF (or superseded) for the appropriate OS version is applied. Reference to the Disable automatic problem reporting of legacy Software FFDC (First Failure Data Capture) - QS9SFWMON job document for additional information.

Starting 7.5, the ability to report a Software problem entry was completely removed. Due to changes introduced in the Support Platform, the WRKPRB entries related to Software are not longer accepted when manually reported in older releases either.

Note: QS9UAK might also be shown as scheduled job. This job is only required when system is a Standalone partition over Power 8 or higher. Otherwise, it's not required. Could be disabled by doing CHGSRVAGTA REFRESHUAK(*NO). More information about UAK and ESA could be found in the Manage update access keys with IBM Electronic Service Agent (ESA) on IBM i document.

More information about scheduled jobs could be found in the Electronic Service Agent Jobs document.

  • The following command orders the latest cumulative package cover letter:

SNDPTFORD PTFID((SF98xxx *ONLYPRD *ONLYRLS))

**Where xxx is the OS release, ie. SF98710 for 7.1, SF98720 for 7.2 and SF98730 for 7.3.
 

  • To test Call Home by ESA, send a test problem using the following steps:

GO SERVICE

Select option 15. Send test problem

Leave the Error log identifier as 00000000 and press <ENTER>

Press F12=Cancel

WRKPRB to monitor the status of the Test Problem

Press F5=Refresh

The test problem should go to a SENT status

  • To test connectivity to IBM backend servers use below commands.  Test plain TCP communication:
- VFYSRVCFG SERVICE(*ECS) VFYOPT(*ALL)
Let it complete, might take a while.

- VFYSRVCFG SERVICE(*FIXREP) VFYOPT(*ALL)
Let it complete, might take a while.

- VFYSRVCFG SERVICE(*PRBRPT) VFYOPT(*ALL)
Let it complete, might take a while.

- VFYSRVCFG SERVICE(*SPCFG) VFYOPT(*ALL)
Let it complete, might take a while.

- VFYSRVCFG SERVICE(*SRVAGT) VFYOPT(*ALL)
Let it complete, might take a while.
The VFYSRVCFG commands log to joblog:  IP address, protocol and port used along with success or failure information.
CPIAC59: Verification was successful.
CPIAC60: Verification was not successful.
CPIAC61: The value does not match an existing service destination.
Recommended PTFs for ESA and 'Delete and re-create service configuration' steps need to be followed correctly.

NOTE: 
Effective 30 June 2020, IBM® had withdrawn Performance Management for Power® Systems (PM) services from marketing.
Support for this product withdrawn 30 September 2020.
Additional information and steps to stop sending PM Agent data are included in Disabling PM Agent
document.

PMAGENT/PM400 free reports, paid reports and website access withdrawn 30 September 2020: https://www.ibm.com/downloads/cas/US-ENUS920-133-CA/name/US-ENUS920-133-CA.PDF

Withdrawal and service discontinuance: IBM Performance Management for Power Systems (PM) is withdrawn as a stand-alone offering; optional Performance Management reporting is removed from other premium support service offerings.

Here are the tools available that could be an alternative for PMAgent:
  1. Performance Tools
  2. Performance Data Collectors
  3. Performance on the web - Performance tools for IBM i
  4. Resources
  5. Graph History is an option at 7.3 in Access Client Solutions.  ACS does not support Graph History at 7.1 & 7.2. Additional information can be found at Article: Graph History in IBM Navigator for i Overview
  6. iDoctor Collection Services Investigator - Historical Summaries option

Following the steps described allow to configure ECS/ESA.

If there are failures at any point in the steps, generate a job log: DSPJOBLOG JOB(*) OUTPUT(*PRINT)

Using the Support Portal , use the Search Support option to search on any error messages for resolution.

If the problem needs to be reported to IBM Support Center, collect the information by using QMGTOOLS. Document QMGTOOLS: Collect ECS/ESA data has the following steps:

  • Assure to have last version of QMGTOOLS prior to the collection: 
    GO QMGTOOLS/MG 
    Use option 13. Check IBM for updated QMGTOOLS. 
     
    If not on system, or manually action is needed, install or update it using following document (Method 3 - Manually update or install from the IBM public FTP site)
    QMGTOOLS: How to check and update QMGTOOLS

[{"Line of Business":{"code":"LOB57","label":"Power"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG60","label":"IBM i"},"ARM Category":[{"code":"a8m0z0000000CGRAA2","label":"Electronic Service Agent"}],"ARM Case Number":"","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Version(s)"}]

Document Information

Modified date:
28 November 2022

UID

nas8N1010756