Q1.1:

Where can I find documentation for my storage server?

A1.1:

For links to all Elastic Storage Server, IBM GPFS Storage Server, IBM Spectrum Scale, and IBM Spectrum Scale RAID documentation, see the IBM Knowledge Center at http://www.ibm.com/support/knowledgecenter/SSYSP8/sts_welcome.html

Note: The IBM Spectrum Scale RAID Administration Guide that is associated with each release is available on the release-specific page for the storage server.

Q1.2:

Who do I contact if I need help with my storage server?

A1.2:

Your contact point for questions that are not answered in this FAQ depends on your storage server's platform and level.

For your Elastic Storage Server, all points of contact are with IBM:

• Hardware and software issues
  • In the United States, you can call 1-800-IBM-SERV (1–800–426–7378).
  • In all other countries, contact your local IBM service center.
  • You can also access IBM Support online at https://www.ibm.com/support/home/.
• Contact scale@us.ibm.com for all other services or consultation on what service is best for your situation.

For your GPFS Storage Server:

• For GSS 2.0 and prior, your point of contact is IBM (scale@us.ibm.com).
• For GSS 2.5 and later, your point of contact is Lenovo (http://shop.lenovo.com/us/en/systems/servers/high-density/gpfs-storage/).

Q1.3:

Are there training courses available for ESS?

A1.3:

Perform the following steps to find ESS related training courses:

1. Go to www.ibm.com/training.
2. Click Find learning.
3. Enter any of the following search strings in the search bar:
   • sse11*
   • Elastic Storage Server
   • Red Hat Linux
   • Linux
   • s822l
     Note: Using any of these search strings generates Power Systems courses that are related to Linux on Power.
   • GPFS
   • Spectrum Scale
4. Scroll down to review the course offerings.
   Note: Alternatively, you can use the training paths that are given at the following link: https://www.ibm.com/services/learning/ites.wss/zz-en?pageType=journey_description&journeyId=itns_302&tag=o-itns-01-07.

   Select the Spectrum Computing option to view the available courses.

Q1.4:

Is it possible to deploy IBM Spectrum Scale RAID in other hardware models?

A1.4:

Yes, IBM Spectrum Scale Erasure Code Edition allows you to deploy IBM Spectrum Scale RAID using commodity hardware as long as that hardware meets the minimum requirements for IBM Spectrum Scale Erasure Code Edition storage servers. For more information, see the IBM Spectrum Scale Erasure Code Edition questions section of the IBM Spectrum Scale FAQ in the IBM Spectrum Scale Knowledge Center.

Q1.5:

Where can I find more information about IBM Spectrum Scale for ESS licensing requirements?

A1.5:

For more information about licensing and pricing, see https://www.ibm.com/support/knowledgecenter/STXKQY/IBMScale_ESS_Licensing.pdf.

Q2.1:

What components of an ESS solution need to be coordinated?

A2.1:

An ESS is defined as a set of software (SW) and hardware (HW) that has been tested together. The major components of an ESS solution release include the following items:

• Power hardware
• Storage hardware
• IBM Spectrum Scale software
• Red Hat Enterprise Linux (RHEL) operating system

The minor components, which must be integration-tested with the major components, include the following items:

• Mellanox network switches
• Power server firmware
• Mellanox networking switch driver firmware
• Enclosure firmware
• HBAs and related firmware

Q2.2:

How do I know what levels of code need to be installed for a particular ESS solution level?

A2.2:

For information on the levels of code to be installed for a particular ESS solution level, see ESS software deployment preparation.

After clicking on the above link, select the ESS version on the top-left hand corner of the page from the Change version drop-down list to know the details of the older version.

Q2.3:

How do I know if my ESS solution level is supported?

A2.3:

While IBM's best effort is made to service a customer at any ESS solution level, you might be required to update your system if a component of your ESS solution level is no longer supported. The tables in this section of the FAQ are intended to assist customers in determining what levels of code are supported. For more information, open a support case at https://www.ibm.com/mysupport/s/?language=en_US or call IBM Service and Support at 1-800-IBM-SERB.

Q2.4:

How do I know if a particular level of RHEL is still supported?

A2.4:

New IBM ESS solution release levels intentionally uplevel the RHEL level to stay current with RHEL support currency. If you follow the IBM recommendation to update your ESS solution software to the most current ESS software level at least once a year, then you will be current with RHEL service and support. Every RHEL release (using the Red Hat EUS (Extended Update Support) license, which is included in the ESS RHEL subscription) is fully supported for 24 months from the initial RHEL release date. The RHEL initial release dates are provided at https://access.redhat.com/articles/3078. The end-of-support dates for RHEL release levels are provided at https://access.redhat.com/support/policy/updates/errata/#Extended_Update_Support.

The following list contains the current end-of-support dates for RHEL 7 release levels under the EUS license:

• 7.1 (ends March 31, 2017)
• 7.2 (ends November 30, 2017)
• 7.3 (ends November 30, 2018)
• 7.4 (ends August 31, 2019)
• 7.5 (ends April 30, 2020)
• 7.6 (ends October 31, 2020)
• 7.7 (ends August 30, 2021)

Q2.5:

How frequently am I required to update my ESS system?

A2.5:

IBM does not mandate how frequently an ESS system must be upgraded. However, it is recommended that customers plan for software upgrade and maintenance on an annual basis at a minimum. The actual timing of this is highly dependent on the customer's environment and the ability to take outages or maintenance windows.

Q2.6:

Can ESS hardware be upgraded non-disruptively?

A2.6:

IBM supports non-disruptive additional storage enclosure MES (miscellaneous equipment specification) upgrades on the ESS GLxS and GSxS models. The ESS must be running RHEL Linux Little Endian (LE) base operating system with ESS 5.3.2 or later. The ESS must also have space available in the same rack for the additional storage enclosures.

An MES upgrade requires the use of the mmvdisk command. For more information about mmvdisk, see Q5.5 Is use of the mmvdisk command required by any ESS function?.

The permitted MES upgrade paths are described in MES upgrade paths:

Table 31. MES upgrade paths

Existing ESS building block	MES upgrade ESS building block	Enclosures added
GS1S	GS2S	One 5147-024 enclosure
GS2S	GS4S	Two 5147-024 enclosures
GL1S	GL2S	One 5147-084 enclosure
GL2S	GL3S	One 5147-084 enclosure
GL2S	GL4S	Two 5147-084 enclosures
GL3S	GL4S	One 5147-084 enclosure
GL4S	GL5S	One 5147-084 enclosure
GL4S	GL6S	Two 5147-084 enclosures
GL5S	GL6S	One 5147-084 enclosure
GL1C	GL2C	One 5147-106 enclosure
GL2C	GL3C	One 5147-106 enclosure
GL2C	GL4C	Two 5147-106 enclosures
GL3C	GL4C	One 5147-106 enclosure
GL4C	GL5C	One 5147-106 enclosure
GL4C	GL6C	Two 5147-106 enclosures
GL5C	GL6C	One 5147-106 enclosure

The new enclosures must be the same model as the existing enclosures and contain the same type and size disks. Multiple MES upgrades must be performed in increments. For example, it is not possible to go directly from a GS1S to a GS4S; the intermediary MES upgrade of the GS1S to GS2S must be completed before the GS2S to GS4S MES upgrade can be performed.

Any ESS building blocks that are not listed in MES upgrade paths can be upgraded disruptively only; that is, by removing all data from the building block, adding new enclosures, recreating the building block, and restoring the data.

Q2.7:

Can ESS software be upgraded non-disruptively?

A2.7:

For information on the levels of code to be installed for a particular ESS solution level, see Upgrade guidance section under ESS Quick Deployment Guide.

After clicking on the above link, select the ESS version on the top-left hand corner of the page from the Change version drop-down list to know the details of the older version.

Q2.8:

Who can I contact for assistance with upgrading my ESS system?

A2.8:

There are two options for obtaining services to upgrade your ESS system:

1. IBM Systems Lab Services offers services for upgrading an ESS system. For details on available services, clients should contact their IBM representative or go to the Lab Services website at https://www.ibm.com/it-infrastructure/services/lab-services.
2. Select IBM Business Partners provide services to perform upgrade services for ESS systems. If you purchased your system from a Business Partner, contact them to find out if they have services available.

Q2.9

Why am I required to register the Red Hat Enterprise Linux subscription keys for the 1st generation and 2nd generation of Elastic Storage Server (ESS)?

A2.9:

The 1st generation and 2nd generation of IBM ESS use IBM Power servers running Red Hat Enterprise Linux (RHEL). For these ESS models, Red Hat requires a Red Hat Level 3 support subscription for Red Hat Enterprise Linux, and a subscription for Red Hat Linux Extended Update Support (EUS). These Red Hat subscriptions ensure proper Red Hat entitlement and subscription support is in place for use of RHEL on the Power servers in IBM Elastic Storage Server, and entitles ESS customers to download RHEL erratas or fixes as needed.

Q2.10

How do I register the Red Hat Enterprise Linux subscription keys for the 1st generation and 2nd generation of Elastic Storage Server (ESS)?

A2.10:

The following procedure is the standard IBM Red Hat Enterprise Linux on Power subscription registration procedure, which is documented in the IBM announcement letters for:

• IBM program product 5639-RH7 - Red Hat Enterprise Linux for Power, Big Endian (BE)
• IBM program product 5639-RLE - Red Hat Enterprise Linux for Power, Little Endian (LE)

You can find copies of these IBM announcement letters by searching on the internet for "IBM announcement letter 5639-RH7" or "IBM announcement letter 5639-RLE".

Note: It is the customer's responsibility to receive these RHEL subscription keys and register them on the Red Hat Portal according to the following procedure. It is the customer's responsibility to maintain the RHEL subscription as current with Red Hat when the subscription is due for renewal. If you have any questions, contact your IBM representative.

1. The customer whose name and address matches the contact information on the IBM ESS purchase order receives, from IBM Software Distribution via a standard mail letter, the printed RHEL official paper certificate with subscription keys for both RHEL and EUS.
   Note:

   1. The IBM purchase order can only have one contact name. Therefore, the customer contact name on the purchase order is typically someone in the customer purchasing or procurement department (often located remotely from the data center). Be aware that the IBM purchase order contact name is almost certainly not the ESS storage administrator or the customer's Red Hat Linux license administrator.
   2. The RHEL and EUS subscription keys are not shipped in the physical hardware shipment of the ESS. They are printed paper subscription key certificates that are mailed to the customer contact name on the IBM purchase order.
2. The customer purchase order contact and the customer Red Hat Enterprise Linux license administrator should be contacted to give the printed RHEL official subscription keys for both RHEL and EUS to the Red Hat Linux license administrator for each ESS installation site.
3. The Red Hat Linux license administrator for each site needs to register the RHEL and EUS keys on the Red Hat Portal in the same manner that they would do for any RHEL server.
4. After these three steps are done, the customer is entitled to download RHEL kernels and erratas that IBM specifies are needed for any particular ESS fix pack from the Red Hat Portal.
5. If you have multiple Red Hat Linux license administrators and multiple sites, distribute the appropriate amount of RHEL and EUS subscription keys that are needed for each ESS site.

Contact your IBM representative with any questions about this procedure.

Note: If the original RHEL subscription keys are unavailable or cannot be found, it is possible for them to be retrieved and resent via email. Contact your IBM representative about this request. The original IBM software order number is needed. Using that information, IBM will request the original RHEL subscription keys from IBM Power Linux Operations on your behalf.

Q2.11

How do I receive RHEL erratas or fixes that are required to maintain the 1st generation and 2nd generation of Elastic Storage Server (ESS)?

A2.11:

After you register your RHEL or EUS subscription keys on the Red Hat Portal, you are entitled to download RHEL erratas and fixes.

Q2.12

What are the supported drives and available capacity upgrade paths for the different ESS models?

A2.12:

For more information, see Supported ESS models.

Table 32. Supported ESS models

ESS Model GSxS
Storage expansion enclosure	5147-024: (2U24)
Number of enclosures supported	x = 1, 2, or 4
Supported drives	3.84 TB or 15 TB 2.5” SSDs
Number of drives per enclosure	24
Capacity upgrade paths supported	GS1S=>GS2S=>GS4S
ESS Model GLxS
Storage expansion enclosure	5147-084: (5U84)
Number of enclosures supported	x = 1, 2, 3, 4, 5, or 6
Supported drives	4, 8, 10 or 14 TB NL-SAS 3.5” HDDs
Number of drives per enclosure	2-SSDs & 82-HDDs in first enclosure 84-HDDs in all other enclosures per model
Capacity upgrade paths supported	GL1S=>GL2S=>GL3S=>GL4S=>GL5S=>GL6S
ESS Model GLxC
Storage expansion enclosure	5147-106: (4U106)
Number of enclosures supported	x = 1, 2, 3, 4, 5, 6, or 8
Supported drives	10 or 14 TB NL-SAS 3.5” HDDs
Number of drives per enclosure	2-SSDs & 104-HDDs in first enclosure 106-HDDs in all other enclosures per model
Capacity upgrade paths supported	GL1C=>GL2C=>GL3C=>GL4C=>GL5C=>GL6C
ESS Model GHxy
Storage expansion enclosure	5147-024: (2U24) and 5147-084: (5U84)
Number of enclosures supported	x (2U24) = 1 or 2 y (5U84) = 2 or 4
Models supported	GH12, GH22, GH14, GH24
Supported drives	3.84 TB or 15 TB 2.5” SSDs (in 5147-024) 4, 8, 10 or 14 TB NL-SAS 3.5” HDDs  (in  5147-084)
Capacity upgrade paths supported

Note: There are considerations which should be made if you are planning on shutting down an SSD based system for an extended period. The JEDEC spec for Enterprise SSD drives requires that the drives retain data for a minimum of 3 months at 40C. This means that after 3 months of a system being powered off in an environment that is at 40C or less, there is a potential of data loss and/or drive failures. This power off time limitation is due to the physical characteristics of Flash SSD media's gradual loss of electrical charge over an extended power down period. There is a potential for data loss and/or flash cell characteristic shift, leading to drive failure.

IBM recommends the following:

• Always perform regular backups and make sure you have recently backed your system up prior to extended shutdowns.
• A system (and its enclosed drives) should be powered up at least 2 weeks after 2 months of system power off. If a drive has an error indicating it is at end of life we recommend not powering off the system for extended periods of time.
• Proper environmental control procedures should still be in place to ensure systems are experiencing less than 40C at all times, even if the systems are powered down.
• If a system is being retired and used for another activity in the future, the drives should be cleared with a format. This will help prevent SSD failure upon power up, even if the data is not needed. This can be performed with the command "chdrive -task format <drive-id>" to any drive once in a candidate state.

Note: If you are planning on shutting down an SSD based system for an extended period, see Potential for SSD data loss after extended shutdown.

Note: SSD data loss can occur after an extended shutdown. Our SSDs are designed and warranted to retain data as per the JEDEC specification. The JEDEC specification requires that the drives retain data for a minimum of three months at 40°C. Within drive specifications, data retention is even better at room temperature. Therefore, if an SSD system is taken offline, the best and most reasonable option is to keep the system at a temperature of 15°C-30°C.

The following considerations are recommended by IBM when shutting down an SSD based system for an extended period:

• Perform regular backups and ensure that the system is backed up prior to the extended shutdowns.
• If a system (and its enclosed drives) is powered off for two months, it should be powered on to allow the disk scrubbing process to complete a run. The completion of the scrub is indicated with the following message in the mmfs log (/var/adm/ras/) for each vdisk in each declustered array:

  [I] End scrubbing tracks of md_DA1_LG2

  Therefore, ensuring that all the vdisks are scrubbed.

  If a drive has an error that indicates that the drive is at the end of life, it is recommended that you do not power off the system for extended periods of time without first performing the needed maintenance.
• Even when the systems are powered down, proper environmental control procedures should still be in place to ensure that the systems are kept at less than 40°C at all times.
• If a system is to be used for another activity in the future, before shutting it down the drives should be cleared with the Sanitize with Block Erase command (All SSD drives have a way to do it). This will help prevent SSD failure upon power up, even if the data is not needed.

Q2.13

Can I install additional software on ESS IO/EMS nodes?

A2.13:

Installing and using additional software, which is not part of the ESS package, is not supported on ESS IO/EMS nodes unless ESS support requests that you do so.

Q2.14

Can I use the ESS IO server/EMS node to provide other services like Cluster Export Services (CES), protocol nodes, AFM gateway nodes, transparent cloud tiering gateway nodes, or the IBM Spectrum Protect Client?

A2.14:

The ESS IO server/EMS can only be used to provide GNR functionality. The ESS IO server/EMS cannot be used to provide any other functionality such as IBM Spectrum Scale Cluster Export Services (CES) protocol nodes, AFM gateway nodes, transparent cloud tiering gateway nodes, or the IBM Spectrum Protect Client.

Q2.15

Can I mix GSS/DSS building blocks and ESS building blocks in the same IBM Spectrum Scale cluster?

A2.15:

No, you cannot have GSS/DSS building blocks and ESS building blocks in the same IBM Spectrum Scale cluster. This is not a supported configuration. You can use the remote mount feature of IBM Spectrum Scale to share file system data across clusters.

Q2.16

How do I know what levels of code need to be installed for a particular ESS 3000, ESS 5000 and ESS 3500 solution level?

A2.16:

For information on the levels of code to be installed for a particular ESS 3000, ESS 5000 and ESS 3500 solution level, see ESS software deployment preparation.

After clicking on the above link, select the ESS version on the top-left hand corner of the page from the Change version drop-down list to know the details of the older version.

Q2.17

What are the prerequisites when using an existing Power8 EMS for deploying ESS 3000?

A2.17

If you are using an existing Power8 EMS server for deploying a new ESS 3000, before you start ESS 3000 container installation, you must be at ESS 5.3.5 level minimum on that EMS. This is because you are required to be at minimum RHEL 7.7 and Python 3 levels.

Q3.1:

When using protocol access methods integrated with IBM Spectrum Scale like CES (cluster export services), Active File Management, file audit logging, or the IBM Security Key Lifecycle Manager for encryption with IBM Spectrum Scale Advanced Edition or Data Management Edition, can I run those services on the ESS hardware?

A3.1:

No, you need to run the cNFS, Protocol Node services, the AFM gateway node, file audit logging, or the ISKLM key server node on one or more other servers that are not the ESS server/EMS nodes but are running IBM Spectrum Scale at the necessary level for support.

Q3.2:

Are there any considerations when accessing my ESS via client nodes using an InfiniBand network?

A3.2:

Yes, ESS 3.0.3 and later versions provide a more graceful handling of serious hardware faults that are generally triggered by an overloaded InfiniBand adapter; therefore, they are better suited to protect client data from such hardware faults.

Q3.3:

What are the current limitations for the GPFS Storage Server V1.0, V1.5, and V2.0?

A3.3:

The GPFS Storage Servers V1.0, V1.5, and V2.0 are not supported with IBM Spectrum Scale V4.1.1 or later. GPFS Storage Servers at these levels do not support IBM Spectrum Scale V4.1.1 or later clusters attached to a GSS. In particular, the integrated protocols access methods are not supported. For more information, see the IBM Spectrum Scale security bulletin at http://www-1.ibm.com/support/docview.wss?uid=ssg1S1005476.

GSS customers who require use of IBM Spectrum Scale V4.1.1 or later should contact Lenovo at http://shop.lenovo.com/us/en/systems/servers/high-density/gpfs-storage/.

Q4.1:

What support services are available for the Elastic Storage Server?

A4.1:

Support services include:

• Customize your support portal at http://www-01.ibm.com/software/support/einfo.html. Update your notifications to include IBM Elastic Storage Server (ESS) and IBM Spectrum Scale RAID at https://www-947.ibm.com/systems/support/myview/subscription/css.wss/ .
  Note: Information in your view of My Notifications is not retroactive. To view notices that were posted before you subscribed, visit the IBM Spectrum Scale RAID/GPFS RAID Announce Forum at http://www.ibm.com/developerworks/forums/forum.jspa?forumID=1606
• For the latest announcements and news, subscribe to the IBM Spectrum Scale RAID/GPFS RAID Announce Forum at http://www.ibm.com/developerworks/forums/forum.jspa?forumID=1606 .
• Contact scale@us.ibm.com

Q4.2:

How do I maintain or rebuild the software on my storage server?

A4.2:

Your maintenance or rebuild path depends on the platform of storage server:

• For the Elastic Storage Server, fix packs that contain packages that are required to maintain or rebuild are available:
  • For ESS 3.5 and later, on Fix Central at http://www.ibm.com/support/fixcentral/swg/quickorder?parent=Software%2Bdefined%2Bstorage&product=ibm/StorageSoftware/IBM+Spectrum+Scale+RAID&release=All&platform=All&function=all&source=fc
  • For ESS 3.0 on Fix Central at http://www-933.ibm.com/support/fixcentral/swg/selectFixes?parent=Cluster%2Bsoftware&product=ibm/Other+software/GPFS+Native+RAID+for+GPFS+Storage+Server&release=All&platform=All&function=all
• For the GPFS Storage Server on x86:
  • For GSS 2.0 and prior, fix packs that contain packages that are required to maintain or rebuild are available on Fix Central athttp://www-933.ibm.com/support/fixcentral/swg/selectFixes?parent=Cluster%2Bsoftware&product=ibm/Other+software/GPFS+Native+RAID+for+GPFS+Storage+Server&release=All&platform=All&function=all
  • For GSS 2.5 and later, contact Lenovo http://shop.lenovo.com/us/en/systems/servers/high-density/gpfs-storage/

Red Hat Enterprise Linux Level and update considerations:

• User can update RHEL as needed to address security updates. It is highly recommended to limit errata updates applied to the Red Hat Enterprise Linux operating system used in the ESS solution to security errata or errata updates requested by service. For more information visit Red Hat's solution article on applying only security updates at https://access.redhat.com/solutions/10021

Note: If you have any questions regarding software/firmware updates to your Elastic Storage Server or GPFS Storage Server V2.0 and prior configuration, contact scale@us.ibm.com or IBM Service

Q4.3:

Where do I obtain IBM Spectrum Scale RAID and GPFS Native RAID fixes for my storage server?

A4.3:

You can obtain fixes for your ESS from Fix Central.

Note: You can narrow the results by selecting a specific version or platform:

• To see fixes for a specific version, select an option from the Installed version drop-down menu in the Change your selection menu.
• To see fixes for a specific platform, select an option from the Platform drop-down menu in the Change your selection menu.

Q4.4:

What are the current advisories for IBM Spectrum Scale RAID?

A4.4:

You can subscribe to notifications for IBM Spectrum Scale and IBM Elastic Storage Server to stay updated on current advisories.

Perform the following steps to subscribe to notifications:

1. Go to https://www-945.ibm.com/systems/support/myview/subscription/css.wss/.
2. Enter your login credentials.
3. Enter IBM Spectrum Scale and IBM Elastic Storage Server in the Product lookup search bar.
4. Click Subscribe.

Q4.5:

Are ESS and GSS serviced and supported as solutions?

A4.5:

ESS and GSS are loosely bundled solutions. This means that the hardware, firmware, and software levels have all been tested and tuned together. However, each individual part maintains its own warranty and support agreements. You do need some expertise in understanding the individual parts so that you can get the correct service assistance when you call 1-800-IBM-SERV. For IBM Spectrum Scale and IBM Spectrum Scale RAID software related issues, the service is delivered by fix packs that maintain the tight linkage between the software component stack and associated hardware and firmware. Always call 1-800-IBM-SERV before maintaining or changing levels of the individual parts.

Q5.1:

What is the mmvdisk command?

A5.1:

Beginning in ESS 5.3.1 (with IBM Spectrum Scale RAID 5.0.1), the mmvdisk command provides an alternative method for managing IBM Spectrum Scale RAID. The mmvdisk command integrates most aspects of IBM Spectrum Scale RAID administration. It encourages and enforces consistent best practices regarding server, recovery group, vdisk NSD, and file system configuration.

For more information, see "Managing IBM Spectrum Scale RAID with mmvdisk" in the IBM Spectrum Scale RAID: Administration guide.

Q5.2:

Are mmvdisk and the IBM Spectrum Scale RAID legacy commands compatible?

A5.2:

A recovery group cannot be managed by both mmvdisk and the IBM Spectrum Scale RAID legacy commands.

Q5.3:

Can mmvdisk be used to manage an existing ESS cluster?

A5.3:

The mmvdisk command does provide a command to place existing ESS recovery groups under mmvdisk management. After a recovery group is placed under mmvdisk management, it cannot be returned to management by the IBM Spectrum Scale RAID legacy commands.

Q5.4:

How do I tell whether a recovery group is being managed by the IBM Spectrum Scale RAID legacy commands or by mmvdisk?

A5.4:

The mmvdisk recoverygroup list command shows the status of all recovery groups in a cluster. A recovery group that is managed by the IBM Spectrum Scale RAID legacy commands is marked as non-mmvdisk. If a recovery group is not marked as non-mmvdisk, it is managed by mmvdisk.

Q5.5:

Is use of the mmvdisk command required by any ESS function?

A5.5:

All ESS MES upgrade procedures that are described in Q2.6 Can ESS hardware be upgraded non-disruptively? require an ESS cluster to be placed under mmvdisk management. IBM Elastic Storage System 3000 requires that the cluster use the mmvdisk command. For all other ESS functionality, the use of mmvdisk management is optional.

Q5.6:

Is it possible to use the ESS GUI in an ESS cluster managed by mmvdisk?

A5.6:

Beginning in ESS 5.3.2, the ESS GUI can be used in ESS clusters that are managed by mmvdisk.

Q6.1:

What are the supported 5148-22L protocol node configurations for ESS?

A6.1:

A protocol node feature code is available starting in ESS V5.3.1.1. This protocol node feature code makes it possible to purchase Power8 nodes with a specific hardware configuration, which are tested and tuned by IBM to provide Cluster Export Services (CES) services. The machine type and model (MTM) for protocol nodes is 5148-22L. The following hardware configuration is included:

• 8247-22L Power8 model
• Two 10 core 3.34 Ghz Power8 processors
• 128 GB or greater memory
• Two 600 GB 10k RPM SAS HDDs in RAID10 mirror using IPRaid adapter
• 1 GbE 4-port network adapter in slot C12
• Three x16 or x8 network adapters in slots C5, C6, or C7
• Four x8 network adapters in slots C2, C3, C10, or C11 available by additional card orders (through MES)

Q6.2:

Can I use non-5148-22L protocol nodes with ESS?

A6.2:

Non-5148-22L protocol nodes can be based on PPC64LE, PPC64BE, or x86_64 architectures. It is suggested that they adhere to the guidance in the IBM Spectrum Scale FAQ. All hardware, cabling or connections, and power sequencing is owned by the customer.

Support for IBM Spectrum Scale on the chosen hardware is handled by the general IBM Spectrum Scale support and not by ESS solution support.

Only 5148-22L protocol nodes are supported for management by the EMS node. If a protocol node is not ordered with this exact model and type from IBM manufacturing, it cannot be added to the EMS xCAT server. While non-5148-22L protocol nodes of type PPC64LE, PPC64BE, or x86_64 can be joined to an ESS cluster using the IBM Spectrum Scale installation toolkit or the mmaddnode command, they must be owned, installed, deployed, upgraded, and managed by the customer. The ESS tool sets cannot be used for the management of OS, kernel, network manager, systemd, OFED, or firmware on non-5148-22L nodes. Using the ESS tool sets, including the EMS xCAT server, to manage customer supplied non-5148-22L protocol nodes is not supported.

The IBM Spectrum Scale code on customer supplied non-5148-22L protocol nodes is also managed separately from the ESS tool sets. The IBM Spectrum Scale installation toolkit can be used in this configuration to install or deploy and upgrade the IBM Spectrum Scale code on any customer supplied non-5148-22L protocol nodes. For more information about the installation toolkit, see IBM Spectrum Scale installation toolkit.

For the latest information on supported levels, see the IBM Spectrum Scale FAQ.

Q6.3:

How do I install and upgrade protocol nodes attached to an ESS cluster?

A6.3:

For more information, see Upgrading a cluster containing ESS and protocol nodes.

Q6.4:

Where can I find additional information about IBM Spectrum Scale protocol nodes?

A6.4:

For more information, see the following topics in the IBM Spectrum Scale IBM Knowledge Center:

• Planning for protocols
• Deploying protocols

Q6.5:

How does call home hardware and software work with protocol nodes?

A6.5:

When a serviceable event occurs on one of the monitored servers or protocol nodes, a call home hardware event is generated. This event is responsible for automatically opening a service request with IBM support, and managing the end-to-end lifecycle of the problem.

The call home software feature collects files, logs, traces, and details of certain system health events from different nodes and services in an IBM Spectrum Scale cluster.

These details are shared with the IBM support center for monitoring and problem determination. For more information, see Installing call home and Understanding call home.

You can configure call home (hardware and software) using the gsscallhomeconf command. You can use the --no-swcallhome option to set up just the call home hardware and skip the software call home setup.

The call home hardware and call home software can be set up using the following command:

[root@ems1 ~]# gsscallhomeconf -E ems1 -N ems1,gss_ppc64 --suffix=-ib

The command gives an output similar to the following example:

2017-02-07T21:46:27.952187 Generating node list...
2017-02-07T21:46:29.108213 nodelist: ems1 essio11 essio12
2017-02-07T21:46:29.108243 suffix used for endpoint hostname: -ib
End point ems1-ib registered successfully with systemid 802cd01aa0d3fc5137f006b7c9d95c26
End point essio11-ib registered successfully with systemid c7dba51e109c92857dda7540c94830d3
End point essio12-ib registered successfully with systemid 898fb33e04f5ea12f2f5c7ec0f8516d4
End point enclosure G5CT018 registered successfully with systemid
c14e80c240d92d51b8daae1d41e90f57
End point enclosure G5CT016 registered successfully with systemid
524e48d68ad875ffbeeec5f3c07e1acf
ESA configuration for ESS Callhome is complete.
Started configuring software callhome
Checking for ESA is activated or not before continuing.
Fetching customer detail from ESA.
Customer detail has been successfully fetched from ESA.
Setting software callhome customer detail.
Successfully set the customer detail for software callhome.
Enabled daily schedule for software callhome.
Enabled weekly schedule for software callhome.
Direct connection will be used for software calhome.
Successfully set the direct connection settings for software callhome.
Enabled software callhome capability.
Creating callhome automatic group
Created auto group for software call home and enabled it.
Software callhome configuration completed.

If you want to skip the call home software setup, use the following command:

[root@ems3 ~]# gsscallhomeconf -E ems3 -N ems3,gss_ppc64 --suffix=-te --register=all --no-swcallhome

The command gives an output similar to the following example:

2017-01-23T05:34:42.005215 Generating node list...
2017-01-23T05:34:42.827295 nodelist:    ems3 essio31 essio32
2017-01-23T05:34:42.827347 suffix used for endpoint hostname: -te
End point ems3-te registered sucessfully with systemid 37e5c23f98090750226f400722645655
End point essio31-te registered sucessfully with systemid 35ae41e0388e08fd01378ae5c9a6ffef
End point essio32-te registered sucessfully with systemid 9ea632b549434d57baef7c999dbf9479
End point enclosure SV50321280 registered sucessfully with systemid 600755dc0aa2014526fe5945981b0e08
End point enclosure SV50918672 registered sucessfully with systemid 92aa6428102b44a4a1c9a293402b324c
ESA configuration for ESS Callhome is complete.

Q6.6:

How do I tune my protocol nodes?

A6.6:

For more information, see GPFS configuration parameters for protocol nodes.

Q7.1:

Is it possible to have an ESS model GLx and an ESS model GLxS in the same cluster?

A7.1:

It is possible to have an ESS model GLx in the same cluster as an ESS model GLxS. The vdisk configuration depends on the use case that you have in place for the new ESS model.

If the new ESS model is used to create a new file system, then the configuration is straight-forward. If you are planning to add a new ESS model to expand the capacity of the existing file system, then you need to plan it more carefully. IBM Spectrum Scale does round-robin placement and parallel access of blocks across all of the NSDs of the same storage pool in a given file system.

The following points are related to mixing NSDs from an ESS model GLx with an ESS model GLxS:

• As the performance characteristics are different for the GLx and GLxS, it is recommended that you create different storage pools in the same file system and move data using IBM Spectrum Scale policies.
• If your use case does not support creating different storage pools, it is recommended that you create the same size NSDs when adding new NSDs into the existing file system.
  Note: Performance will be limited by the slowest NSDs. The aggregate performance will be the total number of NSDs x the slowest NSD performance.
• The new NSDs are used as soon as they are added into the file system. To balance the capacity utilization across new and old NSDs, run the mmrestripe command. It is advisable to wait for some time because some balancing might automatically happen depending upon your workload characteristics.
  Note: Running the mmrestripe command is an IO intensive operation.
• If multiple file systems are created using NSDs on the same ESS, the bandwidth will be shared across all the NSDs; therefore, the workload on one file system might impact the other file systems.

For more information, see General Tips for Storage with GPFS.

Q7.2:

Are ESS and IBM Spectrum Scale packages signed by IBM?

A7.2:

• Starting with Elastic Storage Server (ESS) release 5.3.6, ESS packages are signed by IBM with a GPG key.
• Starting with IBM Elastic Storage System (ESS) release 6.0.1, ESS packages are signed by IBM with a GPG key.
• Starting with the IBM Spectrum Scale release 5.0.4, all IBM Spectrum Scale packages on Red Hat Enterprise Linux and SLES operating systems on supported architectures are signed by IBM with a GPG key. Starting with the IBM Spectrum Scale release 5.0.5.1, repository metadata is also signed by IBM.

You can use the available public key to verify the signatures on the packages and the repository metadata.

The latest public key contents are as follows:

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: EKM
mQENBF0tE4ABCADTU4imcpDlIHvcK/qWdMMrs72lL9EYDtA/JNL5YCPNeIa/54aIe3xXFJZbzkjs
v+5INaxYv0DEQxXEFq8vA1pQGPIG1elb3fXgP7Iyfiy13KDrVEB8AY/Cr/zTmHV8IJNMN8jcBl6Z
vAED7fXE82Q4jQ3djbg0OYBq2PeVS+wM5Y8n1+tmpVmcD9oLzYhJPeCsbFi6BAVgXBmyh4arrn15
OLSfD5jBnnOT926N2mpnsfubyGitQlywjJJuESnF9Ub9QMT7jNjGcg6frxHVOMUsIstmg01GBnvx
I/P/BvdiIqGjOTInka78+rYJpxZWPlbu/Xg/NXJ9sERjXuT30GCHABEBAAG0DVNwZWN0cnVtU2Nh
bGWJATkEEwEIACMFAl0tE4ACGy8HCwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRC9vnXD7+tu
654UCAClAR99Jhsdm47V2JvOBYLxcxdHqoqY+MqgKxeuy11Tp/enpqoGigZAcPbzlRvlJTOyh0Pa
PQC1y0oaKDROR5aOuCd0Cz3xbSQ92mWX0FkA7D9KNlAuxlGD6Ic58AvQ6RBv/mxblfH6gXHlc+Q0
+YFOY5YlvgKLYJ+exGngzieZfxspyyTab7FZe06G/lCm9U+mOfQ/7ODH6AvNIRmsCCg5uUeAmQOa
3+0RpWzN09nlkSYlkMlXvyZSWpwTEXLPtfDW0kzxsl1k4IzgyFMOsw6oLO5TVMZyL828MOuJtqU8
O6rS6+/RIho7GhiQ8SklugSFlFnT9fx5TRJcCiJmhHeF
=v7fu
-----END PGP PUBLIC KEY BLOCK-----

Save these contents in a file and import it to verify the signature manually.

For IBM Spectrum Scale packages, the installation toolkit does the signature verification automatically before installation or upgrade. For more information, see https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.5/com.ibm.spectrum.scale.v5r05.doc/bl1ins_verifysignature.htm.

Q8.1:

What type of optical cable should I use for 100GbE?

A8.1:

There is an EEPROM in the QSFP+/QSFP28 that specifies the manufacturer, speed, and length of the cable. Many devices check this EEPROM and will work with an EEPROM from that manufacturer only. To be safe, you need a cable with a QSFP/EPROM on each end with the manufacturer codes from that manufacturer. These cables will work only if the proper end is attached to the correct manufacturer’s device.

IBM’s cables have Mellanox EEPROMs on both ends.

Q8.2:

What is a DAC cable?

A8.2:

This is a QSFP+/QSFP28 short distance copper cable that does not have the EPROM check described in 8.1 What type of optical cable should I use for 100GbE?. It is meant for within a rack.

Q8.3:

Can I connect a 100GbE adapter port to a 40GbE switch?

A8.3:

Yes, you need to use a 40GbE cable with the EEPROMs as described in 8.1 What type of optical cable should I use for 100GbE?.

Q8.4:

Can I connect a 100GbE adapter port to a 10GbE or 25GbE switch?

A8.4:

Yes, you will need a QSA adapter from Mellanox. For more information, see https://community.mellanox.com/s/article/all-about-qsa-adapter.

IBM, the IBM logo, and ibm.com® are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( ® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information at www.ibm.com/legal/copytrade.shtml

Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom

Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Java™ and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

Red Hat, the Red Hat "Shadow Man" logo, and all Red Hat-based trademarks and logos are trademarks or registered trademarks of Red Hat, Inc., in the United States and other countries.

UNIX is a registered trademark of the Open Group in the United States and other countries.

Microsoft, Windows, Windows NT, and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries, or both.

Other company, product, and service names may be the trademarks or service marks of others.