Storage on rack-mounted models

Topology

The Integrated Analytics System storage topology is completely redundant using multi-pathing on top of the redundant fabric for both high availability and performance.

GPFS

Integrated Analytics System uses GPFS for all shared file systems, both fault-domain-wide and system-wide. There is a separate GPFS cluster for each fault domain in the system. GPFS evenly distributes the allocations and input/output among the underlying storage devices for the flash-based storage.

Appliance storage

Details on rack-mounted appliance storage:

  • Server Local Storage (600GB protected storage per node)
    • 2 HDDs (Mirrored) for OS, GPFS, Docker, Containers, /tmp, node-local diagnostics information
    • Four partitions:
      / (root)
      349 GB for the operating system, etc.
      /var
      20 GB for Node Local Logs, Platform Manager Private (etcd) storage
      /swap
      128 GB for operating system swap space
      /dockerfs
      100 GB for Node Local docker agent storage (StorageDriver= device mapper)
  • Foundation Local Storage (approximately 5 TiB protected storage per appliance)
    • Extra LUN on FlashSystem 900-1 with Extra Storage Modules
    • appliance wide logs
    • GPFS file system cross-mounted at /opt/ibm/appliance/storage/platform
  • Db2® installation and database directory metadata
    • not performance critical
    • 1 LUN From FlashSystem 900-1 (1 TiB)
    • GPFS file system, cross-mounted on all nodes at /opt/ibm/appliance/storage/head
    • passed to containers at -v /opt/ibm/appliance/storage/head:/head
  • Archive Logs, landing zone, load copies;
    • not performance critical
    • 1 LUN from FlashSystem 900-1 (10 TiB)
    • GPFS file system, cross-mounted in all nodes as /opt/ibm/appliance/storage/scratch
    • passed to containers at –v /opt/ibm/appliance/storage/scratch:/scratch
  • High Performance for user data and Db2 Transient (logs, temp tables)
  • FlashSystem 900 FLASH Arrays
    • M4001 8 Flash models:
      • each array has eight 5.2 TiB Flash Modules
      • configured as RAID5 6 data drives + 1 Parity + 1 Spare (32 TiB Protected storage)
    • M4001 12 Flash models:
      • each array has twelve 5.2 TiB Flash Modules
      • configured as RAID5 10 data drives + 1 Parity + 1 Spare
    • M4002 12 Flash models:
      • each array has twelve 7.7 TiB Flash Modules
      • configured as RAID5 10 data drives + 1 Parity + 1 Spare
  • File Systems - GPFS to evenly distribute the allocations and I/O across all of the underlying devices
  • One file system on the Block Devices from the half of the LUNs from each FlashSystem 900 array in the rack (/opt/ibm/appliance/storage/data)
  • Another file system on the Block Devices from the other half of the LUNs from each FlashSystem 900 array (/opt/ibm/appliance/storage/local)

File system sizes

The following table lists file system sizes.

Table 1. File system sizes in M4001 models
File system User Type Scope M4001-003 M4001-006 M4001-010
/ Platform ext4 Node 350 GB/node
/swap Platform ––– Node 128 GB/node
/var Platform ext4 Node 20 GB/node
/platform Platform GPFS System-wide 3.0 TiB 3.0 TiB 3.0 TiB
/head Application GPFS System-wide 1.0 TiB 1.0 TiB 1.0 TiB
/scratch Application GPFS System-wide 10 TiB 10 TiB 10 TiB
/data * Application GPFS System-wide 16.6 TiB 31.5 TiB 46.6 TiB
/local Application GPFS System-wide 15.0 TiB 30.0 TiB 45.0 TiB

* /data includes space for MLN0000 (catalog partition)

Table 2. File system sizes in M4002 models
File system User Type Scope M4002-003 M4002-006 M4002-010 M4002-020
/ Platform ext4 Node 470 GB/node
/swap Platform ––– Node 128 GB/node
/var Platform ext4 Node 200 GB/node
/platform Platform GPFS System-wide 1.0 TiB 1.0 TiB 1.0 TiB 1.0 TiB
/head Application GPFS System-wide 1.0 TiB 1.0 TiB 1.0 TiB 1.0 TiB
/scratch Application GPFS System-wide 5 TiB 10 TiB 15 TiB 15 TiB
/data * Application GPFS System-wide 27 TiB 54 TiB 81 TiB 162 TiB
/local Application GPFS System-wide 27 TiB 54 TiB 81 TiB 162 TiB

* /data includes space for MLN0000 (catalog partition)