Hardware and software requirements

This section specifies the minimum hardware and software requirements for Cloudera Data Platform (CDP) Private Cloud Base, IBM Storage Scale CES HDFS transparency and IBM Storage Scale.

Hardware requirements

Following are the Hardware requirements for each product and component:

Cloudera Data Platform Private Cloud Base

For Cloudera Data Platform Private Cloud Base hardware requirements information for a specific release, see Hardware Requirements.

CES HDFS

For production, it is recommended to have two NameNodes and three or more DataNodes.

For production, it is recommended that each NameNode x86 server should have a minimum of two sockets with at least eight cores each with 128 GB memory.

For production, it is recommended that each DataNode x86 server should have a minimum of two sockets with at least eight cores each with 64 GB memory.

For better performance, reserve 20% of the system physical memory, subjected to a maximum of 20 GB per node, for IBM Storage Scale pagepool.

IBM Storage Scale

For IBM Storage Scale hardware requirements information, see the Hardware requirements topic in IBM Storage Scale: Concepts, Planning, and Installation Guide.

Shared storage

Shared storage can either be ESS, ECE or SAN based storage. Hadoop services including HDFS Transparency cannot be installed onto the storage nodes (SAN, ECE, ESS).

If you have ESS as a shared storage, refer the documentation for your model under IBM Elastic Storage® System documentation. ESS is set up and tuned by IBM® Lab Services.

If you have ECE as a shared storage, refer IBM Storage Scale Erasure Code Edition Hardware requirements in the IBM Storage Scale Erasure Code Edition Guide.

Note:

Ensure that the CDP Private Cloud Base cluster and the CES HDFS cluster are on the same OS and architecture platform. Cloudera requires nodes to install its component software on the same OS and architecture platform. CES protocol does not support mixed architecture levels. For example, if CES HDFS protocol is on x86_64 platform, then all the CES protocol nodes and CDP Private Cloud Base must be on x86_64 platform. The shared storage can be on a different architecture platform. For example, if CES protocol nodes is on x86_64, ESS can be on power.
FPO is not supported for CDP Private Cloud Base with IBM Storage Scale.

Software requirements

Following are the Software requirements for each product and component:

Cloudera Data Platform Private Cloud Base: OpenJDK 8; PostgreSQL 10 server; Python 2
CES HDFS: OpenJDK 8; Python 2 and Python 3.6 or later
IBM Storage Scale: For IBM Storage Scale software requirements information, see the Software requirements topic in IBM Storage Scale: Concepts, Planning, and Installation Guide.

Note: Ensure that the CDP Private Cloud Base cluster and the CES HDFS cluster are at the same Operating System level. The shared storage can be at a different Operating System level. For example, if CES protocol nodes are on RH 8.2, ESS can be on RH 7.7.

For more information on requirements for Hardware, Operating system, Database, and Java™, see CDP Private Cloud Base Requirements and Supported Versions.

For information on limitations for CDP Private Cloud Base with IBM Storage Scale, see Limitations.

Cluster host and role assignments

Cloudera Data Platform Private Cloud Base: Refer CDP Private Cloud Base for the specific CDH release in the Runtime Cluster Hosts and Role Assignments section.
CES HDFS: NameNodes - 2 (HA); DataNodes - 3 or more