Storage (HDFS)

If you are using the Hadoop Distributed File System (HDFS), you can view the HDFS web interface through the cluster management console ( Resources > Storage (HDFS)). You must have also configured the DFS_GUI_HOSTNAME and the DFS_GUI_PORT parameters.

Note: By default, the cluster management console uses an HTTPS (secure and encrypted with TLS) connection; the HDFS web interface uses an HTTP connection. To enable both HTTPS and HTTP pages, refer to your browser documentation to enable your browser to display mixed content.

HDFS web interface

The HDFS web interface is the front page of the NameNode in your cluster. Use it to complete the following tasks:
  • Monitor your HDFS cluster with basic statistics. You can view a count of DataNodes and basic information about the health, capacity, and usage of the cluster.
  • Browse the contents of the file system, including input and output directories and files. You can also view HDFS input and output directories for each job from the job’s Summary tab at Workload > MapReduce > Jobs.
    Note: You can access files only if you have an OS execution user account for each application/consumer.
  • View log files generated by the Hadoop daemons on the NameNode and Secondary NameNodes for distributed debugging and troubleshooting.
Important: IBM® Spectrum Symphony only enables you to connect to the HDFS web interface through the cluster management console. It does not provide any security mechanism for HDFS through IBM Spectrum Symphony. Be aware that some information, such as the web server version, may be exposed through this interface.

Configuration for the HDFS web interface

To view the HDFS web interface in the cluster management console, you must have configured the following parameters:
  • DFS_GUI_HOSTNAME: Specifies the hostname of the HDFS NameNode.
  • DFS_GUI_PORT: Specifies the port that the HDFS NameNode listens on. This port is by default 50070.
You can configure these parameters either during or after installation:
  • During installation, set up these parameters as environment variables.
  • If you did not set up variables during installation, set up these parameters after installation using the wsm.conf configuration file. Follow these steps:
    1. Open the wsm.conf file located under $EGO_CONFDIR.
    2. Set values for the DFS_GUI_HOSTNAME and the DFS_GUI_PORT parameters.
    3. Save the file.
    4. Restart the WEBGUI service.

      The HDFS node tree will appear on the cluster management console.

With the default configuration, the HDFS web interface is located at http://namenode-name:50070/.