Setting up Big Data Extension clusters and services

Use the Ambari installation wizard in your browser to complete the deployment of clusters and setting up of Big Data Extension services and Hadoop components.

1. Open a browser and access the Ambari server dashboard.
   Use the following default URL:http://<myserver.ibm.com>:8080

   Note: You can use the fully qualified domain name (FQDN) or the IP Address of the server.

   The default user name is admin, and the default password is admin.

2. Click Launch Install Wizard on the Ambari Welcome page.

   The CLUSTER INSTALL WIZARD opens.

3. Enter a name for the cluster you want to create on the Get Started page and click Next.
   Note: The name cannot contain blank spaces or special characters.
4. Select BigInsights 4.2.TNPM_BDE stack on the Select Stack page and click Next.
5. Complete the following steps on the Install Options page:
   1. List all of the nodes that are used in the IBM® Open Platform with Apache Spark and Apache Hadoop cluster in Target Hosts pane.
      Specify one node per line, as in the following example:

      node1.abc.com
      node2.abc.com
      node3.abc.com
      

      Note: The host name must be the FQDN. For example, <myserver.ibm.com>.
   2. Select Provide your SSH Private Key to automatically register hosts and click SSH Private Key link on Host Registration Information pane.

      If the root user installed the Ambari server, the private key file is /root/.ssh/id_rsa. You can browse to the .ssh/id_rsa file and the Ambari web interface uploads the contents of the key file, or you can open the file and copy and paste the contents into the SSH key field.

   3. Click Register and Confirm.
6. Verify that the correct hosts for your cluster are located successfully on the Confirm Hosts page.
   If hosts that are selected are incorrect, remove the hosts one-by-one by following these steps:

   1. Click the box next to the server to be removed.
   2. Click Remove in the Action column.
      Note:

      • If warnings are found during the check process, click Click here to see the warnings to see the warnings. The Host Checks page identifies any issues with the hosts. For example, a host might have Transparent Huge Pages or Firewall issues.
      • Ignore the process issues that are not related to Big Data Extension.
   3. After you resolve the issues, click Rerun Checks on the Host Checks page. After you have confirmed the hosts, click Next.
7. Select the following services on the Choose Services page:

   Service	Version	Description
   HDFS	2.7.2	Apache Hadoop Distributed File System (HDFS)
   YARN + MapReduce2	2.7.2	Apache Hadoop NextGen MapReduce (YARN)
   ZooKeeper	3.4.6	Centralized service that provides reliable distributed coordination.
   Ambari Metrics	0.1.0	A system for metric collection that provides storage and retrieval capability for metrics that are collected from the cluster.
   Kafka	0.9.0.1	A high-throughput messaging system.
   TNPM BDE	1.4.4	Tivoli® Netcool® Performance Manager Big Data Extension cluster service
   TNPM BDE Spark Client Scala 2.11	2.0.1	Apache Spark is an engine for large-scale data processing. The Apache Spark client library is compiled on Scala 2.11 and is specific to Big Data Extension.

8. Click Next.
9. Assign the master nodes to hosts in your cluster for the services you selected on the Assign Masters page and click Next.

   You can accept the current default assignments. To assign a new host to run services, click the list next to the master node in the left column and select a new host.

10. Click Next.
11. Assign the slave and client components to hosts in your cluster on the Assign Slaves and Clients page. Select all services for assignment.

    Click all or none to decide the host assignments. Or, you can select one or more components next to a selected host.

12. Click Next.
13. Update the configuration settings for the following services and components on Customize Services pane.
    You can see a set of tabs from where you can manage the settings for Hadoop and Big Data Extension components.

    • Set up YARN
    • Set up HDFS
    • Setting up Kafka Broker logs
    • Set up Big Data Extension services
    • Set up communication with Tivoli Netcool Performance Manager system
14. Click Next after you have reviewed your settings, and completed the configuration of the services.
15. Verify that your settings are correct and click Deploy on the Review page.
16. See the progress of the installation on Install, Start, and Test page.

    The progress bar gives the overall status and the main section of the page gives the status for each host. When you click the task, log for a specific task can be displayed.

17. Click Next after the services are installed successfully.
18. Review the completed tasks on the Summary page and click Complete.