IBM Big SQL 3.0 新功能系列,第二部分

IBM InfoSphere BigInsights 3.0 环境搭建

Comments

系列内容:

此内容是该系列 # 部分中的第 # 部分: IBM Big SQL 3.0 新功能系列,第二部分

敬请期待该系列的后续内容。

此内容是该系列的一部分:IBM Big SQL 3.0 新功能系列,第二部分

敬请期待该系列的后续内容。

搭建 BigInsights 3.0 环境

本次示例中的环境为 7 个物理机器的集群,每台机器 CPU 配置为 2 C 8 Core Intel ( R ) Xeon ( R ) CPU X5570 @ 2.93 GHz,内存配置为 32 GB,磁盘配置为 1 块 500 GB 的 7200 转 SATA 盘,操作系统为 Red Hat Enterprise Linux Server release 6.4 ( Santiago )。整个集群将部署为 1 个管理节点,6 个数据节点,管理节点中部署 NameNode、JobTracker、Big SQL、Web Console、Hive、HBase Master、Jaql UDF server、Hive Metadata Database、Secondary NameNode、HttpFS、Monitoring、Zookeeper、Oozie 和 Alert 等组件,数据节点将部署 HBase region server 和 Big SQL 工作节点(Work Nodes)等。

需要注意的是:代码清单因为每行有长度限制,比较长的语句已经使用反斜杠 ( \ ) 进行续行。如果遇上个别因排版原因带来的回车换行符,大家可以在实际执行过程中将回车换行符去掉再执行。

第一步、修改 Hosts 文件

在修改 hosts 文件前,为了将每台服务器的主机名永久有效,需要编辑 /etc/sysconfig/network 文件,将“HOSTNAME = 机器名”。接下来我们需要将每台物理服务器的 /etc/hosts 文件进行修改,在该文件中指定每台机器的 IP 地址,机器长名和机器短名,命令如下:

清单 1. 修改 hosts 文件示例
[ root @ test01 ~ ] cat >> /etc/hosts << EOF
192.168.0.1 test01.localdomain test01
192.168.0.2 test02.localdomain test02
192.168.0.3 test03.localdomain test03
192.168.0.4 test04.localdomain test04
192.168.0.5 test05.localdomain test05
192.168.0.6 test06.localdomain test06
192.168.0.7 test07.localdomain test07
EOF

需要注意的是,每台机器的机器名需要固定下来,可以通过编辑 /etc/sysconfig/network 文件来完成,其中“HOSTNAME = 用户永久的主机名”。

第二步、配置物理服务器节点间的互信

首先在 test01 节点上生成节点间互信所需文件,再通过读取 /etc/hosts 获取服务器 IP 地址、长名和短名列表,将需要修改的配置参数以及互信文件分发到各个节点,具体命令如下:

清单 2. 配置节点间互信示例
[ root @ test01 ~ ] # mkdir -p /root/.ssh; chmod 700 /root/.ssh
[ root @ test01 ~ ] # ssh-keygen -t dsa -N '' -f /root/.ssh/id_dsa
[ root @ test01 ~ ] # cp /root/.ssh/id_dsa.pub /root/.ssh/authorized_keys
[ root @ test01 ~ ] # chmod 600 /root/.ssh/authorized_keys
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test | \
 awk ' { print $1" "$2" "$3 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh -o StrictHostKeyChecking=no $target " setenforce 0 "
ssh -o StrictHostKeyChecking=no $target " mkdir -p /root/.ssh ; chmod 700 /root/.ssh "
scp /root/.ssh/* $target:/root/.ssh/ 
scp /etc/hosts $target:/etc/hosts
done
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test | grep -v test01 | awk ' \
 { print $3 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
scp /root/.ssh/known_hosts $target:/root/.ssh/ 
ssh $target " hostname $target "
done

第三步、关闭 IPV6 功能

在 test01 节点上使用 ssh 命令登录到各个节点,将 ipv6 功能关闭掉,集群短名列表是通过读取 hosts 文件获得的,具体命令如下:

清单 3. 关闭 IPV6 示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test | awk ' { print $3 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh $target " echo \" install ipv6 /bin/true\ " > \
/etc/modprobe.d/ipv6off.conf ; cat /etc/modprobe.d/ipv6off.conf "
done

命令执行完成后需要重启整个集群所有计算机。重启完成后,可以登录各个节点使用 smod | grep -i ipv6 或 ifconfig | grep -i inet6 验证 IPV6 是否全部关闭了。

第四步、关闭各种服务

在所有节点上关闭 selinux,命令如下:

清单 4. 关闭 selinux 示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test | awk ' { print $3 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh $target " sed s/=enforcing/=disabled/g /etc/selinux/config -i ; grep \" \
SELINUX=disabled\ " /etc/selinux/config "
ssh $target " setenforce 0 "
done

关闭集群中各个节点的防火墙,命令如下:

清单 5. 关闭防火墙示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test | \
 awk ' { print $3} ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh $target " service iptables save " 
ssh $target " service iptables stop "
ssh $target " chkconfig iptables off "
ssh $target " sed -i \" /service iptables stop/d\ " /etc/rc.local ; echo \" \
 service iptables stop\ " >> /etc/rc.local "
done

关闭集群中各个节点的 dnsmasq,命令如下:

清单 6. 关闭 dnsmasq 示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test | \ 
awk ' { print $3 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh $target " service dnsmasq stop " 
ssh $target " chkconfig dnsmasq off "
done

第五步、检查 BigInsights 所需的监听端口是否被占用

在 test01 节点上通过使用 ssh 命令登录到各个节点,检查 BigInsights 所需的各个端口是否被占用,具体命令如下:

清单 7. 检查集群中各个节点端口信息示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test \ 
| awk ' { print $3 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh $target " lsof -
i:53,1528,2181,2182,7052,7869,7870,7871,7872,7873,7874,7875,7876,7877,\
7878,7879,7880,7881,7882,7883,8004,8006,8007,8080,8200,8280,8888,9000,\
9001,9093,9099,9999,10000,10101,10102,14000,18080,50010,50020,50030,\
50070,50075,50090,60000,60010,60020,60030 "
done

我们将使用如下命令杀掉占用 BigInsights 所需端口的各个进程,具体命令如下:

清单 8. 在集群中各节点上杀掉占用 BigInsights 端口进程示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test \
| awk ' { print $3 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh $target " lsof -
i:53,1528,2181,2182,7052,7869,7870,7871,7872,7873,7874,7875,7876,7877,\
7878,7879,7880,7881,7882,7883,8004,8006,8007,8080,8200,8280,8888,9000,\
9001,9093,9099,9999,10000,10101,10102,14000,18080,50010,50020,50030,\
50070,50075,50090,60000,60010,60020,60030 | grep -v COMMAND \
| grep -v grep | awk '{print \$2}' | uniq | xargs kill -9 "
done

再次确认 BigInsights 所需的各个端口是否已经释放出来了,具体命令如下:

清单 9. 再次检查集群中各个节点端口信息示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test \
| awk ' { print $3 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh $target " lsof -
i:53,1528,2181,2182,7052,7869,7870,7871,7872,7873,7874,7875,7876,\
7877,7878,7879,7880,7881,7882,7883,8004,8006,8007,8080,8200,8280,\
8888,9000,9001,9093,9099,9999,10000,10101,10102,14000,18080,50010,\
50020,50030,50070,50075,50090,60000,60010,60020,60030 \
| grep -v COMMAND | grep -v grep | awk '{print \$2}' \
| uniq | xargs kill -9 "
done

第六步、准备 BigInsights 安装所需的用户

由于 BigInsights 3.0 采用多用户的方式,每个服务都有自己的用户,建议安装 BigInsights 的各个物理服务器使用全新的操作系统,这样不会出现用户和组冲突问题。如果是已经存在的操作系统,则需要检查 /etc/passwd 和 /etc/group 文件,对比其中的用户和组,看是否有和 BigInsights 所需的用户和组冲突,如果存在则将其删除掉,比如可以在各个服务器上使用以下命令删除冲突的用户和组:

清单 10. 删除冲突的用户和组示例
[ root @ test01 ~ ] # userdel -r user1
[ root @ test01 ~ ] # userdel -r user2
[ root @ test01 ~ ] # userdel -r user3
[ root @ test01 ~ ] # userdel -r dataadmin1
[ root @ test01 ~ ] # userdel -r dataadmin2
[ root @ test01 ~ ] # userdel -r appadmin1
[ root @ test01 ~ ] # userdel -r appadmin2
[ root @ test01 ~ ] # userdel -r sysadmin1
[ root @ test01 ~ ] # userdel -r biadmin
[ root @ test01 ~ ] # userdel -r hadoop
[ root @ test01 ~ ] # userdel -r monitoring
[ root @ test01 ~ ] # userdel -r hbase
[ root @ test01 ~ ] # userdel -r oozie
[ root @ test01 ~ ] # userdel -r console
[ root @ test01 ~ ] # userdel -r alert
[ root @ test01 ~ ] # userdel -r httpfs
[ root @ test01 ~ ] # userdel -r catalog
[ root @ test01 ~ ] # userdel -r mapred
[ root @ test01 ~ ] # userdel -r hive
[ root @ test01 ~ ] # userdel -r zookeeper
[ root @ test01 ~ ] # userdel -r orchestrator
[ root @ test01 ~ ] # userdel -r hdfs
[ root @ test01 ~ ] # groupdel supergroup
[ root @ test01 ~ ] # groupdel appAdmins
[ root @ test01 ~ ] # groupdel sysAdmins
[ root @ test01 ~ ] # groupdel dataAdmins
[ root @ test01 ~ ] # groupdel biadmin

接下来为每个服务器创建 BigInsights 所需的管理员用户 biamdin,将其 uid 指定为 200,并在各个节点为 biadmin 用户准备节点间互信:

清单 11. 创建 biadmin 用户并配置互信示例
[ root @ test01 ~ ] # useradd -u 200 biadmin
[ root @ test01 ~ ] # mkdir -p /home/biadmin/.ssh
[ root @ test01 ~ ] # chmod 700 /home/biadmin/.ssh
[ root @ test01 ~ ] # ssh-keygen -t dsa -N '' -f /home/biadmin/.ssh/id_dsa
[ root @ test01 ~ ] # cat /home/biadmin/.ssh/id_dsa.pub > \
/home/biadmin/.ssh/authorized_keys
[ root @ test01 ~ ] # chmod 600 /home/biadmin/.ssh/authorized_keys
[ root @ test01 ~ ] # chown -R biadmin.biadmin /home/biadmin/.ssh
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test \
| grep -v test01 | awk ' { print $1 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh -o StrictHostKeyChecking=no $target " useradd -u 200 biadmin ; \
mkdir -p /home/biadmin/.ssh ; chmod 700 /home/biadmin/.ssh "
scp /home/biadmin/.ssh/* $target:/home/biadmin/.ssh/
ssh -o StrictHostKeyChecking=no $target \
" chown -R biadmin.biadmin /home/biadmin/.ssh "
done
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test \
 | awk ' { print $1 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh -o StrictHostKeyChecking=no $target " sed \"/biadmin/d\" -i \
/etc/sudoers;sed –i \"s/Defaults requiretty/#Defaults requiretty/g\" \
 /etc/sudoers; echo \"biadmin ALL=(ALL) NOPASSWD: ALL\" >> /etc/sudoers "
done
[ root @ test01 ~ ] # su - biadmin
[biadmin @ test01 ~ ] $ nodes = " ` cat /etc/hosts | grep test | awk ' \
{ print $1" "$2" "$3 } ' ` "
[biadmin @ test01 ~ ] $ echo $nodes
[biadmin @ test01 ~ ] $ for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh -o StrictHostKeyChecking=no $target " ls "
done
[biadmin @ test01 ~ ] $ for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
scp /home/biadmin/.ssh/known_hosts \
$target:/home/biadmin/.ssh/known_hosts 
done

第七步、安装操作系统各种补丁包

安装补丁包需要 RedHat 的安装介质,在本次环境搭建过程中使用的是 RedHat 安装介质的 ISO 文件,具体命令如下:

清单 12. 配置 OS 安装源示例
[ root @ test01 ~ ] # isopath=" /var/www/html/iso "
[ root @ test01 ~ ] # mkdir -p $isopath
[ root @ test01 ~ ] # echo " /root/builds/\
RHEL6.4-20130130.0-Server-x86_64-DVD1.iso \
$isopath iso9660 loop,ro,auto 0 0 " >> /etc/fstab
[ root @ test01 ~ ] # mount -o loop /root/builds/\
RHEL6.4-20130130.0-Server-x86_64-DVD1.iso $isopath 
[ root @ test01 Packages ~ ] # cd /var/www/html/iso/Packages
[ root @ test01 Packages ~ ] # yum install -y httpd

为 RedHat 的安装介质配置 http 服务,使其作为所有节点的 YUM 源,具体命令如下:

清单 13. 配置 http 服务和 YUM 源示例
[ root @ test01 ~ ] # head -n 7 /etc/yum.repos.d/rhel-source.repo \
| sed "s/rhel-source/rhel-http/;\
s/=0/=1/;s/=ftp.*/=http:\/\/`hostname -i`\/iso/" > \
/etc/yum.repos.d/rhel-http.repo
[ root @ test01 ~ ] # cat /etc/yum.repos.d/rhel-http.repo 
" = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > 
以下是 rhel-http.repo 输出结果"
[ rhel – http ]
Name = Red Hat Enterprise Linux $releasever - $basearch - Source
baseurl = http://192.168.0.1/iso
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
[ root @ test01 ~ ] # service httpd start
[ root @ test01 ~ ] # chkconfig httpd on
[ root @ test01 ~ ] # nodes="`cat /etc/hosts | grep test \
| awk '{print $1}' `"
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
scp /etc/yum.repos.d/rhel-http.repo $target:/etc/yum.repos.d/rhel-http.repo
done

接下来在各个节点(需要手工登录各个节点)安装所需要的补丁包如 expect、nc、zip、bash、tar、trilead-ssh2、unzip、yum-utils.noarch、gcc-c++ kernel-devel、rpm-build、ksh 和 NetworkManager 等,具体命令如下:

清单 14. 安装操作系统补丁包示例
[ root @test- [ 01 ] - [ 07 ] ~ ] # yum install -y expect
[ root@ test - [ 01 ] - [ 07 ] ~ ] # yum install -y expect nc zip \
bash tar trilead-ssh2 unzip yum-utils.noarch gcc-c++ kernel-devel \
rpm-build ksh NetworkManager

第八步、配置各个节点间的时间同步

本次测试的集群由于在内网,无法访问互联网,所以我们将让其他 6 个节点与 test01 进行时间同步,具体命令如下:

清单 15. 配置主节点示例
[ root @ test01 ~ ] # cp /etc/ntp.conf /etc/ntp.conf.orig
[ root @ test01 ~ ] # cat > /etc/ntp.conf << EOF
restrict default nomodify notrap noquery
restrict 127.0.0.1
restrict 192.168.0.0 mask 255.255.255.0 nomodify
# server 0.pool.ntp.org
# server 1.pool.ntp.org
# server 2.pool.ntp.org
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
driftfile /var/lib/ntp/drift
broadcastdelay 0.008
keys /etc/ntp/keys
EOF
[ root @ test01 ~ ] # service ntpd restart
[ root @ test01 ~ ] # chkconfig ntpd on

在集群中注册 crontab 任务,以同步时间,具体命令如下:

清单 16. 集群时间同步示例
[ root @ test01 ~ ] # cat > /tmp/sync.conf << EOF 
# Add time sync:
*/20 * * * * ( /usr/sbin/ntpdate -u test01 ) && ( hwclock --systohc )
EOF
[ root @ test01 ~ ] # nodes=" ` cat /etc/hosts.temp | grep test \
| grep -v test01 | awk ' { print $1 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
scp /tmp/sync.conf $target:/tmp/sync.conf
ssh $target " crontab /tmp/sync.conf ; service crond reload ; "
done

之后可以在整个集群所有服务上检查 crontab 列表,具体命令如下:

清单 17. 检查集群 crontab 示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts.temp | grep test \
| grep -v test01 | awk ' { print $1 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
ssh $target " /usr/sbin/ntpdate -u test01 "
ssh $target " crontab -l "
done

如果想手工强制时间同步,可以在各个节点(需要手工登录各个节点)输入以下命令:

[ root @test- [ 01 ] - [ 07 ] ~ ] # ( /usr/sbin/ntpdate -u test01) \
&& ( hwclock --systohc )

然后在所有节点启动服务 ntpd 服务:

[ root @test- [ 01 ] - [ 07 ] ~ ] # service ntpd restart

在准备环境的过程中,如果遇上服务器之间时区不一致的情况,比如在本次测试准备时,test05 和 test06 和其他的节点时区不一致,使用以下办法修改:

[ root @test-05 etc ~ ] # cp /etc/localtime /etc/localtime.bak
[ root @test-05 etc ~ ] # scp root@test01:/etc/localtime /etc/localtime

第九步、配置 Ulimits 调整文件数打开限制

由于 HDFS 和 GPFS 都会在同一时间打开很多文件,所以需要配置 Ulimits 将 root 和 biadimn 用户的文件数打开限制调大,具体命令如下:

清单 18. 修改主节点 Ulimits 示例
[ root @ test01 ~ ] # cat >> /etc/security/limits.conf << EOF
biadmin hard nofile 65536
biadmin soft nofile 65536
biadmin hard nproc 65536
biadmin soft nproc 65536
root hard nofile 65536
root soft nofile 65536
root hard nproc 65536
root soft nproc 65536
EOF
[ root @ test01 ~ ] # sed -i " /pam_limits/d " /etc/pam.d/login; echo " \
session required pam_limits.so " >> /etc/pam.d/login

然后将调整后的 root 和 biadmin 打开文件限制同步到其他所有数据节点,具体命令如下:

清单 19. 集群中同步 Ulimits 示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test \
| grep -v test01 | awk ' { print $1 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
scp /etc/security/limits.conf $target:/etc/security/limits.conf 
ssh $target " sed -i \"/pam_limits/d\" /etc/pam.d/login "
ssh $target " echo \" session required pam_limits.so\ " \
>> /etc/pam.d/login "
done

第十步、设置 Linux 内核参数 /etc/sysctl.conf

打开 /etc/sysctl.conf,修改 file-max 以及网络等参数,file-max 表示能够打开的文件句柄的最大数量,文件句柄设置表示在 linux 系统中可以打开的文件数量,Sem 表示设置的信号量,以下是本次安装的参数调整示例:

清单 20. 修改主节点内核参数 /etc/sysctl.conf 示例
[ root @ test01 ~ ] # cat >> /etc/sysctl.conf << EOF
fs.file-max = 1048576
kernel.sem = 250 32000 100 128
net.core.netdev_max_backlog = 400000
net.core.optmem_max = 30000000
net.core.rmem_default = 30000000
net.core.wmem_default = 30000000
net.core.rmem_max = 30000000
net.core.wmem_max = 30000000
net.core.somaxconn = 300000
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.ip_forward = 0
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_mem = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 30000000 30000000 30000000
net.ipv4.tcp_wmem = 30000000 30000000 30000000
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_sack = 0
EOF
[ root @ test01 ~ ] # /sbin/sysctl –p

接下来将 sysctl 设置同步到集群中的其他数据节点,具体命令如下:

清单 21. 集群同步 /etc/sysctl.conf 示例
[ root @ test01 ~ ] # nodes = " ` cat /etc/hosts | grep test \
| grep -v test01 | awk ' { print $1 } ' ` "
[ root @ test01 ~ ] # echo $nodes
[ root @ test01 ~ ] # for target in $nodes ; do 
echo $target " = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = > "
scp /etc/sysctl.conf $target:/etc/sysctl.conf
ssh $target " /sbin/sysctl –p "
done

第十一步、启动 BigInsights 安装向导,进行集群安装

首先在 test01 服务器上,进入到 BigInsights 3.0 的安装包所在目录,解压安装包后进入到安装介质解压目录,通过 start.sh 命令启动安装向导,具体命令如清单 22 所示:

清单 22. 启动 BigInsights 安装向导
[ root @ test01 ~ ] # cd /root/builds/
[ root @ test01 ~ ] # tar xvf biginsights-3.0.0.0-SNAPSHOT.tar.gz
[ root @ test01 ~ ] # cd biginsights-3.0.0.0-SNAPSHOT
[ root @ test01 biginsights-3.0.0.0-SNAPSHOT ~ ] # cd \
/root/builds/biginsights-3.0.0.0-SNAPSHOT
[ root @ test01 biginsights-3.0.0.0-SNAPSHOT ~ ] # ./start.sh 
artifacts/ibm-java-sdk-6.0-12.0-linux-x86_64.tgz
Running local precheck script ...
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
BigInsights Pre-Installation Check Script v1.2.2
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Machine : test01.localdomain
Architecture : xSeries
OS : Red Hat v6.4
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
[ INFO ] ( CDYIN0017I ) Running in INSTALL_PRE_UI mode .
Verify there is no install process in the background [ OK ]
Verify install ports not in use [ OK ]
Extracting Java . . . .
Java extraction complete , using JAVA_HOME = /root/builds/biginsights-
3.0.0.0-SNAPSHOT/_jvm/ibm-java-x86_64-60
Verifying port 8300 availability
port 8300 available
Starting BigInsights Installer . . . . .
Application server is up and running . . .
BigInsights Installer started , please use a browser to access 
one of the following URL ( s ) :
 http://192.168.0.1:8300/Install
After you are finished, run the following command to stop the 
installer web server :
 start.sh shutdown

在安装向导中,点击 next 进入到 License Agreement 界面,选择同意后进入到第三个标签页“install type”,在此选择“Install infoSphere BigInsights”,需要留意的是如果你之前在安装 BigInsights 的过程中有保留响应文件,可以选中“Use the previously created response file to complete an installation .”简化后续安装,具体如图 1 所示:

图 1. BigInsights 3.0 安装向导:install type 标签页
图 1. BigInsights 3.0 安装向导:install type 标签页
图 1. BigInsights 3.0 安装向导:install type 标签页

如图 2 所示,在选择文件系统标签页,使用默认的选择“Install Hadoop Distributed File System ( HDFS ) ”,用户也可以根据需要选择安装 GPFS 文件系统,或者使用已经存在的 GPFS 文件系统,或者使用一个已经存在的共享目录空间。BigInsights 默认的安装目录为:“/opt/ibm/biginsights”,默认的日志目录为:“/var/ibm/biginsights”,如果之前安装过 BigInsights 导致安装目录已经存在的话,可以将覆盖安装选项选中。Cache directory 使用默认的“hadoop/mapred/local”,Cache directory 是 MapReduce 中间临时数据存放的目录,如果有多个目录,目录可以使用逗号隔开,Hadoop 日志文件目录和 MapReduce 系统目录都使用默认的值。需要留意的是,在本页面涉及的各种目录,会在所有节点进行创建,需要保证各个节点都可以创建这些目录。

图 2. BigInsights 3.0 安装向导:文件系统标签页
图 2. BigInsights 3.0 安装向导:文件系统标签页
图 2. BigInsights 3.0 安装向导:文件系统标签页

在“Secure shell”标签页,选择第二个选项,在 root password 出输入 root 密码,并在下方输入 BigInsights 管理员用户 biadmin 的相关信息。如果用户在安装之前已经配置完集群的 SSH,创建完 biadmin 用户并配置完 sudo,也可以选择第一个选项进行安装,具体如图 3 所示:

图 3. BigInsights 3.0 安装向导:Secure shell 标签页
图 3. BigInsights 3.0 安装向导:Secure shell 标签页
图 3. BigInsights 3.0 安装向导:Secure shell 标签页

如图 4 所示,在 Hosts 标签页,将集群中的所有节点长名都输入。需要留意的是,在添加节点时,如果你使用的是一个公网(外网),则需要将“Hosts are public”选项选中,否则你将不能把需要部署在公网主机上的服务部署到私有主机上,比如 InfoSphere BigInsights Console、HttpFS、Hive Server 和 Jaql Server 等。

图 4. BigInsights 3.0 安装向导:hosts 标签页
图 4. BigInsights 3.0 安装向导:hosts 标签页
图 4. BigInsights 3.0 安装向导:hosts 标签页

如图 5 所示,在组件 1 标签页,BigInsights console 节点使用默认的 test01 节点,SSO domain 使用 localdomain,BigInsights console 的 HTTP 端口为 8080,jmx 端口为 9180。选中安装 Jaql UDF 服务器,Jaql UDF 服务器使用 test01 节点,端口为 8200。图中的 DB2 数据库是用来存放 Hive 的元数据的,在 BigInsights 3.0 中,Hive Metastore 默认使用 DB2 数据库(不同于 BigInsights 2.1.0 使用 Derby 数据库)。DB2 节点在本次安装中选择 test01 节点,DB2 数据库端口使用默认的 50000 端口,DB2 实例所有者使用默认值“catalog”,DB2 实例所有者 UID 为 224,输入要设置的 catalog 的密码“password”。

图 5. BigInsights 3.0 安装向导:组件 1 标签页上半部分
图 5. BigInsights 3.0 安装向导:组件 1 标签页上半部分
图 5. BigInsights 3.0 安装向导:组件 1 标签页上半部分

如图 6 所示,Big SQL 1 服务器节点选择 test01 节点,Big SQL 1 是指老版本的 Big SQL,在 BigInsights 3.0 中主要用来实现对 HBase 的 SQL 访问。Big SQL 的头节点使用 test01 节点,Big SQL 工作节点将使用除了 test01 以外的所有节点,Big SQL 管理员用户为 bigsql,对应的 DB2 实例所有者用户名也为 bigsql。管理员用户 UID 使用 222,敲入要为 bigsql 设置的密码“password”。Big SQL 数据目录使用默认的 /var/ibm/biginsights/database/bigsql/data,如果用户有多个目录,则目录间可以使用逗号间隔。在 Big SQL 高级选项中,Big SQL 1 服务器端口指定为 7052,Big SQL 服务器端口 51000,Big SQL FCM 开始端口为 62000,调度器(Scheduler)服务端口为 7053,调度器管理端口为 7054,Big SQL 各个节点在各自物理服务器上可以使用的资源使用默认值 25%,实际在用的过程中,此值可以设置成 25%到 75%之间的值。

图 6. BigInsights 3.0 安装向导:组件 1 标签页下半部分
图 6. BigInsights 3.0 安装向导:组件 1 标签页下半部分
图 6. BigInsights 3.0 安装向导:组件 1 标签页下半部分

如图 7 所示,NameNode 节点使用 test01(采用非 HA 模式),其端口号为 9000,HTTP 端口为 50070,NameNode 表和日志目录为使用默认值/hadoop/hdfs/name。需要注意的是,如果你想配置 HA 模式,则整个集群至少需要 5 个节点,其中 2 个节点将固定用来提供 NameNode 服务,在这两个固定节点上不能运行除了 NameNode 服务以外的其他服务。“Configure High Availability”选择将在集群 hosts 数量大于等于 5 的时候显亮,表示可以选择此功能。Secondary NameNode 节点同样使用 test01,HTTP 端口为 50090,数据目录为/hadoop/hdfs/namesecondary。JobTracker 节点使用 test01,端口为 9001, HTTP 端口为 50030,JMX 端口为 51130。

图 7. BigInsights 3.0 安装向导:组件 2 标签页上半部分
图 7. BigInsights 3.0 安装向导:组件 2 标签页上半部分
图 7. BigInsights 3.0 安装向导:组件 2 标签页上半部分

如图 8 所示,数据节点和 TaskTracker 节点选择使用除了 NameNode 以外的所有节点,数据目录使用默认的“/hadoop/hdfs/data”,如果有多个目录,目录间要使用逗号分隔开。数据节点的端口号为 50010,IPC 端口为 50020,HTTP 端口为 50075,JMX 端口为 51110,TaskerTracker 的 HTTP 端口为 50060。HttpFS 节点选择 test01 节点,端口默认为 14000。

图 8. BigInsights 3.0 安装向导:组件 2 标签页下半部分
图 8. BigInsights 3.0 安装向导:组件 2 标签页下半部分
图 8. BigInsights 3.0 安装向导:组件 2 标签页下半部分

如图 9 所示,HBase Master 服务器节点使用 test01 节点,HBase Region 服务器节点将和数据节点一样使用除了 test01 节点以外的所有节点。Zookeeper 节点使用 test01,Oozie 节点使用 test01,Hive 和 Alert 节点也选择使用 test01。

图 9. BigInsights 3.0 安装向导:组件 3 标签页
图 9. BigInsights 3.0 安装向导:组件 3 标签页
图 9. BigInsights 3.0 安装向导:组件 3 标签页

如图 10 所示,关于 BigInsights 的认证方法,可以选择使用 flat files 方式让操作系统进行身份的验证 PAM,或者 LDAP 认证的 PAM,或者 Keberos 认证。本测试使用的是第一种方式,并且选中了安装示例组和用户。

图 10. BigInsights 3.0 安装向导:安全标签页
图 10. BigInsights 3.0 安装向导:安全标签页
图 10. BigInsights 3.0 安装向导:安全标签页

如图 11 所示,在汇总标签页可以对之前的选择的组件、节点和角色等进行确认,如果没有问题就可以单击 install 按钮进行安装了。安装过程会耗费较长时间,如果界面滚动条显示异常或不动时可以在 shell 窗口使用 tail –f 命令查看对应的 log 文件关注安装进度。

图 11. BigInsights 3.0 安装向导:汇总标签页
图 11. BigInsights 3.0 安装向导:汇总标签页
图 11. BigInsights 3.0 安装向导:汇总标签页

结束语

本文详细介绍了安装 IBM InfoSphere BigInsights 3.0 前集群环境应该如何初始化,具体包括各种安装前准备如修改 hosts 文件、配置节点间互信、关闭 IPV6 功能、关闭 selinux/防火墙/dnsmasq 等服务、检查 BigInsights 所需的监听端口是否关闭、准备 BigInsights 所需的用户、安装各种操作系统补丁包、配置节点间时间同步、修改 Ulimits 和修改 /etc/sysctl.conf 等,另外还详细介绍了 BigInsights3.0 的具体安装步骤等内容。在本系列下一部分中将重点介绍 Big SQL 3.0 环境初始化、Big SQL 建表语句和文件存储格式以及如何创建存储格式为 TEXTFILE 的 Big SQL 表等内容。


相关主题


评论

添加或订阅评论,请先登录注册

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=10
Zone=Information Management
ArticleID=980344
ArticleTitle=IBM Big SQL 3.0 新功能系列,第二部分: IBM InfoSphere BigInsights 3.0 环境搭建
publish-date=08142014