Topic
  • 10 replies
  • Latest Post - ‏2013-01-15T11:25:18Z by YangWeiWei
SystemAdmin
SystemAdmin
603 Posts

Pinned topic installation slow and not working, maybe a timeout

‏2012-10-03T15:50:42Z |
I'm trying to install BigInsights Enterprise on a virtual machine with CentOS 6.3, 64 bit. The VM has 4 GB of RAM (6 GB on the host system) and the processor is an i5 with VT-x enabled.

The installation take a long time, about 4 hours, and at some point I get an error in the installation log file (attached):

INFO @localhost.localdomain - oozie installed
com.ibm.xap.mgmt.util.TimeoutException: Timeout when executing process, timeout = 300000

and then
ERROR DeployManager - console failed
com.ibm.xap.mgmt.DeployException: console install failed -- localhost.localdomain=-1 (com.ibm.xap.mgmt.util.TimeoutException: Timeout when executing process, timeout = 300000

and

ERROR DeployManager - sheets failed
com.ibm.xap.mgmt.DeployException: sheets install failed -- localhost.localdomain=-1 (com.ibm.xap.mgmt.ConfigurationException: Failed to install sheets

all suggesting that the problem is caused by some sort of timeout, also considering how long the installation is taking and the fact that the VM becomes unusable during the process.

How can I solve the problem?
Updated on 2013-01-15T11:25:18Z at 2013-01-15T11:25:18Z by YangWeiWei
  • SystemAdmin
    SystemAdmin
    603 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2012-10-04T18:37:22Z  
    Hi,

    In the log snippet I see reference to localhost.localdomain.
    Please correct /etc/hosts and specify actual ip (needs to be static) and hostname and try re-installing.
    For other installation pre-req's, please see this link:
    http://pic.dhe.ibm.com/infocenter/bigins/v1r4/topic/com.ibm.swg.im.infosphere.biginsights.install.doc/doc/c0057608.html

    Thank you,

    Zach
  • SystemAdmin
    SystemAdmin
    603 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2012-10-05T08:39:48Z  
    Hi,

    In the log snippet I see reference to localhost.localdomain.
    Please correct /etc/hosts and specify actual ip (needs to be static) and hostname and try re-installing.
    For other installation pre-req's, please see this link:
    http://pic.dhe.ibm.com/infocenter/bigins/v1r4/topic/com.ibm.swg.im.infosphere.biginsights.install.doc/doc/c0057608.html

    Thank you,

    Zach
    Thanks for the answer!

    The hosts file assigned 127.0.0.1 to localhost, isn't the loopback interface suitable, since I need it in local just to try it ?

    Now I've set the actual IPv4 network address obtained from ifconfig, which is assigned via DHCP every time the VM starts so I'll have to change hosts at each session.
  • SystemAdmin
    SystemAdmin
    603 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2013-01-07T15:19:16Z  
    Hi, I'm installing BigInsights 1.4 on SUSE Linux 1.4.
    package name: biginsights-enterprise-linux64_b20120604_2018
    environment detail: Single node cluster
    hardware details : 1 TB disk space
    24 GB RAM
    I created a user/group hadoop, assign SSH to this user and given sudo as well. I ran installer script through hadoop user. whenever installation reach to 58%, installer start complaining with below exception :

    ERROR @ccihdpvm.webmaster.ibm.com - java.lang.Exception
    at com.ibm.xap.mgmt.util.TaskRunner.run(TaskRunner.java:65)
    at com.ibm.xap.mgmt.hdm.HadoopMgmtFacade.execInternal(HadoopMgmtFacade.java:195)
    at com.ibm.xap.mgmt.hdm.HadoopMgmtFacade.execInternal(HadoopMgmtFacade.java:161)
    at com.ibm.xap.mgmt.hdm.HadoopMgmtFacade.format(HadoopMgmtFacade.java:401)
    at com.ibm.xap.mgmt.hdm.HadoopDeployer.installConfig(HadoopDeployer.java:238)
    at com.ibm.xap.mgmt.DeployManager$InstallThread.doInstall(DeployManager.java:2160)
    at com.ibm.xap.mgmt.DeployManager$InstallThread.work(DeployManager.java:2195)
    at com.ibm.xap.mgmt.DeployManager$WorkerThread.run(DeployManager.java:2112)

    com.ibm.xap.mgmt.util.TimeoutException: Timeout when executing process, timeout = 300000
    at com.ibm.xap.mgmt.util.Code.exec(Code.java:699)
    at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:75)
    at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:38)
    at com.ibm.xap.mgmt.hdm.ExecTask.doTask(ExecTask.java:77)
    at com.ibm.xap.mgmt.util.Task.run(Task.java:77)
    at com.ibm.xap.mgmt.util.TaskRunner$1.run(TaskRunner.java:52)

    And at the end UI blown up with this message :
    IUI0005E: The installation failed with fatal error: Failed to install BigInsights component(s).

    Installation failed on component(s): Hadoop, BigInsights orchestrator, BigInsights console, Sheets

    Verification failed on component(s): ZooKeeper, Jaql, Pig, HBase, Oozie

    I tried all possible ways but not understanding why it is failing and how to fix this.
    Please see the log I attached as install_2013-01-07T05.30.04.855_EST.log, any work around or pointer to this issues will highly appreciated.

    Thanks in advance,
  • LindaZhao
    LindaZhao
    9 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2013-01-07T19:10:51Z  
    Hi,

    By checking the installation log file, it seems file permission issue.

    WARN Deployer - Failed to create /tmp directory in HDFS, try to execute "$HADOOP_HOME/bin/hadoop dfs -mkdir /tmp" manually if it doesn't exist
    WARN Deployer - Failed to modify access authority in HDFS, try to execute "$HADOOP_HOME/bin/hadoop dfs -chmod a+w /tmp" manually
    WARN Deployer - Failed to create /biginsights/hive/warehouse directory in HDFS, try to execute "$HADOOP_HOME/bin/hadoop dfs -mkdir /biginsights/hive/warehouse" manually if it doesn't exist
    WARN Deployer - Failed to modify access authority in HDFS, try to execute "$HADOOP_HOME/bin/hadoop dfs -chmod g+w /biginsights/hive/warehouse" manually
    Please confirm with user you used to install BigInsights? Are you using root?

    Thanks,

    Linda
  • YangWeiWei
    YangWeiWei
    72 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2013-01-08T01:23:28Z  
    Hi, I'm installing BigInsights 1.4 on SUSE Linux 1.4.
    package name: biginsights-enterprise-linux64_b20120604_2018
    environment detail: Single node cluster
    hardware details : 1 TB disk space
    24 GB RAM
    I created a user/group hadoop, assign SSH to this user and given sudo as well. I ran installer script through hadoop user. whenever installation reach to 58%, installer start complaining with below exception :

    ERROR @ccihdpvm.webmaster.ibm.com - java.lang.Exception
    at com.ibm.xap.mgmt.util.TaskRunner.run(TaskRunner.java:65)
    at com.ibm.xap.mgmt.hdm.HadoopMgmtFacade.execInternal(HadoopMgmtFacade.java:195)
    at com.ibm.xap.mgmt.hdm.HadoopMgmtFacade.execInternal(HadoopMgmtFacade.java:161)
    at com.ibm.xap.mgmt.hdm.HadoopMgmtFacade.format(HadoopMgmtFacade.java:401)
    at com.ibm.xap.mgmt.hdm.HadoopDeployer.installConfig(HadoopDeployer.java:238)
    at com.ibm.xap.mgmt.DeployManager$InstallThread.doInstall(DeployManager.java:2160)
    at com.ibm.xap.mgmt.DeployManager$InstallThread.work(DeployManager.java:2195)
    at com.ibm.xap.mgmt.DeployManager$WorkerThread.run(DeployManager.java:2112)

    com.ibm.xap.mgmt.util.TimeoutException: Timeout when executing process, timeout = 300000
    at com.ibm.xap.mgmt.util.Code.exec(Code.java:699)
    at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:75)
    at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:38)
    at com.ibm.xap.mgmt.hdm.ExecTask.doTask(ExecTask.java:77)
    at com.ibm.xap.mgmt.util.Task.run(Task.java:77)
    at com.ibm.xap.mgmt.util.TaskRunner$1.run(TaskRunner.java:52)

    And at the end UI blown up with this message :
    IUI0005E: The installation failed with fatal error: Failed to install BigInsights component(s).

    Installation failed on component(s): Hadoop, BigInsights orchestrator, BigInsights console, Sheets

    Verification failed on component(s): ZooKeeper, Jaql, Pig, HBase, Oozie

    I tried all possible ways but not understanding why it is failing and how to fix this.
    Please see the log I attached as install_2013-01-07T05.30.04.855_EST.log, any work around or pointer to this issues will highly appreciated.

    Thanks in advance,
    Can you provide the logs of hadoop in /var/ibm/biginsights/hadoop/* for us to diagnosis the problem ? I think there is a connection problem between hadoop master and slaves, that's why the client kept trying connect without a response and finnally went to timeout.
  • YangWeiWei
    YangWeiWei
    72 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2013-01-08T01:31:29Z  
    Can you provide the logs of hadoop in /var/ibm/biginsights/hadoop/* for us to diagnosis the problem ? I think there is a connection problem between hadoop master and slaves, that's why the client kept trying connect without a response and finnally went to timeout.
  • YangWeiWei
    YangWeiWei
    72 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2013-01-08T01:31:29Z  
    Can you provide the logs of hadoop in /var/ibm/biginsights/hadoop/* for us to diagnosis the problem ? I think there is a connection problem between hadoop master and slaves, that's why the client kept trying connect without a response and finnally went to timeout.
  • SystemAdmin
    SystemAdmin
    603 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2013-01-08T05:14:13Z  
    • LindaZhao
    • ‏2013-01-07T19:10:51Z
    Hi,

    By checking the installation log file, it seems file permission issue.

    WARN Deployer - Failed to create /tmp directory in HDFS, try to execute "$HADOOP_HOME/bin/hadoop dfs -mkdir /tmp" manually if it doesn't exist
    WARN Deployer - Failed to modify access authority in HDFS, try to execute "$HADOOP_HOME/bin/hadoop dfs -chmod a+w /tmp" manually
    WARN Deployer - Failed to create /biginsights/hive/warehouse directory in HDFS, try to execute "$HADOOP_HOME/bin/hadoop dfs -mkdir /biginsights/hive/warehouse" manually if it doesn't exist
    WARN Deployer - Failed to modify access authority in HDFS, try to execute "$HADOOP_HOME/bin/hadoop dfs -chmod g+w /biginsights/hive/warehouse" manually
    Please confirm with user you used to install BigInsights? Are you using root?

    Thanks,

    Linda
    Yes I started the installation script start.sh with root user and in UI make hadoop as an admin user for BigInsights.
    I checked hadoop user permission, this user have sudo privileges. Is this sufficient or i need to something else? Now could you please help me to identify hadoop user have enough privileges to run BI by using some command so we'll sure that nothing need to be done in userid?

    Thanks,
    Harihar
  • SystemAdmin
    SystemAdmin
    603 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2013-01-08T09:12:37Z  
    Can you provide the logs of hadoop in /var/ibm/biginsights/hadoop/* for us to diagnosis the problem ? I think there is a connection problem between hadoop master and slaves, that's why the client kept trying connect without a response and finnally went to timeout.
    Thanks Yang for your reply.

    I looked at /var/ibm/biginsights/hadoop/logs directory there is no file created.
  • YangWeiWei
    YangWeiWei
    72 Posts

    Re: installation slow and not working, maybe a timeout

    ‏2013-01-15T11:25:18Z  
    Thanks for the answer!

    The hosts file assigned 127.0.0.1 to localhost, isn't the loopback interface suitable, since I need it in local just to try it ?

    Now I've set the actual IPv4 network address obtained from ifconfig, which is assigned via DHCP every time the VM starts so I'll have to change hosts at each session.
    Hi JacopoF

    Did not see your update on this post, is the timeout issue resolved ? How ?