Topic
  • 10 replies
  • Latest Post - ‏2014-01-24T12:51:40Z by YDJY_Ron_Weinstein
Ratnakar Pawar
Ratnakar Pawar
2 Posts

Pinned topic biginsights installation

‏2013-10-07T15:04:55Z |

Hi ,

I am trying to install biginsights in pseudo mode , getting this error and installation fails

 

 

[ERROR] @myhost - java.lang.Exception
        at com.ibm.xap.mgmt.util.TaskRunner.run(TaskRunner.java:65)
        at com.ibm.xap.mgmt.hdm.HadoopMgmtFacade.deploy(HadoopMgmtFacade.java:389)
        at com.ibm.xap.mgmt.hdm.HadoopMgmtFacade.deploy(HadoopMgmtFacade.java:414)
        at com.ibm.xap.mgmt.hdm.HadoopDeployer.installCopy(HadoopDeployer.java:264)
        at com.ibm.xap.mgmt.DeployManager$InstallThread.doInstall(DeployManager.java:2426)
        at com.ibm.xap.mgmt.DeployManager$InstallThread.work(DeployManager.java:2478)
        at com.ibm.xap.mgmt.DeployManager$WorkerThread.run(DeployManager.java:2387)

java.lang.RuntimeException: hdm package is still not up-to-date after been deployed twice on myhost, check server status and try again
        at com.ibm.xap.mgmt.hdm.DeployTask.doTask(DeployTask.java:108)
        at com.ibm.xap.mgmt.util.Task.run(Task.java:77)
        at com.ibm.xap.mgmt.util.TaskRunner$1.run(TaskRunner.java:52)
 

can anyone help please ?

thanks

 

  • DiLi
    DiLi
    69 Posts

    Re: biginsights installation

    ‏2013-10-07T15:09:19Z  

    Hello

    could you attach the install.xml and the full install log please?

  • Ratnakar Pawar
    Ratnakar Pawar
    2 Posts

    Re: biginsights installation

    ‏2013-10-07T15:31:49Z  
    • DiLi
    • ‏2013-10-07T15:09:19Z

    Hello

    could you attach the install.xml and the full install log please?

    attached logs and install xml

  • DiLi
    DiLi
    69 Posts

    Re: biginsights installation

    ‏2013-10-07T16:13:35Z  

    attached logs and install xml

    Does user used to install the BI have sudo privileges and the root user has passwordless ssh to all nodes including the installer node itself?

    Can you try the "Cluster installation" option on the "Select an installation type" page, instead of the "Single node installation" option?

  • aaronglg
    aaronglg
    14 Posts

    Re: biginsights installation

    ‏2013-10-17T16:00:24Z  
    • DiLi
    • ‏2013-10-07T16:13:35Z

    Does user used to install the BI have sudo privileges and the root user has passwordless ssh to all nodes including the installer node itself?

    Can you try the "Cluster installation" option on the "Select an installation type" page, instead of the "Single node installation" option?

    Hello, sir

     

    I am installing the Quick Start Edition, but I still face a problem. In the log, the progress shows 100%, however, at the end of the log there is still some problems. You can see the details in the log.

    Problem coming with it:

    1.I can open the console with localhost:8080, but on this it shows that the HIVE is not running and unavailable.

    2. When I try to enable Eclipse IDE, it cannot work ,the edition of Eclipse is Eclipse IDE 3.6.1 for Java

     

    I installed with silent installation, Is my problem fatal? How can I fix it?

     

    Thank you, hope to hear from you soon. My email address is aaronglg@gmail.com

  • DiLi
    DiLi
    69 Posts

    Re: biginsights installation

    ‏2013-10-17T16:17:17Z  
    • aaronglg
    • ‏2013-10-17T16:00:24Z

    Hello, sir

     

    I am installing the Quick Start Edition, but I still face a problem. In the log, the progress shows 100%, however, at the end of the log there is still some problems. You can see the details in the log.

    Problem coming with it:

    1.I can open the console with localhost:8080, but on this it shows that the HIVE is not running and unavailable.

    2. When I try to enable Eclipse IDE, it cannot work ,the edition of Eclipse is Eclipse IDE 3.6.1 for Java

     

    I installed with silent installation, Is my problem fatal? How can I fix it?

     

    Thank you, hope to hear from you soon. My email address is aaronglg@gmail.com

    Hello

    Could you log on to the BI Console node, then su to the BigInsights Administrator user (the linux user that owns the $BIGINSIGHTS_HOME), then manually run the following from cli, see if they finish successfully?
    healthchech.sh hbase
    healthcheck.sh hadoop

    Also, please attach the install xml file used for your silent install, and the hadoop namenode and job tracker logs.

    thx

  • aaronglg
    aaronglg
    14 Posts

    Re: biginsights installation

    ‏2013-10-17T16:53:04Z  
    • DiLi
    • ‏2013-10-17T16:17:17Z

    Hello

    Could you log on to the BI Console node, then su to the BigInsights Administrator user (the linux user that owns the $BIGINSIGHTS_HOME), then manually run the following from cli, see if they finish successfully?
    healthchech.sh hbase
    healthcheck.sh hadoop

    Also, please attach the install xml file used for your silent install, and the hadoop namenode and job tracker logs.

    thx

    hello,

    Thank you for your reply. I log on the console start.sh, and then run the healthcheck.sh hadoop, it shows the following:

    [bhall@w520-cardwell bin]$ healthcheck.sh hadoop
    [INFO] DeployCmdline - [ IBM InfoSphere BigInsights QuickStart Edition ]
    [INFO] Progress - Health check hadoop
    [INFO] Deployer - Running Hadoop terasort example
    [INFO] Progress - 100%
    [ERROR] DeployManager -
    [ERROR] DeployManager - OPERATION ERROR -- Health check [hadoop]:
    [ERROR] DeployManager - -------------------------------------------------------
    [ERROR] DeployManager - hadoop failed
    java.io.IOException: exit code: 2 -- "/var/opt/ibm/builds/myBIInstall/biginsights/hdm/bin/hdm" "checkdeploy"
    [INFO] Progress - Checking Hadoop cluster started
    [INFO] DeployCmdline - [ IBM InfoSphere BigInsights QuickStart Edition ]
    [INFO] Progress - Start hadoop
    [INFO] @localhost - namenode already running, pid 3982
    [INFO] @localhost - secondarynamenode already running, pid 4231
    [INFO] @localhost - datanode already running, pid 4384
    [INFO] Progress - 50%
    [INFO] Deployer - Waiting for Namenode to exit safe mode...
    [INFO] Deployer - HDFS cluster started successfully
    [INFO] @localhost - jobtracker already running, pid 5000
    [INFO] @localhost - tasktracker already running, pid 5210
    [INFO] Progress - 100%
    [INFO] Deployer - MapReduce cluster started successfully
    [INFO] DeployManager - Start; SUCCEEDED components: [hadoop]; Consumes : 2657ms
    [INFO] Progress - Waiting for exit of safe mode
    [INFO] HadoopMgmtCmdline - Running safemode wait
    [INFO] Progress - Running terasort example
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop dfs -rmr /hdm-tera-input /hdm-tera-output /hdm-tera-report
    rmr: cannot remove /hdm-tera-input: No such file or directory.
    rmr: cannot remove /hdm-tera-output: No such file or directory.
    rmr: cannot remove /hdm-tera-report: No such file or directory.
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop jar /var/opt/ibm/builds/myBIInstall/biginsights/IHC/hadoop-examples-1.1.1.jar teragen -Dmapred.map.tasks=1 -Ddfs.block.size=536870912 -Dmapred.map.child.java.opts=-Xmx1024m -Dmapred.reduce.child.java.opts=-Xmx2048m 10000 /hdm-tera-input
    Generating 10000 using 1 maps with step of 10000
    13/10/17 12:49:08 INFO mapred.JobClient: Running job: job_201310171115_0001
    13/10/17 12:49:09 INFO mapred.JobClient:  map 0% reduce 0%
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000002_0, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000002_0:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_0&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_0&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000002_1, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000002_1:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_1&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_1&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000002_2, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000002_2:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_2&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_2&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000001_0, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000001_0:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_0&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_0&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000001_1, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000001_1:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_1&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_1&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000001_2, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000001_2:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_2&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_2&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Job complete: job_201310171115_0001
    13/10/17 12:49:09 INFO mapred.JobClient: Counters: 4
    13/10/17 12:49:09 INFO mapred.JobClient:   Job Counters
    13/10/17 12:49:09 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=0
    13/10/17 12:49:09 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
    13/10/17 12:49:09 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
    13/10/17 12:49:09 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
    13/10/17 12:49:09 INFO mapred.JobClient: Job Failed: JobCleanup Task Failure, Task: task_201310171115_0001_m_000001
    java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1327)
        at org.apache.hadoop.examples.terasort.TeraGen.run(TeraGen.java:352)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.terasort.TeraGen.main(TeraGen.java:357)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop jar /var/opt/ibm/builds/myBIInstall/biginsights/IHC/hadoop-examples-1.1.1.jar terasort -Dmapred.reduce.tasks=4 -Ddfs.block.size=536870912 -Dmapred.map.child.java.opts=-Xmx1024m -Dmapred.reduce.child.java.opts=-Xmx2048m -Dio.sort.record.percent=0.17 /hdm-tera-input /hdm-tera-output
    13/10/17 12:49:10 INFO terasort.TeraSort: starting
    13/10/17 12:49:10 INFO mapred.FileInputFormat: Total input paths to process : 0
    java.lang.ArithmeticException: divide by zero
        at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:118)
        at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:243)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:257)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop jar /var/opt/ibm/builds/myBIInstall/biginsights/IHC/hadoop-examples-1.1.1.jar teravalidate -Dmapred.map.child.java.opts=-Xmx1024m -Dmapred.reduce.child.java.opts=-Xmx2048m /hdm-tera-output /hdm-tera-report
    13/10/17 12:49:12 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost:9000/user/bhall/.staging/job_201310171115_0002
    13/10/17 12:49:12 ERROR security.UserGroupInformation: PriviledgedActionException as:bhall cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://localhost:9000/hdm-tera-output
    org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://localhost:9000/hdm-tera-output
        at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:197)
        at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
        at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:209)
        at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:1051)
        at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1043)
        at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323)
        at org.apache.hadoop.examples.terasort.TeraValidate.run(TeraValidate.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.terasort.TeraValidate.main(TeraValidate.java:153)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop dfs -ls /hdm-tera-report
    ls: Cannot access /hdm-tera-report: No such file or directory.
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop dfs -rmr /hdm-tera-input /hdm-tera-output /hdm-tera-report
    Deleted hdfs://localhost:9000/hdm-tera-input
    rmr: cannot remove /hdm-tera-output: No such file or directory.
    rmr: cannot remove /hdm-tera-report: No such file or directory.
    [INFO] =============== Summary of Hadoop Installation ===============
    [INFO] TeraSort ..................................Failed

        at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:101)
        at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:41)
        at com.ibm.xap.mgmt.hdm.HadoopDeployer.healthCheck(HadoopDeployer.java:816)
        at com.ibm.xap.mgmt.DeployManager$HealthCheckThread.work(DeployManager.java:2657)
        at com.ibm.xap.mgmt.DeployManager$WorkerThread.run(DeployManager.java:2387)
    [ERROR] DeployManager - Health check; SUCCEEDED components: []; FAILED components: [hadoop]; Consumes : 17541ms
    Error exit.

     

    Then I run the healthcheck.sh hbase, seems this is OK, it shows the following:

    [bhall@w520-cardwell bin]$  healthcheck.sh hbase
    [INFO] DeployCmdline - [ IBM InfoSphere BigInsights QuickStart Edition ]
    [INFO] Progress - Health check hbase
    [INFO] Deployer - Try to start hbase if hbase service is stopped...
    [INFO] Deployer - Double check whether hbase is started successfully...
    [INFO] @localhost - hbase-master(active) started, pid 6135
    [INFO] @localhost - hbase-regionserver started, pid 6277
    [INFO] Deployer - hbase service started
    [INFO] Deployer - hbase service is healthy
    [INFO] Progress - 100%
    [INFO] DeployManager - Health check; SUCCEEDED components: [hbase]; Consumes : 25317ms

     

    You can check my xml in the attachment.

    Thank you very much !

     

    Attachments

  • DiLi
    DiLi
    69 Posts

    Re: biginsights installation

    ‏2013-10-17T17:01:02Z  
    • aaronglg
    • ‏2013-10-17T16:53:04Z

    hello,

    Thank you for your reply. I log on the console start.sh, and then run the healthcheck.sh hadoop, it shows the following:

    [bhall@w520-cardwell bin]$ healthcheck.sh hadoop
    [INFO] DeployCmdline - [ IBM InfoSphere BigInsights QuickStart Edition ]
    [INFO] Progress - Health check hadoop
    [INFO] Deployer - Running Hadoop terasort example
    [INFO] Progress - 100%
    [ERROR] DeployManager -
    [ERROR] DeployManager - OPERATION ERROR -- Health check [hadoop]:
    [ERROR] DeployManager - -------------------------------------------------------
    [ERROR] DeployManager - hadoop failed
    java.io.IOException: exit code: 2 -- "/var/opt/ibm/builds/myBIInstall/biginsights/hdm/bin/hdm" "checkdeploy"
    [INFO] Progress - Checking Hadoop cluster started
    [INFO] DeployCmdline - [ IBM InfoSphere BigInsights QuickStart Edition ]
    [INFO] Progress - Start hadoop
    [INFO] @localhost - namenode already running, pid 3982
    [INFO] @localhost - secondarynamenode already running, pid 4231
    [INFO] @localhost - datanode already running, pid 4384
    [INFO] Progress - 50%
    [INFO] Deployer - Waiting for Namenode to exit safe mode...
    [INFO] Deployer - HDFS cluster started successfully
    [INFO] @localhost - jobtracker already running, pid 5000
    [INFO] @localhost - tasktracker already running, pid 5210
    [INFO] Progress - 100%
    [INFO] Deployer - MapReduce cluster started successfully
    [INFO] DeployManager - Start; SUCCEEDED components: [hadoop]; Consumes : 2657ms
    [INFO] Progress - Waiting for exit of safe mode
    [INFO] HadoopMgmtCmdline - Running safemode wait
    [INFO] Progress - Running terasort example
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop dfs -rmr /hdm-tera-input /hdm-tera-output /hdm-tera-report
    rmr: cannot remove /hdm-tera-input: No such file or directory.
    rmr: cannot remove /hdm-tera-output: No such file or directory.
    rmr: cannot remove /hdm-tera-report: No such file or directory.
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop jar /var/opt/ibm/builds/myBIInstall/biginsights/IHC/hadoop-examples-1.1.1.jar teragen -Dmapred.map.tasks=1 -Ddfs.block.size=536870912 -Dmapred.map.child.java.opts=-Xmx1024m -Dmapred.reduce.child.java.opts=-Xmx2048m 10000 /hdm-tera-input
    Generating 10000 using 1 maps with step of 10000
    13/10/17 12:49:08 INFO mapred.JobClient: Running job: job_201310171115_0001
    13/10/17 12:49:09 INFO mapred.JobClient:  map 0% reduce 0%
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000002_0, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000002_0:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_0&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_0&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000002_1, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000002_1:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_1&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_1&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000002_2, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000002_2:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_2&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_2&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000001_0, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000001_0:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_0&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_0&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000001_1, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000001_1:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_1&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_1&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000001_2, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000001_2:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_2&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_2&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Job complete: job_201310171115_0001
    13/10/17 12:49:09 INFO mapred.JobClient: Counters: 4
    13/10/17 12:49:09 INFO mapred.JobClient:   Job Counters
    13/10/17 12:49:09 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=0
    13/10/17 12:49:09 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
    13/10/17 12:49:09 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
    13/10/17 12:49:09 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
    13/10/17 12:49:09 INFO mapred.JobClient: Job Failed: JobCleanup Task Failure, Task: task_201310171115_0001_m_000001
    java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1327)
        at org.apache.hadoop.examples.terasort.TeraGen.run(TeraGen.java:352)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.terasort.TeraGen.main(TeraGen.java:357)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop jar /var/opt/ibm/builds/myBIInstall/biginsights/IHC/hadoop-examples-1.1.1.jar terasort -Dmapred.reduce.tasks=4 -Ddfs.block.size=536870912 -Dmapred.map.child.java.opts=-Xmx1024m -Dmapred.reduce.child.java.opts=-Xmx2048m -Dio.sort.record.percent=0.17 /hdm-tera-input /hdm-tera-output
    13/10/17 12:49:10 INFO terasort.TeraSort: starting
    13/10/17 12:49:10 INFO mapred.FileInputFormat: Total input paths to process : 0
    java.lang.ArithmeticException: divide by zero
        at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:118)
        at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:243)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:257)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop jar /var/opt/ibm/builds/myBIInstall/biginsights/IHC/hadoop-examples-1.1.1.jar teravalidate -Dmapred.map.child.java.opts=-Xmx1024m -Dmapred.reduce.child.java.opts=-Xmx2048m /hdm-tera-output /hdm-tera-report
    13/10/17 12:49:12 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost:9000/user/bhall/.staging/job_201310171115_0002
    13/10/17 12:49:12 ERROR security.UserGroupInformation: PriviledgedActionException as:bhall cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://localhost:9000/hdm-tera-output
    org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://localhost:9000/hdm-tera-output
        at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:197)
        at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
        at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:209)
        at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:1051)
        at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1043)
        at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323)
        at org.apache.hadoop.examples.terasort.TeraValidate.run(TeraValidate.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.terasort.TeraValidate.main(TeraValidate.java:153)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop dfs -ls /hdm-tera-report
    ls: Cannot access /hdm-tera-report: No such file or directory.
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop dfs -rmr /hdm-tera-input /hdm-tera-output /hdm-tera-report
    Deleted hdfs://localhost:9000/hdm-tera-input
    rmr: cannot remove /hdm-tera-output: No such file or directory.
    rmr: cannot remove /hdm-tera-report: No such file or directory.
    [INFO] =============== Summary of Hadoop Installation ===============
    [INFO] TeraSort ..................................Failed

        at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:101)
        at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:41)
        at com.ibm.xap.mgmt.hdm.HadoopDeployer.healthCheck(HadoopDeployer.java:816)
        at com.ibm.xap.mgmt.DeployManager$HealthCheckThread.work(DeployManager.java:2657)
        at com.ibm.xap.mgmt.DeployManager$WorkerThread.run(DeployManager.java:2387)
    [ERROR] DeployManager - Health check; SUCCEEDED components: []; FAILED components: [hadoop]; Consumes : 17541ms
    Error exit.

     

    Then I run the healthcheck.sh hbase, seems this is OK, it shows the following:

    [bhall@w520-cardwell bin]$  healthcheck.sh hbase
    [INFO] DeployCmdline - [ IBM InfoSphere BigInsights QuickStart Edition ]
    [INFO] Progress - Health check hbase
    [INFO] Deployer - Try to start hbase if hbase service is stopped...
    [INFO] Deployer - Double check whether hbase is started successfully...
    [INFO] @localhost - hbase-master(active) started, pid 6135
    [INFO] @localhost - hbase-regionserver started, pid 6277
    [INFO] Deployer - hbase service started
    [INFO] Deployer - hbase service is healthy
    [INFO] Progress - 100%
    [INFO] DeployManager - Health check; SUCCEEDED components: [hbase]; Consumes : 25317ms

     

    You can check my xml in the attachment.

    Thank you very much !

     

    please provide the same info to forum thread "https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014936581", if it's the same issue.

    Wei Wei has asked in that thread to check the uid.

  • aaronglg
    aaronglg
    14 Posts

    Re: biginsights installation

    ‏2013-10-18T13:40:00Z  
    • DiLi
    • ‏2013-10-17T17:01:02Z

    please provide the same info to forum thread "https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014936581", if it's the same issue.

    Wei Wei has asked in that thread to check the uid.

    Hi sir I defined a user to biadmin, with group number of 168 still cannot install the hive.

    I installed with Installation pane this time. The progress is 100% however with erros of [ERROR] DeployManager -

    [ERROR] DeployManager - hadoop failed

     

    The log is in the attachment, same problem of :Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 168 - Success.

    I entered into this file and see this:

    mapred.tasktracker.tasks.sleeptime-before-sigkill=#sleep time before sig kill is to be sent to process group after sigterm is sent. Should be in seconds
    hadoop.log.dir=/var/ibm/biginsights/hadoop/logs
    mapred.local.dir=/hadoop/mapred/local
    mapreduce.tasktracker.group=168
    min.user.id=100

    and I checked the group ID for biadmin it is like this:

    uid=503(biadmin) gid=168(biadmin) groups=168(biadmin),503(shadow)


     

    What can I do? I am installing a single node, and server is localhost, is it because of that?

    Do I need to change the group of ID into 503(shadow)?

    Thank you.

     

    Attachments

  • aaronglg
    aaronglg
    14 Posts

    Re: biginsights installation

    ‏2013-10-18T14:50:02Z  
    • DiLi
    • ‏2013-10-17T16:17:17Z

    Hello

    Could you log on to the BI Console node, then su to the BigInsights Administrator user (the linux user that owns the $BIGINSIGHTS_HOME), then manually run the following from cli, see if they finish successfully?
    healthchech.sh hbase
    healthcheck.sh hadoop

    Also, please attach the install xml file used for your silent install, and the hadoop namenode and job tracker logs.

    thx

    Hi sir Thank you I have solved the problem, but sorry I still have a problem after successful installation.

    In the console, the HIVE still cannot work:

    Hive Node:
    localhost:10000
    Hive Node Status:
    Unavailable
    Hive Node Process ID:
    88486
    Hive Web Interface:
    localhost:9999/hwi
    Hive Web Interface Status:
    Unavailable
    Hive Web Interface Process ID:
    88182
    JDBC URL:  

    Can you help me with it?

     

  • YDJY_Ron_Weinstein
    YDJY_Ron_Weinstein
    1 Post

    Re: biginsights installation

    ‏2014-01-24T12:51:40Z  
    • aaronglg
    • ‏2013-10-17T16:53:04Z

    hello,

    Thank you for your reply. I log on the console start.sh, and then run the healthcheck.sh hadoop, it shows the following:

    [bhall@w520-cardwell bin]$ healthcheck.sh hadoop
    [INFO] DeployCmdline - [ IBM InfoSphere BigInsights QuickStart Edition ]
    [INFO] Progress - Health check hadoop
    [INFO] Deployer - Running Hadoop terasort example
    [INFO] Progress - 100%
    [ERROR] DeployManager -
    [ERROR] DeployManager - OPERATION ERROR -- Health check [hadoop]:
    [ERROR] DeployManager - -------------------------------------------------------
    [ERROR] DeployManager - hadoop failed
    java.io.IOException: exit code: 2 -- "/var/opt/ibm/builds/myBIInstall/biginsights/hdm/bin/hdm" "checkdeploy"
    [INFO] Progress - Checking Hadoop cluster started
    [INFO] DeployCmdline - [ IBM InfoSphere BigInsights QuickStart Edition ]
    [INFO] Progress - Start hadoop
    [INFO] @localhost - namenode already running, pid 3982
    [INFO] @localhost - secondarynamenode already running, pid 4231
    [INFO] @localhost - datanode already running, pid 4384
    [INFO] Progress - 50%
    [INFO] Deployer - Waiting for Namenode to exit safe mode...
    [INFO] Deployer - HDFS cluster started successfully
    [INFO] @localhost - jobtracker already running, pid 5000
    [INFO] @localhost - tasktracker already running, pid 5210
    [INFO] Progress - 100%
    [INFO] Deployer - MapReduce cluster started successfully
    [INFO] DeployManager - Start; SUCCEEDED components: [hadoop]; Consumes : 2657ms
    [INFO] Progress - Waiting for exit of safe mode
    [INFO] HadoopMgmtCmdline - Running safemode wait
    [INFO] Progress - Running terasort example
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop dfs -rmr /hdm-tera-input /hdm-tera-output /hdm-tera-report
    rmr: cannot remove /hdm-tera-input: No such file or directory.
    rmr: cannot remove /hdm-tera-output: No such file or directory.
    rmr: cannot remove /hdm-tera-report: No such file or directory.
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop jar /var/opt/ibm/builds/myBIInstall/biginsights/IHC/hadoop-examples-1.1.1.jar teragen -Dmapred.map.tasks=1 -Ddfs.block.size=536870912 -Dmapred.map.child.java.opts=-Xmx1024m -Dmapred.reduce.child.java.opts=-Xmx2048m 10000 /hdm-tera-input
    Generating 10000 using 1 maps with step of 10000
    13/10/17 12:49:08 INFO mapred.JobClient: Running job: job_201310171115_0001
    13/10/17 12:49:09 INFO mapred.JobClient:  map 0% reduce 0%
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000002_0, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000002_0:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_0&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_0&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000002_1, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000002_1:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_1&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_1&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000002_2, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000002_2:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_2&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000002_2&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000001_0, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000001_0:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_0&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_0&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000001_1, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000001_1:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_1&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_1&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Task Id : attempt_201310171115_0001_m_000001_2, Status : FAILED
    Error initializing attempt_201310171115_0001_m_000001_2:
    java.io.IOException: Job initialization failed (24) with output: Reading task controller config from /var/bi-task-controller-conf/taskcontroller.cfg
    Can't get group information for 500 - Success.

        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1331)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1306)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1221)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2581)
        at java.lang.Thread.run(Thread.java:738)
    Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186)
        ... 8 more

    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_2&filter=stdout
    13/10/17 12:49:09 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201310171115_0001_m_000001_2&filter=stderr
    13/10/17 12:49:09 INFO mapred.JobClient: Job complete: job_201310171115_0001
    13/10/17 12:49:09 INFO mapred.JobClient: Counters: 4
    13/10/17 12:49:09 INFO mapred.JobClient:   Job Counters
    13/10/17 12:49:09 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=0
    13/10/17 12:49:09 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
    13/10/17 12:49:09 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
    13/10/17 12:49:09 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
    13/10/17 12:49:09 INFO mapred.JobClient: Job Failed: JobCleanup Task Failure, Task: task_201310171115_0001_m_000001
    java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1327)
        at org.apache.hadoop.examples.terasort.TeraGen.run(TeraGen.java:352)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.terasort.TeraGen.main(TeraGen.java:357)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop jar /var/opt/ibm/builds/myBIInstall/biginsights/IHC/hadoop-examples-1.1.1.jar terasort -Dmapred.reduce.tasks=4 -Ddfs.block.size=536870912 -Dmapred.map.child.java.opts=-Xmx1024m -Dmapred.reduce.child.java.opts=-Xmx2048m -Dio.sort.record.percent=0.17 /hdm-tera-input /hdm-tera-output
    13/10/17 12:49:10 INFO terasort.TeraSort: starting
    13/10/17 12:49:10 INFO mapred.FileInputFormat: Total input paths to process : 0
    java.lang.ArithmeticException: divide by zero
        at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:118)
        at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:243)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:257)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop jar /var/opt/ibm/builds/myBIInstall/biginsights/IHC/hadoop-examples-1.1.1.jar teravalidate -Dmapred.map.child.java.opts=-Xmx1024m -Dmapred.reduce.child.java.opts=-Xmx2048m /hdm-tera-output /hdm-tera-report
    13/10/17 12:49:12 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost:9000/user/bhall/.staging/job_201310171115_0002
    13/10/17 12:49:12 ERROR security.UserGroupInformation: PriviledgedActionException as:bhall cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://localhost:9000/hdm-tera-output
    org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://localhost:9000/hdm-tera-output
        at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:197)
        at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
        at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:209)
        at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:1051)
        at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1043)
        at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323)
        at org.apache.hadoop.examples.terasort.TeraValidate.run(TeraValidate.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.terasort.TeraValidate.main(TeraValidate.java:153)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop dfs -ls /hdm-tera-report
    ls: Cannot access /hdm-tera-report: No such file or directory.
    >> /var/opt/ibm/builds/myBIInstall/biginsights/IHC/bin/hadoop dfs -rmr /hdm-tera-input /hdm-tera-output /hdm-tera-report
    Deleted hdfs://localhost:9000/hdm-tera-input
    rmr: cannot remove /hdm-tera-output: No such file or directory.
    rmr: cannot remove /hdm-tera-report: No such file or directory.
    [INFO] =============== Summary of Hadoop Installation ===============
    [INFO] TeraSort ..................................Failed

        at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:101)
        at com.ibm.xap.mgmt.util.ExecUtil.exec(ExecUtil.java:41)
        at com.ibm.xap.mgmt.hdm.HadoopDeployer.healthCheck(HadoopDeployer.java:816)
        at com.ibm.xap.mgmt.DeployManager$HealthCheckThread.work(DeployManager.java:2657)
        at com.ibm.xap.mgmt.DeployManager$WorkerThread.run(DeployManager.java:2387)
    [ERROR] DeployManager - Health check; SUCCEEDED components: []; FAILED components: [hadoop]; Consumes : 17541ms
    Error exit.

     

    Then I run the healthcheck.sh hbase, seems this is OK, it shows the following:

    [bhall@w520-cardwell bin]$  healthcheck.sh hbase
    [INFO] DeployCmdline - [ IBM InfoSphere BigInsights QuickStart Edition ]
    [INFO] Progress - Health check hbase
    [INFO] Deployer - Try to start hbase if hbase service is stopped...
    [INFO] Deployer - Double check whether hbase is started successfully...
    [INFO] @localhost - hbase-master(active) started, pid 6135
    [INFO] @localhost - hbase-regionserver started, pid 6277
    [INFO] Deployer - hbase service started
    [INFO] Deployer - hbase service is healthy
    [INFO] Progress - 100%
    [INFO] DeployManager - Health check; SUCCEEDED components: [hbase]; Consumes : 25317ms

     

    You can check my xml in the attachment.

    Thank you very much !

     

    Ensure that the /var directory was not mounted with nosuid.