Topic
  • 2 replies
  • Latest Post - ‏2014-08-22T01:14:09Z by Bart Guisset
David Adler
David Adler
1 Post

Pinned topic App hang in BigInsights Quick Start Edition VM

‏2014-06-10T13:38:33Z |

BigInsights seems to start correctly but running any application (WordCount, Distributed File Copy) hang in 'map' task with no indication of what is going on. The application status shows that the map task is 'pending' after successful setup task.

There are no failure indications.

Is there some place to look to see what the problem is? If the input or output specifications are incorrect, should there be some failure indication?

This is very frustrating for a newbie getting started with the tutorials.

 

  • dvillegas
    dvillegas
    18 Posts

    Re: App hang in BigInsights Quick Start Edition VM

    ‏2014-06-18T16:49:06Z  

    Hi David,

    if there is any error in the task, you should be able to see it in the task log. One way to access it is through the JobTracker UI at http://[job_tracker_hostname]:50030

    Open that URL in your browser, and find the job that is not finishing, then click on its name and look for the tasks that are pending until you see the task attempt Id. Then look for the task log and click on the "ALL" link. That should have some information if there are issues with the task itself.

    Another possibility is that your cluster does not have enough slots to run tasks. That would keep the tasks in pending state without more indication. The way to find out if this is the reason is to also open the JobTracker UI and look for the values of Map Task Capacity and Reduce Task Capacity under the Cluster summary heading. You should have a value that is higher than 1 for the first option. Having a value of 1 would make Oozie jobs (and therefore also BI apps) hang indefinitely, since Oozie requires at least one slot for the launcher task and at least another for the execution task.

    If that's the case, I'd increase the number of slots in $BIGINSIGHTS_HOME/hdm/hadoop-conf-staging/mapred-site.xml by editing that file and setting the values of mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum to at least 2, then run syncconf.sh hadoop and finally restart the services (stop.sh / start.sh).

    I hope that helps.

    David.
     

  • Bart Guisset
    Bart Guisset
    2 Posts

    Re: App hang in BigInsights Quick Start Edition VM

    ‏2014-08-22T01:14:09Z  
    • dvillegas
    • ‏2014-06-18T16:49:06Z

    Hi David,

    if there is any error in the task, you should be able to see it in the task log. One way to access it is through the JobTracker UI at http://[job_tracker_hostname]:50030

    Open that URL in your browser, and find the job that is not finishing, then click on its name and look for the tasks that are pending until you see the task attempt Id. Then look for the task log and click on the "ALL" link. That should have some information if there are issues with the task itself.

    Another possibility is that your cluster does not have enough slots to run tasks. That would keep the tasks in pending state without more indication. The way to find out if this is the reason is to also open the JobTracker UI and look for the values of Map Task Capacity and Reduce Task Capacity under the Cluster summary heading. You should have a value that is higher than 1 for the first option. Having a value of 1 would make Oozie jobs (and therefore also BI apps) hang indefinitely, since Oozie requires at least one slot for the launcher task and at least another for the execution task.

    If that's the case, I'd increase the number of slots in $BIGINSIGHTS_HOME/hdm/hadoop-conf-staging/mapred-site.xml by editing that file and setting the values of mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum to at least 2, then run syncconf.sh hadoop and finally restart the services (stop.sh / start.sh).

    I hope that helps.

    David.
     

    Problem is the Scheduled applications that take up all the available slots: goto Application tab and disable the schedule for the applications: ClusterAggregation and Downsampling, also verify if there are others. Once you do this you can run all manual jobs started. Next time you can continue to start the BigInsight via the desktop icon as in the lab description.