I am performing a performance benchmarking on 4 node IBM Insights 2.0 cluster. The comparision is :
Running teraSort on 50 GB (approx) without Adaptive Mapreduce and
Running teraSort on 50 GB (approx) with Adaptive MapReduce
But the results were not what we expected.
The benchmarking is as follows :
TeraSort 46.6 GB 1 Hr 29 min 22 sec 746(Mappers) 6(Reducers) 3 Data Nodes
TeraSort 46.6 GB 3 Hr 37 min 42 sec 72(Adaptive mapReduce on) 6(Reducers) 3 Data Nodes
So My query is :
1) Should we use Adaptive MapReduce in specfic scenario.
2) Or I didnt follow the right way(Didn't config properly)
I changed the setting in the mapred-site.xml.