Edit Question - KnowledgeHub

Title Body

I am currently running a job I fixed the number of map task to 20 but and getting a higher number. I also set the reduce task to zero but I am still getting a number other than zero. The total time for the MapReduce job to complete is also not display. Can someone tell me what I am doing wrong. I am using this command hadoop jar Test_Parallel_for.jar Test_Parallel_for Matrix/test4.txt Result 3 \ -D mapred.map.tasks = 20 \ -D mapred.reduce.tasks =0 Output: 11/07/30 19:48:56 INFO mapred.JobClient: Job complete: job_201107291018_0164 11/07/30 19:48:56 INFO mapred.JobClient: Counters: 18 11/07/30 19:48:56 INFO mapred.JobClient: Job Counters 11/07/30 19:48:56 INFO mapred.JobClient: Launched reduce tasks=13 11/07/30 19:48:56 INFO mapred.JobClient: Rack-local map tasks=12 11/07/30 19:48:56 INFO mapred.JobClient: Launched map tasks=24 11/07/30 19:48:56 INFO mapred.JobClient: Data-local map tasks=12 11/07/30 19:48:56 INFO mapred.JobClient: FileSystemCounters 11/07/30 19:48:56 INFO mapred.JobClient: FILE_BYTES_READ=4020792636 11/07/30 19:48:56 INFO mapred.JobClient: HDFS_BYTES_READ=1556534680 11/07/30 19:48:56 INFO mapred.JobClient: FILE_BYTES_WRITTEN=6026699058 11/07/30 19:48:56 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=1928893942 11/07/30 19:48:56 INFO mapred.JobClient: Map-Reduce Framework 11/07/30 19:48:56 INFO mapred.JobClient: Reduce input groups=40000000 11/07/30 19:48:56 INFO mapred.JobClient: Combine output records=0 11/07/30 19:48:56 INFO mapred.JobClient: Map input records=40000000 11/07/30 19:48:56 INFO mapred.JobClient: Reduce shuffle bytes=1974162269 11/07/30 19:48:56 INFO mapred.JobClient: Reduce output records=40000000 11/07/30 19:48:56 INFO mapred.JobClient: Spilled Records=120000000 11/07/30 19:48:56 INFO mapred.JobClient: Map output bytes=1928893942 11/07/30 19:48:56 INFO mapred.JobClient: Combine input records=0 11/07/30 19:48:56 INFO

Tags (comma-separated) Cancel