Alex Rivera | Logout

Hadoop testing using MRUnit

Asked 2011-05-25T01:51:46.790
9

I'm retrofitting a bunch of existing Hadoop unit tests that were previously run in an in-memory cluster (Using MiniMRCluster) into MRUnit. The existing test cases essentially provide input to the Map phase and then test the output from the Reduce phase.

I have three questions, and the best answer to any of them will qualify:

1) What do I lose, architecturally, by unit testing with MRUnit instead of an in-memory cluster?

2) Is it worthwhile to break the existing test cases up into Map-only tests and Reduce-only tests or not? Are there any cases where I would have to break them up?

3) Are there any testing scenarios that MRUnit is unable to cover?

Edit
Report

1 Answer

0

Take a look at MRUNIT-101, within a week or so we will have added the ability to test real outputformats

answered 2012-04-19T13:03:02.143

Your Answer