Alex Rivera | Logout

does anyone find Cascading for Hadoop Map Reduce useful?

Asked 2010-09-10T00:20:46.940
19

I've been trying Cascading, but I cannot see any advantage over the classic map reduce approach for writing jobs.

Map Reduce jobs gives me more freedom and Cascading seems to be putting a lot of obstacles.

Might make a good job for making simple things simple, but complex things.. I find them extremely hard

Is there something I'm missing. Is there an obvious advantage of Cascading over the classic approach?

In what scenario should I chose cascading over the classic approach? Any one using it and happy?

hadoop cascading
Edit
Report

1 Answer

6

I used Cascading with Bixo to write the complete anti-spam link classification pipeline for a large social network.

The Cascading pipeline resulted in 27 MR jobs, which would have been very difficult to maintain in plain MR. I have written MR jobs before, but using something like Cascading feels like switching from Assembler to Java (insert_fav_language_here).

One of the big advantages over Hive or Pig IMHO is that Cascading is a single jar, which you bundle with your job. Pig and Hive have more dependencies (e.g. MySQL) or are not as easy to embed.

Disclaimer: While I know Chris Wensel personally, I really think Cascading is kick a**. Considering its complexity it is extremely impressive that I haven't found a single bug using it.

answered 2010-09-11T06:01:18.737

Your Answer