Alex Rivera | Logout

Can someone explain map-reduce in C#?

Asked 2011-01-20T11:45:33.647
17

Can anyone please explain the concept of map-reduce, particularly in Mongo?

I also use C# so any specifics in that area would also be useful.

Edit
Report

1 Answer

14

Here's a spot to get started with Map Reduce in Mongo. The cookbook has a few examples, I would focus on these two.

I like to think of map-reduces in the context of "data warehousing jobs" or "rollups". You're basically taking detailed data and "rolling up" a smaller version of that data.

In SQL you would normally do this with sum() and avg() and group by. In MongoDB you would do this with a Map Reduce. The basic premise of a Map Reduce is that you have two functions.

The first function (map) is a basically a giant for loop that runs over your data and "emits" certain keys and values. The second function (reduce), is a giant loop over all of the emitted data. The map says "hey this is the data you want to summarize" and the reduce says "hey this array of values reduces to this single value"

The output from a map-reduce can come in many forms (typically flat files). In MongoDB, the output is actually a new collection.

C# Specifics

In MongoDB all of the Map Reduces are performed inside of the javascript engine. So both the map & reduce function are all written in javascript. The various drivers will allow you to build the javascript and issue the command, however, this is not how I normally do it.

The preferred method for running Map Reduce jobs is to compile the JS into a file and then mongo map_reduce.js. Generally you'll do this on the server somewhere as a cron job or a scheduled task.

Why?

Well, map reduce is not a "real-time", especially with a big data set. It's really designed to be used in a batch fashion. Don't get me wrong, you can call it from

answered 2011-01-20T20:38:00.570

Your Answer