(MapReduce is a framework for computing certain kinds of distributable problems using a large number of computers)
In the post Grigorik asks "What if you could contribute to a computational (Map-Reduce) job by simply pointing your browser to a URL? Surely your social network wouldn't mind opening a background tab to help you crunch a dataset or two!Instead of focusing on high-throughput proprietary protocols and high-efficiency data planes to distribute and deliver the data, we could use battle tested solutions: HTTP and your favorite browser. It just so happens that there are more Javascript processors around the world (every browser can run it) than for any other language out there - a perfect data processing platform."
Grigorik even includes some functional ruby & javascript that actually works!
So in keeping with the concept of browser based distributed computing, I thought I'd pitch in a few ideas. For instance, why not tie in a crowdsourcing aspect. For example my elasticvapor blog does about 2,000-3,000 pageviews a day. Each of those pageviews could potentially be a series of map reduce jobs, the last script that loads on the page. My blog visitors would be completely unaware that they're actually helping run distributed computing jobs.
Another idea could be to use a Google Adwords / Adsense approach whereby instead of servering up a Google text ad on your blog or website, participating websites could serve up a distributed batch job at the same cost as an ad click-through. The costs could be managed in a similar Google adwords like interface where users could determine how much they are prepared to spend on their map/reduce jobs. Say 5 cents per 10 map/reduce jobs. After the budget has been set, these jobs are then distributed to a series of partner websites that render the jobs in parallel or in place of a Google Adwords/Adsense advertisement.
Lots of potential with this idea! Great job Ilya.
