MAPREDUCE Tutorial: Difference between revisions
From NoSQLZoo
Created page with "==MapReduce the basics== <p>This tutorial introduces the <code>MapReduce</code> command.</p> MapReduce examples are available. <div>The MapReduce function allows to func..." |
Provided a much better target answer for Q8. |
||
(18 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
==MapReduce | ==MapReduce - The Basics== | ||
<p>This tutorial introduces the '''MapReduce''' command.</p> | |||
[[MapReduce]] examples are available. | [[MapReduce]] examples are available. | ||
<div>The MapReduce function allows to functions to be applied over a collection. The two functions map and reduce are important because: | <div>The MapReduce function allows to functions to be applied over a collection. The two functions map and reduce are important because: | ||
*Together they can solve a huge range of common database queries | *Together they can solve a huge range of common database queries. | ||
*They can be distributed effectively; you can share the work load among a number of processors and get your answer faster | *They can be distributed effectively; you can share the work load among a number of processors and get your answer faster. | ||
<div class='extra_space' style='width:1em; height:6em;'></div> | |||
It is recommended to have some familiarity with JavaScript before attempting the following questions: | |||
[[JavaScript: Fundamentals]]<br/> | |||
[https://developer.mozilla.org/bm/docs/Web/JavaScript JavaScript | MDN] | |||
<div class='extra_space' style='width:1em; height:6em;'></div></div> | |||
==Map== | ==Map== | ||
The map function can be used to filter and transform the data: | The map function can be used to filter and transform the data: | ||
<div class=q data-lang="mongo"> | <div class="q" data-lang="mongo"> | ||
<p class=strong>Use a map function to get the countries | <p class="strong">Use a map function to get the names of countries in Europe.</p> | ||
<pre class=def> | <pre class="def"><nowiki> | ||
db.world.mapReduce( | db.world.mapReduce( | ||
function(){if (this.continent=="Europe") emit(this.name,null);}, | function () { | ||
{out:{inline:1}} | if (this.continent === "Europe"){ | ||
) | emit(this.name, null); | ||
</pre> | } | ||
< | }, | ||
db.world.mapReduce( | function (k, v) { | ||
return v; | |||
}, | |||
) | {out: {inline: 1}} | ||
</ | );</nowiki></pre> | ||
<pre class="ans"><nowiki>db.world.mapReduce(function(){if(this.continent=="Europe"){emit(this.name,null)}},function(_,v){return v},{out:{inline:1}})</nowiki></pre> | |||
</div> | </div> | ||
<div class=q data-lang=" | <div class="q" data-lang="mongo"> | ||
<p class=strong>Use the previous answer to find the population of the world to the nearest million</p> | <p class="strong">Use the previous answer to find the population of the world to the nearest million.</p> | ||
<div class=hint title="How to round to the nearest million">Use the JavaScript round function : Math.round(population/1000000)*1000000 </div> | <div class="hint" title="How to round to the nearest million">Use the JavaScript round function : Math.round(population/1000000)*1000000 </div> | ||
<pre class=def> | <pre class="def"><nowiki> | ||
</pre> | db.world.mapReduce( | ||
< | function () { | ||
emit(null, null); | |||
}, | |||
function (k, v) { | |||
return null; | |||
}, | |||
{out: {inline: 1}} | |||
);</nowiki></pre> | |||
<pre class="ans"><nowiki>db.world.mapReduce(function(){emit('World Population in Millions',this.population)},function(_,values){return Math.round(Array.sum(values)/1000000)*1000000},{out:{inline:1}});</nowiki></pre> | |||
</ | |||
</div> | </div> | ||
<div class=q data-lang=" | <div class="q" data-lang="mongo"> | ||
<p class=strong> | <p class="strong">For each letter, determine how many country names begin with that letter.</p> | ||
<pre class=def> | <pre class="def"><nowiki> | ||
</pre> | db.world.mapReduce( | ||
< | function () { | ||
emit(null, null); | |||
}, | |||
function (k, v) { | |||
return null; | |||
}, | |||
{out: {inline: 1}} | |||
);</nowiki></pre> | |||
<pre class="ans"><nowiki>db.world.mapReduce(function(){emit((this.name).substring(0,1),1)},function(_,values){return Array.sum(values)},{out:{inline:1}});</nowiki></pre> | |||
</ | |||
</div> | </div> | ||
<div class=q data-lang=" | <div class="q" data-lang="mongo"> | ||
<p class=strong>Show the number of countries | <p class="strong">Show the number of countries in each continent.</p> | ||
<pre class=def> | <pre class="def"><nowiki> | ||
</pre> | db.world.mapReduce( | ||
< | function () { | ||
emit(null, null); | |||
}, | |||
function (k, v) { | |||
return null; | |||
}, | |||
{out: {inline: 1}} | |||
);</nowiki></pre> | |||
<pre class="ans"><nowiki>db.world.mapReduce(function(){emit(this.continent,1)},function(_,v){return Array.sum(v)},{out:{inline:1}})</nowiki></pre> | |||
</ | |||
</div> | </div> | ||
<div class=q data-lang=" | <div class="q" data-lang="mongo"> | ||
<p class=strong>Show the smallest 3 countries | <p class="strong">Show the name and area of the smallest 3 countries by area. Ignore records where the area is 0 or null.</p> | ||
<pre class=def> | <pre class="def"><nowiki> | ||
</pre> | db.world.mapReduce( | ||
< | function () { | ||
emit(null, null); | |||
}, | |||
function (k, v) { | |||
return null; | |||
}, | |||
{out: {inline: 1}} | |||
);</nowiki></pre> | |||
<pre class="ans"><nowiki>db.world.mapReduce(function(){emit(this.name,this.area)},function(_,v){return v},{limit:3,out:{inline:1},query:{"$and":[{"area":{"$ne":null}},{"area":{"$ne":0}}]},sort:{"area":1}});</nowiki></pre> | |||
</ | |||
</div> | </div> | ||
<div class=q data-lang=" | <div class="q" data-lang="mongo"> | ||
<p class=strong> | <p class="strong">For each continent, find the first and last country alphabetically.</p> | ||
<pre class=def> | <pre class="def"><nowiki> | ||
</pre> | db.world.mapReduce( | ||
< | function () { | ||
emit(null, null); | |||
}, | |||
function (k, v) { | |||
return null; | |||
}, | |||
{out: {inline: 1}} | |||
);</nowiki></pre> | |||
<pre class="ans"><nowiki>db.world.mapReduce(function(){emit(this.continent,{first:this.name,last:this.name})},function(_,v){let r={first:'ZZZ',last:'AAA'};for(let i=0;i<v.length;i++){if(r.first>v[i].first){r.first=v[i].first} | |||
if(r.last<v[i].last){r.last=v[i].last}} | |||
return r},{out:{inline:1}});</nowiki></pre> | |||
</div> | </div> | ||
<div class="q" data-lang="mongo"> | |||
<p class="strong">Return country names or capital cities that start with a letter 'M' as keys, use null as the value.</p> | |||
<pre class="def"><nowiki> | |||
db.world.mapReduce( | |||
function () { | |||
emit(null, null); | |||
}, | |||
function (k, v) { | |||
return null; | |||
}, | |||
{out: {inline: 1}} | |||
);</nowiki></pre> | |||
<pre class="ans"><nowiki>db.world.mapReduce(function(){if((this.name).startsWith('M')){emit(this.name,null)} | |||
if((this.capital).startsWith('M')){emit(this.capital,null)}},function(_,v){return v},{out:{inline:1}});</nowiki></pre> | |||
</div> | </div> | ||
<div class="q" data-lang="mongo"> | |||
<div class=q data-lang=" | <p class="strong">For each letter in the alphabet find the amount of cities that start with that letter. <br/> | ||
<p class=strong> | Additionally, find the two cities that come first and last alphabetically for that letter. e.g.</p> | ||
< | <syntaxhighlight lang="JavaScript"> | ||
</ | { | ||
< | "_id" : "B", | ||
"value" : { | |||
"first" : "Baghdad", | |||
"last" : "Bujumbura", | |||
"count" : 25 | |||
} | |||
}, | |||
</syntaxhighlight> | |||
<pre class="def"><nowiki> | |||
db.world.mapReduce( | |||
function () { | |||
emit(null, null); | |||
}, | |||
function (k, v) { | |||
return null; | |||
</ | }, | ||
{out: {inline: 1}} | |||
);</nowiki></pre> | |||
<pre class="ans"><nowiki>db.world.mapReduce(function(){this.capital&&emit(this.capital[0],{first:this.capital,last:this.capital,count:1})},function(t,i){return i.sort(function(t,i){return t.first<i.first?-1:1}),{first:i[0].first,last:i[i.length-1].first,count:i.length}},{out:{inline:1}});</nowiki></pre> | |||
</div> | </div> | ||
<div class=q data-lang=" | <div class="q" data-lang="mongo"> | ||
<p class=strong | <p class="strong">Show country count for countries in the following ranges:</p> | ||
0 to 1000000 | 0 to 1000000 | ||
1000000 to 2000000 | 1000000 to 2000000 | ||
Line 176: | Line 156: | ||
5000000 to 10000000 | 5000000 to 10000000 | ||
10000000 to 15000000 | 10000000 to 15000000 | ||
More than 15000000 | More than 15000000 | ||
<pre class=def> | <pre class="def"><nowiki> | ||
</pre> | db.world.mapReduce( | ||
< | function () { | ||
emit(null, null); | |||
}, | |||
function (k, v) { | |||
return null; | |||
}, | |||
{out: {inline: 1}} | |||
);</nowiki></pre> | |||
<pre class="ans"><nowiki>db.world.mapReduce(function(){var p=this.population;switch(!0){case p<1000000:emit("0 TO 1000000",1);break;case p<2000000:emit("1000000 TO 2000000",1);break;case p<3000000:emit("2000000 TO 3000000",1);break;case p<5000000:emit("3000000 TO 5000000",1);break;case p<10000000:emit("5000000 TO 10000000",1);break;case p<15000000:emit("10000000 TO 15000000",1);break | |||
case p>15000000:emit("MORE THAN 15000000",1);break}},function(_,v){return Array.sum(v)},{out:{inline:1}});</nowiki></pre> | |||
</ | |||
</div> | </div> | ||
[https://goo.gl/forms/ep8rBbCQSa381ic82 {{huge| Survey}}] <br/> | |||
Do you have thoughts about this website that you would like to share? Help improve NoSQLZoo! |
Latest revision as of 20:19, 6 February 2019
MapReduce - The Basics
This tutorial introduces the MapReduce command.
MapReduce examples are available.
- Together they can solve a huge range of common database queries.
- They can be distributed effectively; you can share the work load among a number of processors and get your answer faster.
It is recommended to have some familiarity with JavaScript before attempting the following questions:
JavaScript: Fundamentals
JavaScript | MDN
Map
The map function can be used to filter and transform the data:
Use a map function to get the names of countries in Europe.
db.world.mapReduce( function () { if (this.continent === "Europe"){ emit(this.name, null); } }, function (k, v) { return v; }, {out: {inline: 1}} );
db.world.mapReduce(function(){if(this.continent=="Europe"){emit(this.name,null)}},function(_,v){return v},{out:{inline:1}})
Use the previous answer to find the population of the world to the nearest million.
db.world.mapReduce( function () { emit(null, null); }, function (k, v) { return null; }, {out: {inline: 1}} );
db.world.mapReduce(function(){emit('World Population in Millions',this.population)},function(_,values){return Math.round(Array.sum(values)/1000000)*1000000},{out:{inline:1}});
For each letter, determine how many country names begin with that letter.
db.world.mapReduce( function () { emit(null, null); }, function (k, v) { return null; }, {out: {inline: 1}} );
db.world.mapReduce(function(){emit((this.name).substring(0,1),1)},function(_,values){return Array.sum(values)},{out:{inline:1}});
Show the number of countries in each continent.
db.world.mapReduce( function () { emit(null, null); }, function (k, v) { return null; }, {out: {inline: 1}} );
db.world.mapReduce(function(){emit(this.continent,1)},function(_,v){return Array.sum(v)},{out:{inline:1}})
Show the name and area of the smallest 3 countries by area. Ignore records where the area is 0 or null.
db.world.mapReduce( function () { emit(null, null); }, function (k, v) { return null; }, {out: {inline: 1}} );
db.world.mapReduce(function(){emit(this.name,this.area)},function(_,v){return v},{limit:3,out:{inline:1},query:{"$and":[{"area":{"$ne":null}},{"area":{"$ne":0}}]},sort:{"area":1}});
For each continent, find the first and last country alphabetically.
db.world.mapReduce( function () { emit(null, null); }, function (k, v) { return null; }, {out: {inline: 1}} );
db.world.mapReduce(function(){emit(this.continent,{first:this.name,last:this.name})},function(_,v){let r={first:'ZZZ',last:'AAA'};for(let i=0;i<v.length;i++){if(r.first>v[i].first){r.first=v[i].first} if(r.last<v[i].last){r.last=v[i].last}} return r},{out:{inline:1}});
Return country names or capital cities that start with a letter 'M' as keys, use null as the value.
db.world.mapReduce( function () { emit(null, null); }, function (k, v) { return null; }, {out: {inline: 1}} );
db.world.mapReduce(function(){if((this.name).startsWith('M')){emit(this.name,null)} if((this.capital).startsWith('M')){emit(this.capital,null)}},function(_,v){return v},{out:{inline:1}});
For each letter in the alphabet find the amount of cities that start with that letter.
Additionally, find the two cities that come first and last alphabetically for that letter. e.g.
{
"_id" : "B",
"value" : {
"first" : "Baghdad",
"last" : "Bujumbura",
"count" : 25
}
},
db.world.mapReduce( function () { emit(null, null); }, function (k, v) { return null; }, {out: {inline: 1}} );
db.world.mapReduce(function(){this.capital&&emit(this.capital[0],{first:this.capital,last:this.capital,count:1})},function(t,i){return i.sort(function(t,i){return t.first<i.first?-1:1}),{first:i[0].first,last:i[i.length-1].first,count:i.length}},{out:{inline:1}});
Show country count for countries in the following ranges:
0 to 1000000 1000000 to 2000000 2000000 to 3000000 3000000 to 5000000 5000000 to 10000000 10000000 to 15000000 More than 15000000
db.world.mapReduce( function () { emit(null, null); }, function (k, v) { return null; }, {out: {inline: 1}} );
db.world.mapReduce(function(){var p=this.population;switch(!0){case p<1000000:emit("0 TO 1000000",1);break;case p<2000000:emit("1000000 TO 2000000",1);break;case p<3000000:emit("2000000 TO 3000000",1);break;case p<5000000:emit("3000000 TO 5000000",1);break;case p<10000000:emit("5000000 TO 10000000",1);break;case p<15000000:emit("10000000 TO 15000000",1);break case p>15000000:emit("MORE THAN 15000000",1);break}},function(_,v){return Array.sum(v)},{out:{inline:1}});
Survey
Do you have thoughts about this website that you would like to share? Help improve NoSQLZoo!