visit2nkant.wordpress.com

learnhadoopwithme.wordpress.com

Hadoop ≅ HDFS + MapReduce (Part – II) | Abode for Hadoop Beginners

https://learnhadoopwithme.wordpress.com/2013/08/13/hadoop-≅-hdfs-mapreduce-part-ii

Abode for Hadoop Beginners. Hadoop HDFS MapReduce (Part II). August 13, 2013. In this post, we will discuss the following with regards to MapReduce framework :-. Motivation for a parallel processing framework. How MapReduce solves a problem? Shuffle and Sort Phase. Motivation for a parallel processing framework. Is one such programming model designed for processing large volumes of data in parallel by dividing the work into a set of independent tasks. Each of these tasks are then run on individual node i...

learnhadoopwithme.wordpress.com

Unit Test MapReduce using MRUnit | Abode for Hadoop Beginners

https://learnhadoopwithme.wordpress.com/2013/09/03/unit-test-mapreduce-using-mrunit

Abode for Hadoop Beginners. Unit Test MapReduce using MRUnit. September 3, 2013. In order to make sure that your code is correct, you need to Unit test your code first. And like you unit test your Java code using JUnit testing framework, the same can be done using MRUnit to test MapReduce Jobs. I will now discuss the template that can be used for writing any unit test for MapReduce job. To Unit test MapReduce jobs:. Create a new test class to the existing project. Add the mrunit jar file to build path.

learnhadoopwithme.wordpress.com

Hello World of MapReduce – Word Count | Abode for Hadoop Beginners

https://learnhadoopwithme.wordpress.com/2013/08/20/hello-world-of-mapreduce-word-count

Abode for Hadoop Beginners. Hello World of MapReduce – Word Count. August 20, 2013. Its finally time to attempt our first MapReduce program. As with any programming language the first program you try is “Hello World”. We execute “Hello World” because it the easiest and we test whether the everything is perfectly installed and configured. The easiest problem in MapReduce is the word count problem and is therefore called MapReduce’s “Hello World” by many people. So let us dive into it. Create a new project.

learnhadoopwithme.wordpress.com

Abode for Hadoop Beginners | Page 2

https://learnhadoopwithme.wordpress.com/page/2

Abode for Hadoop Beginners. Install Hadoop in Standalone mode. August 18, 2013. We discussed in the last post. The different modes in which Hadoop can be run. Depending upon what kind of user you are and what you want to do with Hadoop, you can decide the mode in which you run Hadoop. You will want to run Hadoop in Standalone mode when you want to test and debug Hadoop programs with small input files that are stored locally (not in HDFS). Steps involved in the installation :-. Download and unpack Hadoop.

learnhadoopwithme.wordpress.com

Basic HDFS commands | Abode for Hadoop Beginners

https://learnhadoopwithme.wordpress.com/2013/08/19/basic-hdfs-commands

Abode for Hadoop Beginners. August 19, 2013. Before we move on to developing our first MapReduce program, it is essential to know few basic HDFS commands to play with. First open the Cloudera’s virtual image from the virtual box. Open the terminal type the following command:. As you can see, it gives you the list of hadoop commands and a short descrition. There is a subsystem associated with HDFS called fsShell. To invoke the shell type the following command:. List the contents of a directory. Cloudera@l...

learnhadoopwithme.wordpress.com

Run MapReduce Job in Standalone Mode | Abode for Hadoop Beginners

https://learnhadoopwithme.wordpress.com/2013/08/30/run-mapreduce-job-in-standalone-mode

Abode for Hadoop Beginners. Run MapReduce Job in Standalone Mode. August 30, 2013. In the last post. We saw how to run our first MapReduce job. If you gone through the previous post, you will remember that I had mentioned the steps that you must conform to before running your code on an actual cluster. You must,. First run you MapReduce code in Standalone Mode. It gives you the chance to put break points in your code and debug it extensively with a small input file stored locally. Select Java Application...

learnhadoopwithme.wordpress.com

Install Cloudera’s Hadoop Demo VM | Abode for Hadoop Beginners

https://learnhadoopwithme.wordpress.com/2013/08/19/install-clouderas-hadoop-demo-vm

Abode for Hadoop Beginners. Install Cloudera’s Hadoop Demo VM. August 19, 2013. Installing Clouder’s Hadoop Demo VM would be the best and easiest way to learn and start working with Hadoop. The virtual Machine is installed in Pseudo Distributed mode. It is best to test your code first in this mode before you run it in the actual cluster. The step to install Clouder’s Hadoop Demo VM using Virtual Box are as follows:-. Choose the version as Virtual Box and click on Download. In this step you need to select...

learnhadoopwithme.wordpress.com

Install Hadoop in Pseudo Distributed mode | Abode for Hadoop Beginners

https://learnhadoopwithme.wordpress.com/2013/08/19/install-hadoop-in-pseudo-distributed-mode

Abode for Hadoop Beginners. Install Hadoop in Pseudo Distributed mode. August 19, 2013. Installing Hadoop in pseudo distributed mode lets you mimic multi server cluster on a single machine. Unlike standalone mode, this mode has all the daemons running. Also the data in pseudo distributed mode is stored in HDFS rather than the local hard disk. If you have followed the last post, the first three steps of this tutorial are the same. Download and unpack Hadoop. Open System Preference Users and Groups. By doi...

learnhadoopwithme.wordpress.com

Implementing Custom Writables in Hadoop – BigramCount | Abode for Hadoop Beginners

https://learnhadoopwithme.wordpress.com/2013/11/21/implementing-custom-writables-in-hadoop-bigramcount

Abode for Hadoop Beginners. Implementing Custom Writables in Hadoop – BigramCount. November 21, 2013. Apologies for the delay in coming up with this post. I was caught up with my studies. Anyways, today we are going to see how to implement a custom Writable in Hadoop. But before we get into that, let us understand some basics and get the motivation behind implementing a custom Writable. We will discuss the following in this post:. What is a Writable in Hadoop? Why does Hadoop use Writable(s)? In the word...