Scribd
Upload
29 views1 page

Ex 1

This document outlines the steps to install Hadoop 1.2.1, including downloading the tar.gz file, extracting it, and creating a symbolic link. It also details setting up a local user named 'hadoop', configuring environment variables, and performing basic Hadoop file operations such as creating directories and uploading files. Finally, it describes running a word count program on an uploaded text file.

Uploaded by

asda
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
29 views1 page

Ex 1

This document outlines the steps to install Hadoop 1.2.1, including downloading the tar.gz file, extracting it, and creating a symbolic link. It also details setting up a local user named 'hadoop', configuring environment variables, and performing basic Hadoop file operations such as creating directories and uploading files. Finally, it describes running a word count program on an uploaded text file.

Uploaded by

asda
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 1

ex1 - part1

* 1. Download hadoop 1.2.1 tar.gz


open a terminal
sudo su
cd /home/hadoop/tools
wget --no-check-certificate https://archive.apache.org
/dist/hadoop/core/hadoop-1.2.1/hadoop-1.2.1-bin.tar.gz

* 3. extract the file


cd /usr/local
sudo tar -xvzf /home/hadoop/tools/hadoop-1.2.1-bin.tar.gz

* 4. Add a link to /opt/hadoop


sudo rm /opt/hadoop
sudo ln -s /usr/local/hadoop-1.2.1 /opt/hadoop
sudo chown -R hadoop:hadoop /usr/local/hadoop-1.2.1
ls -al /opt
it should show hadoop -> /usr/local/hadoop-1.2.1

5. Create a local user named hadoop


Change ownership of /usr/local/hadoop... to the hadoop user
* switch to the hadoop user in the shell
su - hadoop
had00p

6. export HADOOP_HOME=/opt/hadoop
ls -al
export JAVA_HOME=<path to your jdk home>
ls -al /usr/bin/java
ls -al /etc/alternatives/java
-> ... amd64/jre/bin/java -> JAVA_HOME=..amd64/
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64

* 8. hadoop fs -ls
* 9. hadoop fs -mkdir user
* 10. hadoop fs -mkdir user/hadoop
11. edit a file -> say /home/hadoop/test1 -> and add some lines to it
12. Upload it to the hdfs
hadoop fs -put test1 user/hadoop
hadoop fs -ls
hadoop fs -cat user/hadoop/test1
13. ls /opt/hadoop
-> hadoop-examples-1.2.1.jar
cd (same as cd /home/hadoop)
a. Create a text file with multiple repeating words
Hello world tcs world is a nice world
b. Upload that file into the input folder using
hadoop fs -mkdir input
hadoop fs -put test2 input
c. Run the word count program on that input folder
(next two lines on same line)
hadoop jar /opt/hadoop/hadoop-examples-1.2.1.jar
wordcount input/test2 output

You might also like