9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
Hadoop Installation for Windows
Home Our Courses Blogs About Gallery Contact
In this blog you will understand how to download and install Hadoop for windows
Steps to Install Hadoop
Install Java JDK 1.8
Download Hadoop and extract and place under C drive
Set Path in Environment Variables
Config files under Hadoop directory
Create folder datanode and namenode under data directory
Edit HDFS and YARN files
Set Java Home environment in Hadoop environment
Setup Complete. Test by executing start-all.cmd
There are two ways to install Hadoop, i.e.
Single node
Multi node
Single node cluster means only one DataNode running and setting up all the NameNode, DataNode, ResourceManager and NodeManager on
a single machine.
This is used for studying and testing purposes.
So for testing whether the Oozie jobs have scheduled all the processes like collecting, aggregating, storing and processing the data in a
proper sequence, we use single node cluster.
It can easily and efficiently test the sequential workflow in a smaller environment as compared to large environments which contains
terabytes of data distributed across hundreds of machines.
While in a Multi node cluster, there are more than one DataNode running and each DataNode is running on different machines. The multi node
cluster is practically used in organizations for analyzing Big Data. In real time when we deal with petabytes of data, it needs to be distributed
across hundreds of machines to be processed. Thus, here we use multi node cluster.
https://brain-mentors.com/hadoopinstallation/ 1/10
9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
Setting up a single node Hadoop cluster
Prerequisites to install Hadoop on windows
VIRTUAL BOX (For Linux): it is used for installing the operating system on it.
OPERATING SYSTEM: You can install Hadoop on Windows or Linux based operating systems. Ubuntu and CentOS are very commonly
used.
JAVA: You need to install the Java 8 package on your system.
HADOOP: You require Hadoop latest version
1. Install Java
Home Our Courses Blogs About Gallery Contact
– Java JDK Link to download
https://www.oracle.com/java/technologies/javase-jdk8-downloads.html
– extract and install Java in C:\Java
– open cmd and type -> javac -version
2. Download Hadoop
– https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.3.0/hadoop-3.3.0.tar.gz
– extract to C:\Hadoop
3. Set the path JAVA_HOME Environment variable
4. Set the path HADOOP_HOME Environment variable
https://brain-mentors.com/hadoopinstallation/ 2/10
9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
Home Our Courses Blogs About Gallery Contact
https://brain-mentors.com/hadoopinstallation/ 3/10
9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
https://brain-mentors.com/hadoopinstallation/ 4/10
9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
https://brain-mentors.com/hadoopinstallation/ 5/10
9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
5. Configurations
Edit file C:/Hadoop-3.3.0/etc/hadoop/core-site.xml,
paste the xml code in folder and save
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
======================================================
Rename “mapred-site xml template” to “mapred-site xml” and edit this file C:/Hadoop-3 3 0/etc/hadoop/mapred-site xml paste xml code
https://brain-mentors.com/hadoopinstallation/ 6/10
9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
Rename mapred site.xml.template to mapred site.xml and edit this file C:/Hadoop 3.3.0/etc/hadoop/mapred site.xml, paste xml code
and save this file.
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
======================================================
Create folder “data” under “C:\Hadoop-3.3.0”
Create folder “datanode” under “C:\Hadoop-3.3.0\data”
Create folder “namenode” under “C:\Hadoop-3.3.0\data”
======================================================
Edit file C:\Hadoop-3.3.0/etc/hadoop/hdfs-site.xml,
paste xml code and save this file.
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/hadoop-3.3.0/data/namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/hadoop-3.3.0/data/datanode</value>
</property>
</configuration>
======================================================
Edit file C:/Hadoop-3.3.0/etc/hadoop/yarn-site.xml,
paste xml code and save this file.
<configuration>
https://brain-mentors.com/hadoopinstallation/ 7/10
9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
g
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>
======================================================
Edit file C:/Hadoop-3.3.0/etc/hadoop/hadoop-env.cmd
by closing the command line
“JAVA_HOME=%JAVA_HOME%” instead of set “JAVA_HOME=C:\Java”
======================================================
6. Hadoop Configurations
Download
https://github.com/brainmentorspvtltd/BigData_RDE/blob/master/Hadoop%20Configuration.zip
or (for hadoop 3)
https://github.com/s911415/apache-hadoop-3.1.0-winutils
– Copy folder bin and replace existing bin folder in
C:\Hadoop-3.3.0\bin
– Format the NameNode
– Open cmd and type command “hdfs namenode –format”
7. Testing
– Open cmd and change directory to C:\Hadoop-3.3.0\sbin
– type start-all.cmd
https://brain-mentors.com/hadoopinstallation/ 8/10
9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
(Or you can start like this)
– Start namenode and datanode with this command
– type start-dfs.cmd
– Start yarn through this command
– type start-yarn.cmd
Make sure these apps are running
– Hadoop Namenode
– Hadoop datanode
– YARN Resource Manager
– YARN Node Manager
https://brain-mentors.com/hadoopinstallation/ 9/10
9/23/22, 10:23 AM Hadoop Installation for Windows – Brain Mentors
Open: http://localhost:8088
Open: http://localhost:9870
======================================================
Hadoop installed Successfully…………
======================================================
Social Share Get In Touch
23, 1st & 2nd Floor
Block-C, Pocket-9
Rohini Sector-7
Delhi-110085
Oppposite to Metro Pillar No-400
https://brain-mentors.com/hadoopinstallation/ 10/10
