hadoop commands
Command Function Additional Info
Lists files and directories in If <dir> is omitted, lists the root
hdfs dfs -ls <dir>
the specified HDFS path. / directory.
Creates a new directory in Fails if the directory already
hdfs dfs -mkdir <dir>
HDFS. exists.
Creates an empty file in Used for testing and
hdfs dfs -touchz <file>
HDFS. placeholders.
hdfs dfs -put <local_path> Uploads a file from the local Ensure the HDFS destination
<hdfs_path> filesystem to HDFS. directory exists.
hdfs dfs -get <hdfs_file> Downloads a file from HDFS Local destination must exist and
<local_path> to the local system. be writable.
Displays the contents of a Useful for quickly checking file
hdfs dfs -cat <file>
file stored in HDFS. content.
Copies a file or directory Source ( <src> ) and Destination
hdfs dfs -cp <src>
from one HDFS location to ( <dest> ) must be valid HDFS
<dest>
another. paths.
hdfs dfs -mv <src> Moves (renames) a file or Works like cp , but removes the
<dest> directory in HDFS. original file after moving.
Use carefully; deleted files
hdfs dfs -rm <file> Deletes a file from HDFS.
cannot be recovered.
Removes an empty directory Fails if the directory contains files
hdfs dfs -rmdir <dir>
from HDFS. or subdirectories.
Recursively deletes a
Use with caution; deletes
hdfs dfs -rm -r <dir> directory along with its
everything inside the directory.
contents.
Displays disk usage statistics Shows size in bytes for each file
hdfs dfs -du <path> for files and directories in and directory within the specified
HDFS. path.
Displays the total size of the Summarizes the total size instead
hdfs dfs -du -s <path>
specified file or directory. of listing individual files.
hadoop commands 1
Displays disk usage statistics Converts sizes into KB, MB, GB,
hdfs dfs -du -h <path>
in a human-readable format. etc., for better readability.
Shows the total size of a file Combines -s and -h for a
hdfs dfs -du -s -h <path> or directory in human- concise summary with easy-to-
readable format. read units.
Displays detailed disk usage
Provides additional metadata
hdfs dfs -du -v <path> information, including
about storage, if supported.
replication details.
Copies a file or directory Similar to -get , but specific to
hdfs dfs -copyToLocal
from HDFS to the local local file transfers. The local
<hdfs_path> <local_path>
filesystem. destination must be writable.
Copies a file or directory Similar to -put , but specific to
hdfs dfs -copyFromLocal
from the local filesystem to local files. The destination
<local_path> <hdfs_path>
HDFS. directory must exist.
Changes the group The group must already exist in the
hdfs dfs -chgrp
ownership of a file or system. User must be the file
<group> <path>
directory in HDFS. owner or have superuser privileges.
Recursively changes the
hdfs dfs -chgrp -R Useful when modifying permissions
group for a directory and all
<group> <path> for multiple files at once.
its contents.
hdfs dfs -chown <user> Changes the owner of a file Only the superuser can change
<path> or directory in HDFS. ownership.
hdfs dfs -chown
Changes both the owner and If only <user> is specified, the
<user>:<group>
<path> group of a file/directory. group remains unchanged.
hdfs dfs -chown -R Recursively changes the
Useful for applying changes to
<user>:<group> owner and group for a
<path>
entire directories.
directory and all its contents.
Permanently removes files Used to free up space after
hdfs dfs -expunge
from the HDFS trash. files have been deleted.
hadoop commands 2
Immediately empties the
Requires superuser or
hdfs dfs -expunge -immediate trash without waiting for the
appropriate permissions.
regular cleanup cycle.
Lists files and directories in
<local_path> should be a
hdfs dfs -ls file:///<local_path> the specified local
valid local directory path.
filesystem path.
Lists contents of "C:\\New Ensure the path is correctly
hdfs dfs -ls file:///"C:\\New
folder" on the local formatted, especially in
folder"
filesystem. Windows.
The output consists of three
Displays the count of numbers: (1) Number of
hdfs dfs -count directories, files, and directories, (2) Number of
<path> total entries in the files, (3) Total number of
specified HDFS path. entries (directories + files +
symbolic links).
Shows additional details like
Displays the quota
hdfs dfs -count -q quota, remaining quota,
and count details for a
<path> space quota, and remaining
directory in HDFS.
space quota.
Displays the count in a Converts file sizes into KB,
hdfs dfs -count -h
human-readable MB, or GB for better
<path>
format. readability.
Command Function Additional Info
hdfs dfs - Appends data to an Cannot create a new file;
appendToFile existing HDFS file destination must exist
Displays file count, Useful for checking
hdfs dfs -count
directory count, and size directory size in HDFS
hdfs dfs - Merges multiple HDFS Does not merge
getmerge files into a single local file directories; only files
Checks if a file or Returns 0 if exists, 1 if
hdfs dfs -test -e
directory exists in HDFS not
Checks if the given path Returns 0 if directory, 1
hdfs dfs -test -d
is a directory otherwise
hadoop commands 3
Checks if the given path Returns 0 if file, 1
hdfs dfs -test -f
is a file otherwise
Lists files and directories Shows permissions, size,
hdfs dfs -ls
in HDFS and modification date
Moves or renames Cannot move across
hdfs dfs -mv
files/directories in HDFS different HDFS clusters
Displays the first few Shows first 1 KB of the
hdfs dfs -head
lines of an HDFS file file content
Displays the last few lines Useful for checking
hdfs dfs -tail
of an HDFS file recent log entries
echo Prints the exit status of
0 = success, 1 = failure
%ERRORLEVEL% the last command
https://chatgpt.com/share/67d97934-ef90-8002-80f4-0410c7555df0
https://chatgpt.com/share/67d97934-ef90-8002-80f4-0410c7555df0
hadoop commands 4
