I NVENTIVE CONFIDENTIAL

Introduction to Unix

Apoorva Mathur
Topics To Cover

• What is Unix
• The Unix File System
• Unix Security
• Process Control and Pipes
• The Unix Toolbox
• Shell Programming
• Makefiles
• More Information
What is Unix?

• A very powerful Operating System

• Very good security features
• A host of tools/utilities to give power to the user
• Process control and Multitasking
• Robust and Scalable
What is Unix?

• Written by Ken Thompson in 1969 in Bell Labs

• Rewritten by Dennis Richie in 1973 in C
• Later millions of utilities were added by Univ students.
The Unix File System

/
usr
grid hm

amathur arvindr abindal bin

common

pkgs
The Unix File System

• Unix has a tree type directory structure

• The directory at the top is called root ( / )
• A file can be addressed by its path
• An absolute path is the path relative to the root directory
– eg /hm/kanwar/work
• to go to a particular place do this :
cd <absolute path >
cd /hm/kanwar/work
The Unix File System

• The “present working directory” is the place where you

are working currently.
• Do find out what your pwd is you can use the pwd
command
• The relative path of a file is the path relative to the pwd
• At all times the . refers to your pwd
• So if you are at /home/kanwar directory and want to read
the README file then you can do :
cat ./test/README ( relative path )
cat /home/kanwar/test/README ( absolute path )
The Unix File System

• The .. directory at all times refers to a directory one level

above the cwd /pwd
cd .. will take u one level above

• To find the files that exist at a particular place use the ls

command.
ls will show you the files that exist
ls -al will give u complete info about
files,links and the permissions
and date last modified
The Unix File System

• the ls command if given without any arguments refers

to the files and dirs in the cwd. To list files in another
directory do :
ls -al <absolute path>
ls -la <relative path >
• Here is what the ls -la output will show you :
drwxr-xr-x 4 kanwar cadence 512 Jul 18 08:26 ./
drwxr-xr-x 7 kanwar cadence 512 May 31 10:17 ../
-rw-r--r-- 1 kanwar cadence 611 May 31 10:17 Makefile
drwxr-xr-x 2 kanwar cadence 512 May 31 10:17 test/

permissions owner group modified date

name of file
bytes
The Unix File System

• You can link a file to another file

ln -s <path> <name of link>
• pushd and popd are commands that help you go to a
place and come back too
pushd /hm/kanwar/
popd
• In cadence people’s homes are referred to by
/hm/<login>
or ~<login>
The Unix File System

• STDIN , STDOUT and STDERR are places where the

default input comes from or output goes .
• By default :
– STDIN is screen
– STDOUT and STDERR are screen
• You can redirect the output
– cat filename > newfile ( overwrite mode)
– cat filename >> newfile ( append mode )
– cat filename >& errfile ( redirecting the STDERR )
The Unix File System

• Whenever you execute a file UNIX looks at the path

for the presence of the file and executes it.
which perl
• The first file that is found by the name is executed .
set path = (/grid/common/pkgs/perl/latest/bin $path )
echo $path
• The set command can be used to set the value of a
shell variable . The echo command to view it
set var = 5
echo $var
The Unix File System

• Shell variables are not passed to the programs executed

from the shell .
• Environment variables are passed .
setenv LM_LICENSE_FILE 5280@noidalic01

• For a project or product. save your paths and env

variables are in a file.
• To source a file explicitly do
source filename
• Everytime a shell is opened .cshrc file is automatically
sourced.
The Unix File System

• To remove a file use the rm command

rm filename
• To remove a directory use

\rm -rf directory

( recursively removes everything)
The Unix File System

• To copy a file :
cp /hm/kanwar/tmp/haha . ( copies haha in cwd )
cp /hm/kanwar/tmp/haha ./hoho ( copy haha as hoho in
cwd)
The Unix Security

• The Unix Security system is based on every file having a

owner and a set of permissions .
• To view the information for a particular file do a ls -la
ls -la filename
-rw-r--r-- 1 kanwar cadence 611 May 31 10:17 Makefile

owner name

owner grp others

The Unix Security

Permission Meaning Numeric Value

r read 4
w write 2
x execute 1

• Suppose you want to give a file rwx permissions for the

owner , rx permissions for group and nothing for others
• Then the codes would be 7(r + w +x = 4 + 2 + 1 ) , 5 ( 4
+ 1 ) and 0 .
chmod 750 filename
The Unix Security

• To change the permissions of a dir recursively do

chmod -R dirname
• To log in as someone else do
%su username (don’t try this)
%Password:
%whoami
Process Control and Pipes

• Unix is a multiprogramming , multitasking Operating

System .
• When a program is executed a set of processes starts
with their own unique process Ids.
• A process ID ( pid ) is a number by which you can refer
to a process .
• A process A may spawn another process B . B is said to
be the child process of A which is said to be the parent
process.
Process Control and Pipes

• Whenever you type a command the shell spawns a

process to execute the command .
• Usually the process started is the child process of the
shell
• If the shell has to wait for the process to finish to execute
another one then the process is said to be a foreground
process .
• If the shell is free to do other tasks while the child is
being executed then the process is said to be a
background process.
Process Control and Pipes

• To start the execution of a command in background do :

commandname &
• To make a foreground command background first stop it
by doing a CTRL-Z and then type the bg command .
• To see what processes have been run from the current
shell use the ps command :
ps
PID TTY TIME CMD
6619 pts/6 0:49 projmgr
3782 pts/6 0:01 csh
Process Control and Pipes

• To see all the processes running on the machine do

ps -ef
• To kill a process use the kill command :
kill -9 <process ID>
eg kill -9 6619 ( will kill projmgr. cf prev slide )

• the kill command sends a signal to a process . the signal

9 is a kill signal . There are others too.
Process Control and Pipes

• To see the processes that are consuming the max cpu

time run the top command.
• It shows you the top ten commands with the one that is
consuming the most resources at the top.
• To check the amount of load on a machine do :
rup machinename
eg
rup sitara
sitara up 9 days ,18:03 load average: 0.75,0.54,0.29
Process Control and Pipes

• After the completion of any command the $status is set .

• This env variable tells you if the command has been
successful or not .
• if $status is set to 0 then the command has been
successful.
• If anything else then the command has failed.
Process Control and Pipes

• Unix has a philosophy that the output of any program

should be usable as the input of another program.
• A pipe ( | ) can be used to connect a set of programs
together to use each other’s output streams.
• The pipe can be as long as you want( or you will ever
need)
cat output.txt | more
cat output.txt | grep ‘date’
cat output.txt | grep ‘date’ | wc -l > haha.txt
Process Control and Pipes

• If you want a T junction instead of a straight pipe .

Flow -> process 2

process 1

who | tee -a save.txt | wc -l

Basic vi

• vi is a very powerful editor .

• Available with all Unix systems.
• helps you quickly search , replace text .
• regular expression support .
• Can define macros .
• A favourite with most Unix lovers.
Basic vi

• To open a file for editing do :

vi filename
• You will see a screen like :

haha
~
~
~
Basic vi

• vi is a modal editor ie it has different modes for different

things.
• text entry mode : This is to add , change , append text.
• command mode : to perform operations like cursor
movement , screen scrolling , search and switching to
opter modes.
• pressing a <escape> key will always get you into the
command mode .
Basic vi

• To go to the insert text mode press i in the command

mode and you will go into the insert mode and then type
stuff.
• To to to the append mode press a when in the append
mode and you will go into the append mode . The
append mode will start one char to the right of the curr
cursor position.
• To open a line below the current line for writing press the
o key when in command mode .
Basic vi

• To go to different places in the file use the arrow keys in

the command mode.
• to delete a character press x .
• To delete a line press dd
• to go to beginning of a line press ^
• to go to end of line press $
• to go to a particular line no do
esc :
you will come to the bottom of the screen . type the line
no
and press enter.
Basic vi

• To search for a word after the curr line do

esc /
and type the word and press enter .
to search again press n
• To search for a word before the curr line do
esc ?
and type the word and press enter .
• To see the lines in the file do esc: and type set nu
Basic vi

• To save the file do esc : w and press enter.

• To exit from file do esc : q
• to exit without saving esc : q!
• to save and exit esc : wq
• to write to a different file esc : w <filename>
Comparing Files

• diff file1 file2

– displays line-by-line differences between pairs of text files.
– diff -r dir1 dir2 recursively diffs common subdirectories.

• cmp file1 file2

– writes the line number at which the first difference occurs.
ftp (file transfer program)

• What? User interface to the internet standard File

Transfer Protocol.
• Why? To download files across the network, eg.
testcases for the pcrs filed by customers.
• ftp <host_name>
– mget|get <file_name> to download a file with the name
file_name.
– cd <remote_dir> to change working dir on host to
remote_dir.
– put <file_name> to upload a file on the network.
– help to list all available commands
– quit/bye to terminate the FTP session and exit ftp.
• As an alternate, lookup rcp and rdist
 grep
• What? a unix command to search a file for a pattern
• Why?
– to remove unnecessary lines before diff
– to count all comment lines in a program
– to list the processes being run by you (ps -ef |grep `whoami`)
– and so on..
• grep [-civw] <reg_ex> [filename…]
where reg_ex is the regular expression for the pattern to be matched.
– -v prints all lines EXCEPT the ones containing the pattern.
– -i ignores case distinction during comparisons.
– -c prints only a count of lines that contain the pattern.
– -w searches for the expression as a word.
eg. “grep Creat cref.log” will match all lines containing Creating
but grep -w Creat cref.log will not.
 sort

• To sort, merge or sequence check text files.

• sort [-bcfimu] [-o output_file] [file …]
– -c sorts the input file.
– -m merges the files, assumed to be pre-sorted.
– -u for unique sorting, suppresses all but one in each set of
lines having equal keys.
– -o specifies the name of the output file to be used.
– -f folds lower-case to upper-case.
– -i ignores non-printable chars.
– -b ignores leading blanks.
 head/tail

• head • tail
• prints first few lines of files • prints the last part of a
• head [-n] [file1] [file2] ... file
n = no. of lines • tail [+|- n] [file]
• By default, n = 10 + starts wrt beginning of file.
- starts relative to end of
file.
• default is -10.
 find

• Why? to find files

• How? find path … expression
– eg. find . -name sch_1 will list the paths where sch_1 is present.
– To do partial matching, do
• find . -name “sch*”
• or, find . -name “sch??”
 set/setenv

• set • setenv
– displays the values of all – displays values of all
shell variables. environment variables.
• set var • setenv var
– assigns null to the variable – assigns null to var.
var. • setenv var value
• set var = value – assigns value to var.
– assigns value to var. where value may be
• a single word/quoted string.
where value may be
• a colon-separated list of words.
• a single word/a quoted string.
• a space-separated list of words enclosed
in parentheses.
 Disk Usage
• df [-k] [file] (eg. df -k .)
– -k flag gives the following info:
• name of the file system.
• total space allocated to the file system in kilobytes.
• space allocated to existing files.
• space available for creation of new files.
• percentage of normally available space that is currently
allocated to the files on the file system.
• du [-s] [-k|h] [file …] (eg. du -sh .)
• gives the total space allocated to the file hierarchy rooted in the
specified file; and also the space allocated to each of its
subdirectory.
• -k|h gives the size in units of 1kB/MB rather than the default
512 byte units.
• -s reports only the total space for the specified file/directory.
Compression utilities

• tar
– ‘tar -cvf <archive_name> *’ Creates an archive of all files in the
current dir.
– ‘tar -tvf <archive_name>’ lists all files present in the archive.
– ‘tar -xvf <archive_name>’ extracts the files from the archive.
• compress <file_name>
– creates a compressed file <file_name.Z>.
• gzip -d <file_name>.Z | uncompress <file_name>.Z
– extracts file_name from the compressed archive.
wc

• word count - displays a count of lines, words and

characters in a file.
• wc [-l] [file …]
• -l counts the number of lines in the specified file(s).
• e.g. to count the total no. of distinct words in a text file,
do
– cat <input_file> | tr -r “ ” “\n” | sort -u | wc -l
I NVENTIVE CONFIDENTIAL

C Shell Scripting
Agenda

• Basics
• Arguments
• Expressions and Operators
• Conditional Statements
• Loops( foreach , while , switch)
• Input data from Terminal
Basics

• A language to automate shell commands.

• Puts a list of shell commands in a file and ties them
together by loops/constructs.
• An interpreted language.
• Lets you use powerful Unix tools naturally.
Basics

#!/usr/bin/csh
echo “ I am a shell script”
# This completes my first shell script

• The #! in the first line tells the shell what interpreter to

use.
• The # in any other line is a comment
Basics

• To run the shell script exit the editor you typed it in

• Make the script executable :
chmod 755 scriptname
• And then just run it
scriptname
csh scriptname ( more
explicitly)
Arguments

• You can pass arguments from the shell to the script.

• The script stores these arguments in a array $argv
• $argv[1] is the first argument. Also called $1
• $argv[2] is the second argument. Also called $2 .
• $0 refers to the name of the script.
• $#argv gives you the no of arguments to the script
 Arguments( an example )

#!/usr/bin/csh
if($#argv == 0 ) then
echo “ need atleast one argument”
exit 1
endif

cat $argv[1]
exit 1
 Arguments( an example )

• This script expects one argument and can be run like:

script.csh inputfile
• If you don’t give any argument it gives an error message
and exits with status 1 .
•
 Expressions and Operators

• Use the set command for string assignments :

set var = “haha”
echo $var

set date = `date`

echo $date
 Expressions and Operators

• Use the @ notation for numerical assignments :

@ counter = 0
@ counter = $counter + 1

• The following are valid arithmetic operators. Start with

@ count = 6
symbol example meaning echo $count
= @ count = 0 assignment 0
+= @ count += 5 @ count = $count + 5 11
++ @ count++ @ count = $count + 1 7
 Expressions and Operators

• You can also make an array kind of thing :

@ var_name[index] = expression.

• This lets you index into a wordlist variable and replace

contents with numeric data
%set d = `date`
% echo $d (output is Tue Jul 25 IST 2000)
% @ d[1] = 2
% @ d[2] = 7
% echo $d ( output is 2 7 25 IST 2000)
 Expressions and Operators

• Shell scripts support the “normal” operators

+ , - , / , %( modulo )

• Logical operators :
@ r = ( $num > 8 || $num < 4 ) ( $r is 1 if $num is 10)
@ r = ( $num > 8 && $num > 11) ( $r is 0 if $num is 10)
@ r = ! $num ( $r is 0 if $num is 10)
 Expressions and Operators

• Comparison Operators
==
!=
>
>=
<=
!=
• String operators are == and !=
 Expressions and Operators

• File Inquiry Operators

• We can check various characteristics of a file usinf these
• operator what it does
r read access
w write access
x execute access
e exists
z zero length
f plain file
d directory
 Expressions and Operators

• for example to check if a file exists do

#!/usr/bin/csh -f

if ( -e “.cshrc”) then
echo “ yeah my .cshrc exists “

end if
Conditional Statements

• Forms of the Cshell if construct

if(expression) then
commands
end if

if ( expression ) then
commands
else
commands
end if
Conditional Statements

if( expression) then

commands
else if ( expression) then
commands
else
commands
end if
Conditional Statements

set names = (`who | wc -l `)

if( $names > 5 ) then

echo “ More than 5 shells open”
else
echo “ less than 5 shells open “
end if
Loops

• foreach loop allows the script to iterate over a set of

items placed in a list

foreach name ( wordlist )

commands
end
Loops

#!/usr/bin/csh -f

foreach person ( kapil prabs sidhu )

echo “ $person ke ghar IT raid kar do “
end
Loops

#!/usr/bin/csh -f

set testcases = `cat testlist`

foreach test ( $testcases)
cd $test
test.csh
cd ..
end
Loops

• The while loop

while ( expression )
commands
end

• The shift statement allows you to refer to all

arguments by remove elements from a list
Loops

#!/usr/bin/csh -f
while($#argv)
if ( -d $argv[1]) then
echo “ $argv[1] is a directory”
end if
shift
end
Loops

• break statement causes execution to resume after

the end statement of the innermost loop
foreach n (numberlist)
if ( $n == 100 ) break
end
echo “ Got it “
Getting Input from the User

• set line = $<

• The control stops here and waits for user input
followed by <ENTER>
make

The make utility automatically determines which pieces of

a large program need to be rerun/recompiled, and issues
commands to rerun/recompile them

Source: GNU Make documentation

(http://www.gnu.org/software/make/manual/make.html)
Getting started with make

• Create a file, called Makefile or makefile, with macros,

rules, targets and commands
• Run make on shell prompt
• The make program uses the makefile data base and the
last-modification times of the files to decide which of the
files need to be updated. For each of those files, it
issues the recipes recorded in the data base
Makefile rules

target … : prerequisites …
recipe
…
…

• Target is usually name of a generated file or

program.
• Prerequisites is input for generating the target
file.
• Recipe is a set of command(s) to generate the
target
Sample Makefile

edit : main.o kbd.o command.o

cc -o edit main.o kbd.o command.o
main.o : main.c defs.h
cc -c main.c
kbd.o : kbd.c defs.h command.h
cc -c kbd.c
command.o : command.c defs.h command.h
cc -c command.c
clean :
rm edit main.o kbd.o command.o
Sample Makefile (cont…)

• What happens when you do:

% make
• Or when you do?
% make clean
Macros/Variables

C_OBJ = main.o kbd.o display.o

CC = gcc
edit: $(C_OBJ)
$(CC) –o edit $(C_OBJ)
main.o: main.h defs.h
$(CC) –c main.c
…
More on make

• Pattern rules
%.o : %.cpp
g++ -c –O3 -Wall $<

$< is an automatic variable that refers to the first pre-

requisite, in this case the .cpp file.
More on make

• Implicit rule
%.o: %cpp
$(CXX) -c $(CFLAGS) -o $@ $<

• This says that for every target .o file with a .cpp

prerequisite, the given command will be executed.
More on make

• Phony targets
– For example clean
– Target name is not a generated file but just a names for recipe
to be executed on an explicit request
compare:
diff test.log test.au
clean
$(RM) $(CLEAN_THESE)
More information