Introduction to UNIX/LINUX Kernel GENERAL OVERVIEW OF THE SYSTEM * The UNIX system bas become quite popular since its inception in 1969 * The system is divided into two parts. > The first part consists of programs and services that have made the UNIX system environment. It is the part readily apparent to users, including such programs as the shell, mail, text processing packages, and source code control systems. » The second part consists of the operating system that supports these programs and services.Several reasons to choose UNIX system : * The system is written in a high-level language, making it easy to read, understand, change, and move to other machines. *It has a simple user interface that has the power to provide the services that users want. * It provides primitives that permit complex programs to be built from simpler programs. * It uses a hierarchical file system that allows easy maintenance and efficient implementation. * It uses a consistent format for files, the byte stream, making application programs easier to write. * It provides a simple, consistent interface to peripheral devices. * It is a multi-user, multi-process system; each user can execute several processes simultaneously. * It hides the machine architecture from the user, making it easier to write programs that run on different hardware implementations.System structure The operating system interacts directly 3 with the hardware, providing common services to programs and insulating them from hardware. Other application programs Other application programs Figure 1.1. Architecture of UNIX Systems* As there are set of layers, the system is commonly called the System Kernel or just Kernel Emphasize its isolation from user programs. * As programs are independent of the underlying hardware, it is easy to move them between UNIX systems running on different hardware. » Programs such as the shell and editors (ed and vi) shown in the outer layers interact with the kernel by invoking a well defined set of system calls. > The system calls instruct the kernel to do various operations for the calling program and exchange data between the kernel and the program. » Several programs shown in the figure are in standard system configurations and are known as commands, but private user programs may also exist in this layer as indicated by the program whose name is a.out, the standard name for executable files produced by the C compiler.* The standard C compiler, cc, is in the outermost layer of the figure: it invokes a C preprocessor, two-pass compiler, assembler, and loader (link-editor), all separate lower-I programs. * Even in figure two-level hierarchy of application programs is shown, users can extend the hierarchy to whatever levels are required.User perspective It reviews high-level features of the UNIX system such as the file system, the processing environment and building block primitives(ex. pipes). 1. The File System The UNIX file system is characterized by ¢ a hierarchical structure, * consistent treatment of file data, * the ability to create and delete files, © dynamic growth of files, * the protection of file data, * the treatment of peripheral devices (such as terminals and tape units) as files.> The file system is organized as a tree with a single root node called root (written “/"); > every non-leaf node of the file system structure is a directory of files . > files at the leaf nodes of the tree are either directories, regular files, or special device files. > The name of a file is given by a path name that describes how to locate the file in the file system hierarchy. >A path name is a sequence of component names separated by slash characters. -> a component is a sequence of characters that designates a file name that is uniquely contained in the previous (directory) component. > A full path name starts with a slash character and specifies a file that can be found by starting at the file system root and traversing the file tree, following the branches that lead to successive component names of the path name. (Ex. /usr/src/cmd/who.c ).USER PERSPECTIVE —-—-;CDRe—ae~” fs1 bin etc unix dev mjbmaury sh date who passwd src bin tty00 ttyO1 cmd date.c — who.c Figure 1.2. Sample File System Tree* Programs in the UNIX system have no knowledge of the internal format in which the kernel stores file data, treating the data as an unformatted stream of bytes. * Programs may interpret the byte stream as they wish, but the interpretation has no bearing on how the operating system stores the data. * Directories are like regular files in this respect; the system treats the data in a directory as a byte stream, but the data contains the names of the files in the directory in a predictable format so that the operating system and programs such as Is (list the names and attributes of files) can discover the files in a directory. * Permission to access a file is controlled by access permissions associated with the file. * Access permissions can be set independently to control read, write and execute permission for three classes of users : > the file owner, > a file group, > an everyone else.Suppose the name of the executable version of the program copy. A user at a terminal invokes the program by typing copy oldfile newfile Here, argc is 3, argv[O] points to the character string copy (the program name is conventionally the Oth parameter), argv[1] points to the character string oldfile, argv[2] points to the character string newfile * The program then checks that it bas been invoked with the proper number of parameters. * If so, it invokes the open system call "read-only" for the file oldfile, and if the system call succeeds, invokes the create system call to create newfile. The permission modes on the newly created file will be 0666 (octal), allowing all users access to the file for reading and writing. All system calls return —1 on failure; if the open or create calls fail, the program prints a message and calls the exit system with return status 1, terminating its execution and indicating that something went wrong.The open and create system calls return an integer called a file descriptor, which the program uses for subsequent references to the files. The write system read system eau returns the number of bytes read, returning O when it reaches the end of file.#include char buffer[2048]; int version = 1; /* Chapter 2 explains this */ main(argc, argv) int arge; char *argv{]; { int fdold, fdnew; if (argc 3) { _ printf("need 2 arguments for copy program\n’); exit(1); } fdold = open(argv[1], O_RDONLY); /* open source file read only */ if (fdold == -1) { _ printf("cannot open file %s\n", argv[1]); exit(1); } fdnew = creat(argv[2], 0666); /* create target file rw for all */ if (fdnew —I) { _ printf("cannot create file %An", argv[2]); exit(1); } copy(fdold, fdnew); exit (0);copy(old, new) int old, new; { int count; while ((count = read(old, buffer, sizeof(buffer))) > 0) write(new, buffer, count);Processing Environment A program is an executable file, and a process is an instance of the program in Execution. Various system calls allow processes to create new processes, terminate processes, synchronize stages of process execution, and control reaction to various events. The system calls allow users to write programs that do sophisticated Operations. The kernel of the UNIX system does not contain many functions that are part of the "kernel" in other systems. The prime example of such a program is the shell, the command interpreter program that users typically execute after logging into the system. The shell interprets the first word of a command line as a command name: for many commands, the shell forks and the child process execs the command associated with the name, treating the remaining words on the command line as parameters to the command.The shell allows three types of commands 1. a command can be an executable file that contains object code produced by compilation of source code (a C program for example). 2. a command can be an executable file that contains a sequence of shell command lines 3. a command can be an internal shell command (instead of an executable file). The internal commands make the shell a programming language in addition to a command interpreter and include commands for looping (for-in-do-done and while-do-done), commands for conditional execution (if-then-else-fl), a "case" statement command, a command to change the current directory of a process (cd), and several others.The shell syntax allows for pattern matching and parameter processing. > The shell usually executes a command synchronously, waiting for the command to terminate before reading the next command line. > it also allows asynchronous execution, where it reads the next command line and executes it without waiting for the prior command to terminate. Commands executed asynchronously are said to execute in the backgroundBuilding Block Primitives : The capability to redirect I/O. UNIX system is to provide operating system primitives that enable users to write small, modular programs that can be used as building blocks to build more complex programs. * Processes conventionally have access to three files: > they read from their standard input file, > write to their standard output file, and > write error messages to their standard error file. * Processes executing at a terminal typically use the terminal for these three files, but each may be "redirected" independently. Example : the command Is => lists all files in the current directory on the standard output, Is> output => redirects the standard output to the file called "output" in the current directory, mail mjb < letter => opens the file "letter" for its standard intput and mails its contents to the user named "mjb."2. the pipe a mechanism that allows a stream of data to be passed between reader and writer processes. * Processes can redirect their standard output to a pipe to be read by other processes that have redirected their standard input to come from the pipe. * The data that the first processes write into the pipe is the input for the second processes. * The second processes could also redirect their output, and so on, depending on programming need. Example : grep main a.c b.cc.c1 we —I counts the number of lines in the files that contain the string "main"; the output from grep is "piped" directly into the wc commandAssumptions about hardware The execution of user processes on UNIX systems is divided into two levels: user and kernel. When a process executes a system call, the execution mode of the process changes from user mode to kernel mode: Processes in user mode can access their own instructions and data but not kernel instructions and data (or those of other processes). Processes in kernel mode, however, can access kernel and user addresses. Some machine instructions are privileged and result in an error when executed in user mode.> Interrupts and Exceptions The UNIX system allows devices such as I/O peripherals or the system clock, that interrupt the CPU asynchronously. On receipt of the interrupt, the kernel saves its current context, determines the cause of the interrupt, and services the interrupt. * The hardware usually prioritizes devices according to the order that interrupts should be handled. When the kernel services an interrupt, it blocks out lower priority interrupts but services higher priority interrupts. * An exception condition refers to unexpected events caused by a process, such as addressing illegal memory, executing privileged instructions, dividing by zero, and so on.> Exceptions happen "in the middle" of the execution of an instruction, and the system attempts to restart the instruction after handling the exception. > interrupts are considered to happen between the execution of two instructions, and the system continues with the next instruction after servicing the interrupt.> Processor Execution Levels The kernel must sometimes prevent the occurrence of interrupts during critical activity, which could result in corrupt data if interrupts were allowed. Computers typically have a set of privileged instructions that set the processor execution level in the processor status word. Setting the processor execution level to certain values masks off interrupts from that level and lower levels, allowing only higher- level interrupt. If the kernel masks out disk interrupts, all interrupts except for clock interrupts and machine error interrupts are prevented. If it masks out software interrupts, all other interrupts may occur.Software Interrupts Figure 1.6. Typical Interrupt Levels Higher Priority Lower Priority> Memory Management * The kernel permanently resides in main memory as does the currently executing Process. * When compiling a program, the compiler generates a set of addresses in the program that represent addresses of variables and data structures or the addresses of instructions such as functions. * The compiler generates the addresses for a virtual machine as if no other program will execute simultaneously on the physical machine.* When the program is to run on the machine, the kernel allocates space in main memory for it, but the virtual addresses generated by the compiler need not be identical to the physical addresses that they occupy in the machine. * The kernel coordinates with the machine hardware to set up a virtual to physical address translation that maps the compiler- generated addresses to the physical machine addresses. * The mapping depends on the capabilities of the machine hardware, and the parts of UNIX systems that deal with them are therefore machine dependent.Architecture of user programs the UNIX : trap operating User Level a a Kernel Level” ~~ ~~ system t + ow eae] buffer cache i character: block device drivers [ hardware controlblock diagram of the kernel, showing various modules and their relationships to each other. it shows the file subsystem on the left and the process control subsystem on the right, the two major components of the kernel. shows three levels: user, kernel, and hardware. Assembly language programs may invoke system calls directly without a system call library. Programs frequently use other libraries such as the standard I/O library to provide a more sophisticated use of the system calls.> The file subsystem accesses file data using a buffering mechanism that regulates data flow between the kernel and secondary storage devices. * The buffering mechanism interacts with block I/O device drivers to initiate data transfer to and from the kernel. Device drivers are the kernel modules that control the operations of peripheral devices. Block I/O devices are random access storage devices alternatively, their device drivers make them appear to be random access storage devices to the rest of the system. > The process control subsystem is responsible for process synchronization inter-process communication, memory management, and process scheduling. The memory management module controls the allocation of memory. If at any time the system does not have enough physical memory for all processes, the kernel moves them between main memory and secondary memory so that all processes get a fair chance to execute. Vv> The scheduler module allocates the CPU to processes. > There are several forms of inter-process communication, ranging from asynchronous signaling of events to synchronous transmission of messages between processes. > the hardware control is responsible for handling interrupts and for communicating with the machine. Devices such as disks or terminals may interrupt the CPU while a process is executing. > If so, the kernel may resume execution of the interrupted process after servicing the interrupt: Interrupts are not serviced by special processes but by special functions in the kernel, called in the context of the currently running process.