AWS Cloud (S6ISPEC011)

6th ISE, Professional Elective

By Keerthan Kumar T G

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 1

 Virtual Machines and Virtualization of
Clusters and Data Centers

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 2

Virtualization
 Virtualization is a technology that allows multiple
virtual machines (VMs) to run on a single physical
machine, enabling efficient resource sharing and
improved performance.

 1960s and has gained renewed importance with the rise

of cloud and distributed computing.

 By separating hardware from software, virtualization

enhances system efficiency.

 It allows better utilization of computing resources such

as CPU, memory, and storage. Virtualization can be
applied at different layers, including operating
Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 3
systems, networks, and storage, making it a key
technology in modern computing.
 One of the earliest examples of virtualization is virtual memory, which
expands accessible memory beyond the physical RAM.

 Similarly, virtualization in cloud computing enables multiple users to run

different operating systems and applications on shared hardware.

 This flexibility is essential for data centers, cloud services, and large-scale
computing systems.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 4

 A 2009 Gartner Report identified virtualization as a transformative
technology in computing.

 With sufficient storage and computing power, virtualization makes it

possible to run different operating systems on the same hardware, even with
different processor architectures.

 This has led to advancements in cloud computing, IT infrastructure

management, and software development.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 5

Cloud Computing
 Cloud Computing: Delivers computing services (e.g., servers,
storage, networking, software) over the internet.

 Resources (computing, storage, etc.) are owned and managed

by a cloud provider.

 Centralized or hybrid architecture with data centers managed

by cloud providers.

 Example: AWS, Google Cloud, Microsoft Azure.

 Use case: Hosting applications, data storage, AI/ML

workloads, web services.
Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 6
Distributed Computing
 A system in which multiple computers work together to solve
a problem by sharing resources.

 Resources are owned and managed by multiple organizations

or distributed systems.

 Decentralized architecture where tasks are distributed

among multiple machines.

 Example: Hadoop, Apache Spark

 Use case: Large-scale data processing, scientific computing,

blockchain networks.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 7

Example

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 8

 Traditional computer vs Virtualized Computer

• In a traditional computer system, the host operating system is designed specifically for the underlying
hardware. However, with virtualization, multiple virtual machines (VMs) can run on the same physical
hardware, each with its own guest OS, independent of the host OS.

• This is achieved through a virtualization layer, known as the hypervisor or Virtual Machine Monitor
(VMM). The hypervisor manages the allocation of virtualized CPU, memory, and I/O resources to
different VMs, allowing multiple applications to run simultaneously on the same hardware.

• The virtualization software functions by creating an abstraction of physical hardware, enabling VMs to
Dr.use virtual
KK || Dept. resources
of ISE, SIT, efficiently. Virtualization can be implemented at different 9levels of a computer
Tumkur || KEERTHANKUMARTG@SIT.AC.IN

system, including:
Levels of Virtualization Implementation

 Virtualization can be implemented at different levels of a

computer system, including:

1. Application Level – Virtualizes applications to run on different

OS environments.
2. Library Support Level – Uses API libraries to provide
virtualization support (e.g., Java Virtual Machine).
3. Operating System Level – Allows multiple isolated user
environments on the same OS (e.g., containers).
4. Hardware Level – Uses a hypervisor to create and manage
VMs.
5. Instruction Set Architecture (ISA) Level – Translates
instructions between different processor architectures.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 10

Levels of Virtualization Implementation Contd.
Instruction Set Architecture (ISA) Level
•At this level, virtualization works by emulating one type of
processor’s instruction set on another.
•For example, an application written for a MIPS processor can
run on an x86 processor with the help of ISA emulation.
•This allows older software (legacy code) to run on new
hardware without modification.
•How it works:
• The simplest method is code interpretation, where each
instruction from the original system is converted one by
one into an instruction for the new system. However, this
is slow.
• A faster method is dynamic binary translation, which
translates groups of instructions at once, improving
efficiency.
•This process creates a Virtual ISA (V-ISA), where software
translation layers ensure compatibility between different
processor architectures.
Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 11
Hardware Abstraction Level
 Hardware-level virtualization is a technique where virtualization is performed directly on the
underlying hardware (bare HW), creating a virtual hardware environment for virtual
machines (VMs).

 This approach enables efficient resource sharing by virtualizing key components such as
processors, memory, and I/O devices.

 The primary goal of hardware virtualization is to enhance hardware utilization by allowing

multiple users to run applications on the same physical machine concurrently.

 This concept was first implemented in IBM VM/370 in the 1960s and has since evolved with
modern hypervisors like Xen, which enables virtualization of x86-based systems to run Linux
and other operating systems efficiently.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 12

Operating System Level

 OS-level virtualization acts as an abstraction layer between the traditional OS and user
applications, enabling the creation of isolated containers on a single physical server.

 These containers function like real servers, allowing OS instances to efficiently utilize
hardware and software in data centers.

 This approach is widely used in virtual hosting environments to allocate hardware resources
among multiple mutually distrusting users.

 Additionally, it plays a role in server hardware consolidation by moving services from

separate hosts into containers or VMs on a single server, optimizing resource usage.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 13

Library Support Level

 Most applications interact with the system using APIs exported by user-level
libraries rather than relying on lengthy system calls from the OS.
 Since many systems provide well-documented APIs, these interfaces become
a viable candidate for virtualization.
 This approach works by controlling the communication link between
applications and the system through API hooks.
 A notable example is WINE, which enables Windows applications to run on
UNIX hosts.

 Another example is vCUDA, which allows applications within VMs to utilize

GPU hardware acceleration for enhanced performance.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 14

User-Application Level

 Application-level virtualization virtualizes an application as a VM, where applications

typically run as processes on a traditional OS. This is why it is also referred to as process-
level virtualization.
 The most common approach involves High-Level Language (HLL) VMs, where the
virtualization layer operates as an application program on top of the operating system.

 This layer provides an abstraction of a VM, enabling programs written in an HLL to run on
it. Examples include Microsoft .NET CLR and Java Virtual Machine (JVM).
 Other forms of application-level virtualization include application isolation, application
sandboxing, and application streaming.

 These techniques wrap the application in a layer isolated from the host OS and other
applications, making it easier to distribute and remove from user workstations.
 An example is the LANDesk application virtualization platform, which deploys software
as self-contained, executable files in an isolated environment, eliminating the need for
installation, system modifications, or elevated security privileges.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 15

Relative Merits of Different Approaches
Compares different virtualization levels based on Higher Performance, Application Flexibility,
Implementation Complexity, and Application Isolation.
•Hardware and OS-level virtualization offer the highest performance but are costly to
implement.
•ISA (Instruction Set Architecture) virtualization provides maximum application flexibility.
•User isolation is the most challenging to achieve.
•The number of X’s in the table indicates the relative advantage, with five X’s as the best and
one X as the worst.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 16

VMM Design Requirements and Providers
 Hardware-Level Virtualization
• Inserts a Virtual Machine Monitor (VMM) layer between hardware and OS.
 Each time programs access the hardware the VMM captures the process. In this sense, the VMM acts as
a traditional OS. One hardware component, such as the CPU, can be virtualized as several virtual copies.
 Allows multiple traditional OS instances (same or different) to run on the same hardware.
 Three Key Requirements for a VMM
Provide Identical environment to the original machine for programs.
programs run in this environment should show, at worst, only minor decreases in speed.
VMM should be in Complete control over system resources.

 Performance Considerations
• A VMM should efficiently allocate resources to multiple VMs.
• Total resource demand may exceed that of the physical machine.
• Time-sharing OS behavior is not classified as a VMM.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 17

VMM Design Requirements and Providers
Contd.
 Efficiency vs. Traditional Emulators
• Traditional emulators and software interpreters (simulators) execute each
instruction via macros/functions (slow).
• To ensure VMM efficiency, most instructions should run directly on the real
processor without software intervention

 VMM Control Over Resources

• Allocates hardware resources to programs.
• Prevents unauthorized resource access.
• Can reclaim allocated resources under specific conditions

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 18

VMM Design Requirements and Providers Contd.
 Processor Challenges & Hardware-Assisted Virtualization
• Some processors (e.g., x86 architecture) make VMM implementation difficult.
• Certain privileged instructions cannot be trapped, requiring hardware modifications.
• Hardware-assisted virtualization enhances processor support for VMM functionality.

 .(Table 3.2) compares four hypervisors and VMMs that are in use today
 Different VMMs and hypervisors vary in performance, control, and efficiency.
• Hardware-assisted virtualization improves modern hypervisors like VMware, Xen, and
Hyper-V.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 19

VMM Design Requirements and Providers
Contd.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 20

Complete control of these resources by a
VMM includes the following aspects:
 (1) The VMM is responsible for allocating hardware resources for programs;

 (2) it is not possible for a program to access any resource not explicitly allocated to it;
and

 (3) it is possible under certain circumstances for a VMM to regain control of resources
already allocated.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 21

Virtualization Support at the OS Level

 Cloud computing leverages VM technology to shift hardware and staffing

costs to third-party providers, similar to banks. While it is transforming
computing, two key challenges remain:
1. Dynamic Resource Scaling – Cloud applications require a variable number of
physical machines and VMs, adjusting from single to hundreds of CPUs
based on demand.
2. Slow VM Instantiation – New VMs take time to start, either through fresh
boots or template replication, lacking awareness of the current application
state.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 22

Why OS-Level Virtualization?
 Challenges with Hardware-Level Virtualization:
• Slow VM initialization – Each VM starts from scratch, making large-scale cloud
deployments inefficient.
• Storage overhead – VM images contain repeated content, increasing storage
requirements.
• Performance drawbacks – Low density, slow execution, and guest OS
modifications (para-virtualization).
• Hardware dependency – Sometimes requires hardware modifications to
optimize performance.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 23

Why OS-Level Virtualization?
Contd.
 Advantages of OS-Level Virtualization:
• Inserts a virtualization layer inside the OS to partition physical resources.
• Supports multiple isolated VMs (Virtual Execution Environments - VE, Virtual
Private Systems - VPS, or Containers).
• Efficient and lightweight – All VEs share the same OS kernel, reducing
overhead.
• Each VE appears as a real server, with its own processes, file system, user
accounts, and network settings.
• Also known as single-OS image virtualization due to kernel sharing.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 24

Figure 3.3 illustrates operating system virtualization from the point of view of a machine stack.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 25

 Advantages of OS Extensions

 Compared to hardware-level virtualization, OS-level virtualization offers two key

advantages:
1. Efficiency & Scalability – Minimal startup/shutdown time, low resource usage, and high
scalability.
2. State Synchronization – OS-level VMs can sync with the host environment when needed.
 These benefits are achieved through:
• Single OS Kernel Sharing – All VMs on a machine use the same OS kernel, reducing
overhead.
• Controlled Resource Access – VMs can use host resources but cannot modify them.
 In cloud computing, these advantages help overcome slow VM initialization and lack of
application state awareness in hardware-level virtualization.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 26

Disadvantages of OS Extensions

 A key limitation of OS-level virtualization is that all VMs on a single container must
use the same OS family. While different distributions are allowed, mixing OS types
(e.g., Windows on a Linux-based container) is not possible.

 This poses a challenge in cloud computing, where users may have diverse OS
preferences, requiring support for both Windows and Linux environments.

 For example, a Windows distribution such as Windows 7 cannot run on a Linux-based

container. However, users of cloud computing have various preferences. Some prefer Windows
and others prefer Linux or other operating systems. Therefore, there is a challenge for OS-level
virtualization in such cases.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 27

Advantages of OS Extensions
 Further to overcome the Challenges of Hardware-Level
Virtualization:
• Slow VM initialization, especially in large-scale cloud
environments.
• Storage overhead due to repeated content across VM images.
• Performance issues – Low density and slow execution.
• Para-virtualization requires modifying the guest OS, and
sometimes hardware modifications.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 28

Advantages of OS Extensions cntd
 OS-Level Virtualization as a Solution:
• Inserts a virtualization layer inside the OS to partition physical resources.
• Enables multiple isolated VMs (Virtual Execution Environments - VE, Virtual Private Systems
- VPS, or Containers).
• VEs function as real servers, each with its own processes, file system, user accounts, and
network settings.
• All VEs share the same OS kernel, making it a single-OS image virtualization model.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 29

 Advantages of OS Extensions cntd.

Implementation Considerations (Figure 3.3):

•Redirects VM access requests to local resource partitions.
•Example: The chroot command in UNIX creates virtual root directories for
VMs.
•Two approaches to virtual root directories:
1.Duplicate resources for each VM (high resource cost).
2.Share most resources with the host OS, creating private copies only
when needed (more efficient).

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 30

Containers vs. Kubernetes

 What are Containers?

• Containers are lightweight, standalone, and executable units that package an
application along with its dependencies, libraries, and configurations.
• They run on top of a shared OS kernel, making them faster and more efficient
than virtual machines (VMs).
• Examples: Docker, LXC, OpenVZ.
 What is Kubernetes?
• Kubernetes (K8s) is an orchestration platform that manages and automates the
deployment, scaling, and operation of containerized applications.
• It helps in load balancing, auto-scaling, networking, and fault tolerance across
multiple containers.
• Originally developed by Google, now an open-source CNCF project.
Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 31
Middleware Support for Virtualization
 Library-level virtualization is also known as user-level Application Binary Interface (ABI) or API emulation.

 This type of virtualization can create execution environments for running alien programs on a platform
rather than creating a VM to run the entire operating system.

 API call interception and remapping are the key functions performed. This section provides an overview of
several library-level virtualization systems: namely the Windows Application Binary Interface (WABI), lxrun,
WINE, Visual MainWin, and vCUDA,

 Windows Application Binary Interface (WABI) – Runs Windows applications on UNIX.

 lxrun – Enables Linux applications to run on Solaris.
 WINE – Runs Windows applications on Linux/macOS.
 Visual MainWin – Allows Windows applications to run on UNIX/Linux.
 vCUDA – Provides GPU virtualization for CUDA applications in virtualized environments.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 32

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 33
Example 3.2 The vCUDA for Virtualization of General-Purpose GPUs

This architecture enables CUDA applications on VMs to

leverage GPU resources efficiently while maintaining
compatibility with the host system.

CUDA is a programming model for running compute-

intensive applications on GPUs, but it faces challenges
when used in hardware-level VMs. vCUDA virtualizes the
CUDA library, allowing CUDA applications to run on guest
OSes by intercepting and redirecting API calls to the host
OS.
vCUDA follows a client-server model with three key
components:
1.vCUDA Library (Guest OS) – Replaces the standard
CUDA library, intercepts API calls, and redirects them.
2.vGPU (Guest OS, Client) – Abstracts GPU hardware,
manages virtual memory, and tracks CUDA API execution.
3.vCUDA Stub (Host OS, Server) – Receives API requests,
Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN
creates execution contexts, 34and manages real GPU
resources.
VIRTUALIZATION STRUCTURES/TOOLS AND MECHANISMS

 there are three typical classes of VM architecture.

 Before virtualization, the operating system manages the hardware. After virtualization,
a virtualization layer is inserted between the hardware and the operating system.

 Depending on the position of the virtualization layer, there are several classes of VM
architectures, namely the hypervisor architecture, paravirtualization, and host-based
virtualization.

 The hypervisor is also known as the VMM (Virtual Machine Monitor). They both
perform the same virtualization operations.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 35

Hypervisor and Xen Architecture

 The hypervisor supports hardware-level virtualization on bare metal devices like

CPU, memory, disk and network interfaces.

 The hypervisor software sits directly between the physical hardware and its OS. This
virtualization layer is referred to as either the VMM or the hypervisor.

 The hypervisor provides hypercalls for the guest OSes and applications.
 Example: HYPERVISOR_memory_op() is a hypercall in Xen used for memory management.
 Depending on the functionality, a hypervisor can assume a micro-kernel architecture
like the Microsoft Hyper-V. Or it can assume a monolithic hypervisor architecture like
the VMware ESX for server virtualization.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 36

Hypervisor and Xen Architecture contd.

 A micro-kernel hypervisor has a minimal core, handling only essential

functions like memory management and CPU scheduling, while device drivers
and other components run outside the hypervisor.

 In contrast, a monolithic hypervisor includes all functions, including device

drivers, making it larger in size.

 Regardless of the type, a hypervisor must efficiently virtualize physical

devices into dedicated virtual resources for VMs.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 37

The Xen Architecture

Fig. 3.5 The Xen architecture’s special domain 0 for control and
I/O, and several guest domains for user applications.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 38

The Xen Architecture contd.

 Xen is an open-source hypervisor developed by Cambridge University. Xen is

a microkernel hypervisor, which separates the policy from the mechanism.
 The Xen hypervisor implements all the mechanisms, leaving the policy to be
handled by Domain 0.

 Xen does not include any device drivers natively. It just provides a mechanism
by which a guest OS can have direct access to the physical devices.

 As a result, the size of the Xen hypervisor is kept rather small. Xen provides a
virtual environment located between the hardware and the OS.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 39

 •A number of vendors are in the process of developing commercial Xen hypervisors,
among them are Citrix XenServer and Oracle VM.

•The core components of a Xen system are the hypervisor, kernel, and
applications.

•The organization of the three components is important. Like other virtualization

systems, many guest OSes can run on top of the hypervisor.

•However, not all guest OSes are created equal, and one in particular controls the
others. The guest OS, which has control ability, is called Domain 0, and the others are
called Domain U

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 40

 Domain 0 is a privileged guest OS of Xen. It is first loaded when Xen boots
without any file system drivers being available.

 Domain 0 is designed to access hardware directly and manage devices.

 Therefore, one of the responsibilities of Domain 0 is to allocate and map

hardware resources for the guest domains (the Domain U domains).

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 41

hardware virtualization Type: Binary Translation with Full Virtualization

 Depending on implementation technologies, hardware virtualization can be classified into two categories:
full virtualization and host-based virtualization.

 Full virtualization does not need to modify the host OS. It relies on binary translation to trap and to virtualize
the execution of certain sensitive, nonvirtualizable instructions.

 The guest OSes and their applications consist of noncritical and critical instructions. In a host-based system,
both a host OS and a guest OS are used.

 A virtualization software layer is built between the host OS and guest OS.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 42

Full Virtualization

 With full virtualization, noncritical instructions run on the hardware directly while
critical
 instructions are discovered and replaced with traps into the VMM to be emulated by
software. Both
 the hypervisor and VMM approaches are considered full virtualization.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 43

 Binary Translation of Guest OS Requests Using a VMM

•This approach was implemented by VMware and other software

companies.

•VMware places the VMM at Ring 0 and the guest OS at Ring 1.

•The VMM scans the instruction stream and identifies privileged,

control-, and behavior-sensitive instructions.

•These instructions are trapped and emulated using binary translation.

•Full virtualization combines binary translation and direct execution,

ensuring the guest OS is unaware of virtualization.

•The performance may not be ideal due to the time-consuming nature of

binary translation, especially for I/O-intensive applications.

•A code cache is used to store translated hot instructions, improving

performance but increasing memory usage.
•On
Dr. KKx86
|| Dept.architecture, full virtualization achieves 80% to 97% of the host
of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 44

machine’s performance.
Host-Based

Virtualization
An alternative VM architecture is to install a virtualization layer on top of the host OS.
This host OS is still responsible for managing the hardware. The guest OSes are
installed and run on top of the virtualization layer.

 This host based architecture has some distinct advantages, as enumerated

next.
 First, the user can install this VM architecture without modifying the host OS.
The virtualizing software can rely on the host OS to provide device drivers and
other low-level services. This will simplify the VM design and ease its
deployment.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 45

Host-Based Virtualization

 Second, the host-based approach appeals to many host machine

configurations. Compared to the hypervisor/VMM architecture, the
performance of the host-based architecture may also be low.

 When an application requests hardware access, it involves four layers of

mapping which downgrades performance significantly. When the ISA of a
guest OS is different from the ISA of the underlying hardware, binary
translation must be adopted. Although the host-based architecture has
flexibility, the performance is too low to be useful in practice.

 Indirect execution of complex instructions via binary translation of guest OS

requests using the VMM plus direct execution of simple instructions on the
same host
Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 46
Para-Virtualization with Compiler Support
 Para-virtualization needs to modify the guest operating systems. A para-virtualized VM provides
special APIs requiring substantial OS modifications in user applications.

FIGURE 3.7 Para-virtualized VM architecture, which involves

modifying the guest OS kernel to replace
nonvirtualizable instructions with hypercalls for the
hypervisor or the VMM to carry out the virtualization FIGURE 3.8
process (See Figure 3.8 for more details.) The use of a para-virtualized guest OS assisted by
an intelligent compiler to replace nonvirtualizable OS
instructions by hypercalls.
Figure 3.7 illustrates the concept of a para-virtualized VM architecture. The guest operating systems
Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 47
are para-virtualized. They are assisted by an intelligent compiler to replace the nonvirtualizable
OS instructions by hypercalls as illustrated in Figure 3.8.
Para-Virtualization with Compiler Support
contd.
 The traditional x86 processor offers four instruction execution rings: Rings 0, 1, 2, and
3.

 The lower the ring number, the higher the privilege of instruction being executed.

 The OS is responsible for managing the hardware and the privileged instructions to
execute at Ring 0, while user-level applications run at Ring 3.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 48

Para-Virtualization Architecture

 When the x86 processor is virtualized, a virtualization layer is inserted between the
hardware and the OS. According to the x86 ring definition, the virtualization layer
should also be installed at Ring 0.

 Different instructions at Ring 0 may cause some problems. In Figure 3.8, we show that
para-virtualization replaces nonvirtualizable instructions with hypercalls that
communicate directly with the hypervisor or VMM.

 However, when the guest OS kernel is modified for virtualization, it can no longer run
on the hardware directly.

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 49

Dr. KK || Dept. of ISE, SIT, Tumkur || KEERTHANKUMARTG@SIT.AC.IN 50
VIRTUALIZATION OF CPU, MEMORY, AND I/O DEVICES
To support virtualization, processors such as the x86 employ a special running
mode and instructions, known as hardware-assisted virtualization.

In this way, the VMM and guest OS run in different modes and all sensitive
instructions of the guest OS and its applications are trapped in the VMM.

To save processor states, mode switching is completed by hardware, The VMM

then emulates or handles these instructions safely before returning control to
the guest OS.

For the x86 architecture, Intel and AMD have proprietary technologies for
hardware-assisted virtualization.

51
Hardware Support for Virtualization
Modern operating systems and processors permit multiple processes to
run simultaneously.
If there is no protection mechanism in a processor, all instructions from
different processes will access the hardware directly and cause a system
crash.
Therefore, all processors have at least two modes, user mode and
supervisor mode, to ensure controlled access of critical hardware.
Instructions running in supervisor mode are called privileged instructions.
Other instructions are unprivileged instructions.
One or more guest OS can run on top of the hypervisor. Ex: KVM (Kernel-
based Virtual Machine).
It is a Linux kernel virtualization infrastructure. KVM can support
hardware-assisted virtualization. 52
Example 3.4 discusses Intel’s hardware support
approach.

53
Intel provides a hardware-assist technique to make virtualization easy and improve performance.

Figure 3.10 provides an overview of Intel’s full virtualization techniques. For processor virtualization, Intel
offers the VT-x or VT-i technique.
VT-x (Intel Virtualization Technology for x86-based platforms)

VT-i (Intel Virtualization Technology for Itanium processors)

VT-x adds a privileged mode (VMX Root Mode) and some instructions to processors. This enhancement traps
all sensitive instructions in the VMM automatically.

54
Processor Virtualization:
 VT-x (Intel Virtualization Technology for x86-based platforms)
 Adds VMX Root Mode (privileged mode)
 Provides instructions for trapping sensitive instructions in the Virtual Machine
Monitor (VMM)
Memory Virtualization:

 EPT (Extended Page Tables)

 Translates virtual addresses to physical addresses for improved performance
I/O Virtualization:

 VT-d (Intel Virtualization Technology for Directed I/O)

 Supports I/O device virtualization and isolation for better performance
 VT-c (Intel Virtualization Technology for Communication)
 Supports efficient communication between virtual machines and I/O
devices
55
CPU Virtualization

A VM (Virtual Machine) is a replica of an existing computer system where most

instructions run directly on the host processor for efficiency.
However, certain critical instructions must be handled carefully to ensure correctness and
stability. These instructions are divided into three categories:
1. Privileged Instructions: These require privileged mode to execute and are trapped if
attempted outside of this mode.
2. Control-sensitive Instructions: These attempt to modify the configuration of system
resources.
3. Behavior-sensitive Instructions: Their behavior depends on the configuration of
resources, including operations like load and store over virtual memory.

56
A CPU architecture is considered virtualizable if it allows both privileged and unprivileged instructions of a virtual
machine (VM) to run in user mode, while the Virtual Machine Monitor (VMM) runs in supervisor mode.
Critical instructions, such as control- and behavior-sensitive instructions, are trapped in the VMM to ensure system
stability.

CPU Architecture Support:

 RISC CPUs: Naturally virtualizable because control- and behavior-sensitive instructions are treated as privileged
instructions, making them easier to manage in a virtualized environment.

 x86 CPUs: Not designed for virtualization, as some sensitive instructions (e.g., SGDT, SMSW) are not privileged.
These instructions cannot be trapped by the VMM, complicating virtualization.
System Calls in Virtualization:

 In a native UNIX-like system, system calls trigger an interrupt (80h) that passes control to the OS kernel for
processing.
 In a paravirtualized system (e.g., Xen), the guest OS triggers the 80h interrupt for a system call, but
simultaneously, the hypervisor triggers the 82h interrupt. This allows the hypervisor to process the system call
and then return control to the guest OS kernel.
Performance Impact: Paravirtualization allows unmodified applications to run in VMs but introduces a small
performance penalty due to the hypervisor’s involvement in handling system calls. 57
Hardware-Assisted CPU Virtualization
Full and paravirtualization methods are complicated because they require changes to the
operating systems or involve complex techniques like binary translation to make the OS work
inside a virtual machine.

Hardware-Assisted Virtualization simplifies the process by enabling the hardware (CPU) to

directly support virtualization.
This helps avoid the complexity of full and paravirtualization, allowing VMs to run more
efficiently and with fewer changes to the guest operating systems.

x86 processors (used in Intel and AMD CPUs) have different levels of access called privilege
levels or "rings".
 Ring 0: This is the most privileged level. The operating system (OS) runs here and has
direct control over hardware.
 Ring 1: This is often referred to as the hypervisor ring, but it’s not always used in
hardware-assisted virtualization.
 Ring 3: This is where user applications run, and they have the least privilege.
 Ring -1: In hardware-assisted virtualization, this is a special level for the hypervisor. The
hypervisor operates beneath Ring 0 to manage the virtual machines. 58
The hypervisor can directly control the hardware without
modifying the guest OS. And Privileged instructions from
the guest OS are automatically trapped by the hypervisor.
This removes the complexity of binary translation and
allows unmodified operating systems to run in virtual
machines.
Technologies like Intel VT-x and AMD-V enable these
features in modern processors.
59
 Hardware-Assisted CPU Virtualization
This technique attempts to simplify virtualization because full or paravirtualization is
complicated.

60
x86 Processors: Not originally designed for virtualization, but extensive efforts have been made to
virtualize them.
RISC Comparison: x86 processors are often compared to RISC processors, which are easier to
virtualize. However, x86-based legacy systems are still widely used and can't be easily discarded.

Intel defines the virtualization privilege level as VMX Root Mode.

VMX stands for Virtual Machine Extensions, which add hardware support for virtualization.
Hypervisor Operation: The hypervisor runs in VMX Root Mode, while guest VMs run in lower
privilege modes.
Intel adds special instructions to control VM start/stop and allocate memory pages to store CPU
state for VMs.
These instructions are used to manage the transition between the hypervisor and the guest
operating systems.

61
Memory Virtualization
In traditional execution environments, modern operating systems
manage virtual memory by mapping virtual addresses to physical machine
memory using page tables.
Memory Management Unit (MMU) and Translation Lookaside Buffer
(TLB) are used to optimize memory performance in x86 CPUs.
Virtual Memory Virtualization in Virtualized Environments
 In virtualized environments, the system's physical memory (RAM) is
shared and dynamically allocated to virtual machines (VMs).
 This involves a two-stage mapping process:
 Guest OS manages the mapping of virtual addresses to guest
physical memory.
 Virtual Machine Monitor (VMM) maps guest physical memory to
actual machine memory (host memory).
62
Two-Level Mapping Procedure
 Guest OS:
 Controls the mapping of virtual memory to guest physical memory within each VM.
 Guest OS does not directly access the host machine memory.
 VMM:
 Responsible for mapping guest physical memory to actual machine memory.
The two-level memory mapping involves:
 Virtual memory → Guest physical memory (managed by the guest OS).
 Guest physical memory → Host machine memory (managed by the VMM).

MMU Virtualization
 MMU virtualization is crucial for efficient virtual memory management.
 It should be transparent to the guest OS, meaning the guest OS doesn’t need to be aware
of the underlying virtualization.
 The VMM handles all interactions with actual physical memory, ensuring guest OS memory
isolation.

63
64
Performance Considerations
 Efficient memory virtualization relies on mechanisms like MMU
virtualization to manage the two-stage mapping with minimal overhead.

 The VMM ensures that the virtual machines can access memory without
directly interacting with the host machine’s physical memory.

65
Extended Page Table by Intel for Memory Virtualization

66
Memory virtualization challenge: Shadow page tables were slow.

Intel’s solution: Introduced Extended Page Table (EPT) for hardware-based memory translation.

Additional enhancement: Virtual Processor ID (VPID) to improve TLB performance.

67
Working of EPT (Memory Translation Flow)

1. Guest Virtual Address (GVA) to Guest Physical Address (GPA)

 CPU first looks up the L4 page table referenced by Guest CR3.

 The address in Guest CR3 is a GPA, requiring translation to a Host Physical Address (HPA).
2. Guest Physical Address (GPA) to Host Physical Address (HPA) using EPT

 CPU checks the EPT TLB for translation.

 If translation is not found in the EPT TLB, it looks into the EPT page tables.
 If the translation is missing in EPT, an EPT violation exception is triggered.
3. Handling Page Table Lookups

 The CPU uses four-level guest page tables for GVA → GPA translation.
 Then, it accesses four-level EPT tables to obtain the final HPA.
 In the worst case, the CPU performs 20 memory accesses (5 EPT lookups × 4 memory accesses per lookup).

68
69
VIRTUAL CLUSTERS AND RESOURCE MANAGEMENT
A physical cluster is a collection of servers (physical machines) interconnected by a physical network such as a
LAN.

When a traditional VM is initialized, the administrator needs to manually write configuration information or
specify the configuration sources.
When more VMs join a network, an inefficient configuration always causes problems with overloading or
underutilization.
Ex: Amazon’s Elastic Compute Cloud (EC2) is a good example of a web service that provides elastic computing
power in a cloud. EC2 permits customers to create VMs and to manage user accounts over the time of their
use.
Most virtualization platforms, including XenServer and VMware ESX Server, support a bridging mode which allows
all domains to appear on the network as individual hosts.
By using this mode, VMs can communicate with one another freely through the virtual network interface card
and configure the network automatically.

70
Physical versus Virtual Clusters
Virtual clusters are built with VMs installed at distributed servers from one or more physical clusters.

The VMs in a virtual cluster are interconnected logically by a virtual network across several physical
networks.

FIGURE 3.18
A cloud
platform
with four
virtual
clusters over
three
physical
clusters
shaded
differently
71
The virtual cluster nodes can be either physical or virtual machines. Multiple VMs running
with different OSes can be deployed on the same physical node.

Each node in this cluster can be either:

1. Physical Machine (Bare Metal Node): A real, physical server that is dedicated to running
workloads. These provide direct hardware access and higher performance but are less
flexible.
2. Virtual Machine (VM Node): A software-based simulation of a physical machine running on
a hypervisor (e.g., VMware, KVM, Hyper-V). Multiple VMs can run on a single physical
machine, providing better resource utilization and flexibility.

• A VM runs with a guest OS, which is often different from the host OS, that manages the
resources in the physical machine, where the VM is implemented.
• The purpose of using VMs is to consolidate multiple functionalities on the same server. This
will greatly enhance server utilization and application flexibility.

72
VMs can be colonized (replicated) in multiple servers for the purpose of promoting
distributed parallelism, fault tolerance, and disaster recovery.

• The size (number of nodes) of a virtual cluster can grow or shrink dynamically, similar
to the way an overlay network varies in size in a peer-to-peer (P2P) network.

• The failure of any physical nodes may disable some VMs installed on the failing
nodes. But the failure of VMs will not pull down the host system.

73
Since system virtualization has been widely used, it is necessary to effectively manage VMs
running on a mass of physical computing nodes (also called virtual clusters) and consequently
build a high-performance virtualized computing environment.
Figure 3.19 shows the concept of a virtual cluster based on application partitioning or
customization.

74
Each VM can be installed on a remote server or replicated on multiple servers
belonging to the same or different physical clusters.

The boundary of a virtual cluster can change as VM nodes are added, removed,
or migrated dynamically over time.

75
Fast Deployment and Effective Scheduling
▪ The system should have the capability of fast deployment. Here, deployment means two things: to
construct and distribute software stacks (OS, libraries, applications) to a physical node inside
clusters as fast as possible, and to quickly switch runtime environments from one user’s virtual cluster
to another user’s virtual cluster.

▪ If one user finishes using his system, the corresponding virtual cluster should shut down or suspend
quickly to save the resources to run other VMs for other users.

High-Performance Virtual Storage

▪ Basically, there are four steps to deploy a group of VMs onto a target cluster: preparing the disk
image, configuring the VMs, choosing the destination nodes, and executing the VM deployment
command on every host.
▪ Templates could implement the COW (Copy on Write) format. A new COW backup file is very small
and easy to create and transfer. Therefore, it definitely reduces disk space consumption. In addition,
VM deployment time is much shorter than that of copying the whole raw image file.

76
Live VM Migration Steps and Performance Effects
Live migration refers to the process of moving a running virtual machine (VM), container, or process from
one physical machine to another with minimal downtime.

•Pre-copy Migration
•The memory pages of the VM are iteratively copied to the destination while the source VM is still running.
•During the final iteration, only modified pages are transferred before switching execution to the destination.
•Advantage: Minimal downtime.
•Disadvantage: High network overhead due to repeated memory transfers.

•Post-copy Migration
•The execution is immediately switched to the destination after transferring minimal state (CPU, registers).
•The remaining memory pages are fetched on-demand from the source.
•Advantage: Reduces total migration time and network overhead.
•Disadvantage: Possible performance degradation due to memory fetch delays.

77
There are four ways to manage a virtual cluster.
 First, you can use a guest-based manager, by which the cluster manager
resides on a guest system () virtual machines (VMs)..

 It treats VMs as physical nodes, managing cluster tasks from within the guest
system.

 The host-based manager supervises the guest systems and can restart the
guest system on another physical machine. Incase a guest system fails, the
host-based manager can restart it on another physical machine. A good
example is the VMware High Availability system that can restart a guest
system after failure.
 A third way to manage a virtual cluster is to use an independent cluster
manager on both the host and guest systems. This will make infrastructure
management more complex, however. This improves fault tolerance and
flexibility but increases complexity. 78
Various cluster management schemes can be greatly enhanced when VM life
migration is enabled with minimal overhead.

VMs can be live-migrated from one physical machine to another; in case of

failure, one VM can be replaced by another VM.

Furthermore, we should ensure that the migration will not disrupt other active
services residing in the same host through resource contention (e.g., CPU,
network bandwidth).

79
A VM can be in one of the following four states. An inactive state is defined by
the virtualization platform, under which the VM is not enabled.
An active state refers to a VM that has been instantiated at the virtualization
platform to perform a real task.

A paused state corresponds to a VM that has been instantiated but disabled to

process a task or paused in a waiting state.
A VM enters the suspended state: The machine file and virtual resources are
stored back to the disk.

80
FIGURE 3.20 Live migration process of a VM from one host to another.
81
Steps in Live Migration
Stage 0: Pre-Migration
1. The VM is actively running on Host A.
2. An alternate physical host (Host B) may be selected in advance.
3. Block devices are mirrored, and free resources are maintained.
Stage 1: Reservation
1. A container is initialized on the target host (Host B) to prepare for migration.
Stage 2: Iterative Pre-Copy
1. Shadow paging is enabled to track memory changes.
2. The VM’s memory pages are copied in multiple rounds, sending dirty pages
(pages modified during copying) in each iteration.
3. This step minimizes downtime by reducing the amount of data that needs to be
transferred in the final step.

82
•Stage 3: Stop and Copy (Downtime begins – VM is out of service)
•The VM is suspended on Host A.
•An ARP (Address Resolution Protocol) update is generated to redirect
network traffic to Host B.
•The remaining VM state (including final memory pages and processor
state) is synchronized to Host B.
•Stage 4: Commitment
•The VM state on Host A is released, ensuring that the VM will now only
run on Host B.
•Stage 5: Activation (VM resumes on Host B)
•The VM starts on Host B.
•It connects to local devices and resumes normal operation.

83
84
VIRTUALIZATION FOR DATA-CENTER AUTOMATION

 Server Consolidation in Data Centers

 Virtual Storage Management

 Cloud OS for Virtualized Data Centers.

85
Server Consolidation in Data Centers

 In data centers, a large number of heterogeneous workloads can run on servers at various times.
 These heterogeneous workloads can be roughly divided into two categories: chatty workloads and non
interactive workloads.
 Chatty workloads may burst at some point and return to a silent state at some other point.
 These workloads involve frequent, small interactions between systems, often requiring low latency and
high responsiveness.
 A web video service, Database queries.
 Non-interactive workloads do not require people’s efforts to make progress after they are submitted.

❑ High-performance computing is a typical example of this. At various stages, the requirements for resources
of these workloads are dramatically different.
❑ Processes large amounts of data in a sequential or parallel manner. Not time-sensitive; can be scheduled to
run during off-peak hours.
❑ Large-scale simulations, Machine learning model training

However, to guarantee that a workload will always be able to cope with all demand levels, the workload is statically
allocated enough resources so that peak demand is satisfied.
86
 Therefore, it is common that most servers in data centers are
underutilized. A large amount of hardware, space, power, and
management cost of these servers is wasted.

 Server consolidation is an approach to improve the low utility

ratio of hardware resources by reducing the number of physical
servers.

 Among several server consolidation techniques such as

centralized and physical consolidation, virtualization-based
server consolidation is the most powerful.

 Server virtualization enables smaller resource allocation than a

physical machine, however server virtualization has the following
side effects/benefits: 87
 Enhanced Resource Utilization – Combines multiple underutilized servers into fewer,
optimizing hardware efficiency.

 Agile Provisioning & Deployment – Virtual machine (VM) images can be easily cloned and
reused, speeding up resource deployment.

 Cost Reduction – Lowers expenses by reducing the need for new servers, minimizing data
center space, and cutting maintenance, power, and cooling costs.

 Improved Availability & Business Continuity – Guest OS failures don’t impact others, and VMs
can be migrated seamlessly across servers without hardware dependency.

88
 To automate data-center operations, one must consider resource scheduling, architectural support,
power management, automatic or autonomic resource management, performance of analytical
models, and so on.

 In virtualized data centers, an efficient, on-demand, fine-grained scheduler is one of the key factors to
improve resource utilization. Scheduling and reallocations can be done in a wide range of levels in a
set of data centers.

 The levels match at least at the VM level, server level, and data-center level. Ideally, scheduling and
resource reallocations should be done at all levels. However, due to the complexity of this, current
techniques only focus on a single level or, at most, two levels.

 Dynamic CPU allocation is based on VM utilization and

 application-level QoS metrics.
89
 Virtual Storage Management

•Before System Virtualization, we Used to aggregate & partition disks for

physical machines.
•In System Virtualization, it Involves storage managed by Virtual Machine
Monitors (VMMs) and guest OSes.

•Types of Data in Virtual Storage:

1.VM Images – Specific to virtualization (OS + applications).
2.Application Data – Same as in traditional OS environments.

Key Concepts for above: Encapsulation & Isolation

• Encapsulation: OS and applications are encapsulated in VMs.
• Isolation: Multiple VMs run independently on the same physical machine.

90
Virtual Storage Management Contd.
Storage Management Challenges in Virtualization
• Complex Storage Operations:
• Guest OS behaves as if using a real disk but cannot access it directly.

• Multiple VMs compete for storage, making management complex.

• Inefficient Storage Primitives:

• Moving VM data across hosts is difficult & slow.

• Checkpointing disks (saving VM state) is complicated.

• VM Image Flooding:
• Thousands of VMs lead to storage overload in data centers.

• Solutions aim to reduce VM image size & improve performance.

91
Solutions for Virtual Storage Management
• Parallax: A distributed storage system designed for virtualization.
• Content Addressable Storage (CAS): Reduces VM image size, supporting
large-scale VM-based systems.

• Parallax Storage Architecture:

• Moves storage functions from high-end hardware into a federation of

storage VMs.
• Each physical machine has a storage appliance VM, acting as a block

virtualization layer.
• This provides a virtual disk for each VM on the same physical machine.

92
Cloud OS for Virtualized Data Centers.
 Data centers must be virtualized to serve as cloud providers.

 Table 3.6 summarizes four virtual infrastructure (VI) managers and OSes. These VI managers and OSes are
specially tailored for virtualizing data centers which often own a large number of servers in clusters.

93
Eucalyptus for Virtual Networking of Private Cloud
 intended mainly for supporting Infrastructure as a Service (IaaS) clouds.

FIGURE 3.27 Eucalyptus for building private clouds by establishing virtual networks over the VMs 94
linking through Ethernet and the Internet.
 intended mainly for supporting Infrastructure as a Service (IaaS) clouds.
 The system primarily supports virtual networking and the management of VMs; virtual storage is not supported.
 Its purpose is to build private clouds that can interact with end users through Ethernet or the Internet.

 The system also supports interaction with other private clouds or public clouds over the Internet.

 The three resource managers in Figure 3.27 are specified below:

 • Instance Manager controls the execution, inspection, and terminating of VM instances on the host
where it runs.
 • Group Manager gathers information about and schedules VM execution on specific instance managers,
as well as manages virtual instance network.
 • Cloud Manager is the entry-point into the cloud for users and administrators. It queries node managers
for information about resources, makes scheduling decisions, and implements them by making
requests to group managers.

95